The problem isn't having multiple cards. NVIDIA makes it hard to use "enterprise" features like virtualisation, passthrough, etc. It’s all doable AFAIK, but even with AMD hardware, getting proper "hot" passthrough from a Fedora host to a Windows guest and back was somewhat painful (Wayland itself makes it tricky since it really clings to the GPU and doesn’t want to let it go :)).
Discussion
oh, yes. passing through video cards to vms is definetely not fun.
I use poman to containerize these things. podman-desktop has even AI extension which makes it a breeze ...