GPU Twins! Plans to local LLM much? :)
Discussion
Feeling cute might load gpt-oss up later :)
Id like to run vgpu and hopefully share that gpu compute around if I can manage it.
Let me know how you experiments go! If this a datacenter GPU, or maybe older consumer grade one? (NVIDIA locks the new consumer stuff so hard the even simple passthrough has been sorta painful)
Will do! They are p100s. So they should be DC, not sure about firmware though if that's what you are suggesting. I had Titan X maxwells installed and just swapped these in.
I've seen PopOS! dealing with multiple cards (rtx in this case) and sharing resources nicely. So I think this is built-in in (proprietary) drivers.
The problem isn't having multiple cards. NVIDIA makes it hard to use "enterprise" features like virtualisation, passthrough, etc. It’s all doable AFAIK, but even with AMD hardware, getting proper "hot" passthrough from a Fedora host to a Windows guest and back was somewhat painful (Wayland itself makes it tricky since it really clings to the GPU and doesn’t want to let it go :)).
oh, yes. passing through video cards to vms is definetely not fun.
I use poman to containerize these things. podman-desktop has even AI extension which makes it a breeze ...