GPU for multiple VMs

[BotDamian]

Hello,

I want to run multiple VMs and I wonder which GPU to get.

2x W6800 32GB

4x RX6800 16GB

6x RTX 2060 12GB

 

The 2060 12GB are cheaper than the others but more GPUs, more PCIe Slots needed. Also is there a way to split x16 into x8 or even x4 slots so I can use more GPUs for my VMs?

[Electronics Wizardy]

What are these vms doing? 

 

What hypervisor are you using? Do you want to share these gpus amount multiple vms?

 

6 minutes ago, BotDamian said:

Also is there a way to split x16 into x8 or even x4 slots so I can use more GPUs for my VMs?

Whats the rest of the system. Sometimes you can get splitters, but Id just get multiple hosts at that point.

[BotDamian]

2 minutes ago, Electronics Wizardy said:

What are these vms doing? 

 

What hypervisor are you using? Do you want to share these gpus amount multiple vms?

 

Whats the rest of the system. Sometimes you can get splitters, but Id just get multiple hosts at that point.

Running Games that require a client to be open like for example Minecraft but you need to be there in other for it to work, if not they the server will close as soon as the last player leaves. But the GPU can't be too slow or the game with also crash and game will close.

So a performance of a GT1030 will be more than enough, even half of that is enough.

 

Hypervisor would be HyperV but if needed i can do Proxmox.

Yes each GPU needs to be split into multiple VirtualGPUs for each VM

 

This Mobo Supermicro H11DSi R2.0 and Epyc CPUs.

[Electronics Wizardy]

1 minute ago, BotDamian said:

Running Games that require a client to be open like for example Minecraft but you need to be there in other for it to work, if not they the server will close as soon as the last player leaves. But the GPU can't be too slow or the game with also crash and game will close.

So a performance of a GT1030 will be more than enough, even half of that is enough.

 

Hypervisor would be HyperV but if needed i can do Proxmox.

Yes each GPU needs to be split into multiple VirtualGPUs for each VM

 

This Mobo Supermicro H11DSi R2.0 and Epyc CPUs.

DO you need to use vms here? Id personally just use aster and then you can run all these games shared on one gpu easily. Then just connect a lot of virtul displays and your good.

 

Hyper-v should work well here. Or you can try vmware workstations, should have enough gpu power to get games running and easily share between systems.

 

Probably get the 6800s here for the extra vram.

[danalog]

I would say go with the 2 Radeon Pro cards

[Windows7ge]

1 hour ago, BotDamian said:

Also is there a way to split x16 into x8 or even x4 slots so I can use more GPUs for my VMs?

Physically & virtually yes. It's called Bi-Furcation and is definitely supported by the motherboard.

 

The problem is this would require a convoluted series of riser cards & PCI_e extension cables and somewhere to mount the GPU's since they wouldn't all fit in a normal chassis anymore. You'd basically have to buy a BTC miner GPU riser bench and run all the cables to the various cards. Wouldn't be pretty. Wouldn't be practical to move around. Overall inconvenient.

[BotDamian]

1 hour ago, Windows7ge said:

Physically & virtually yes. It's called Bi-Furcation and is definitely supported by the motherboard.

 

The problem is this would require a convoluted series of riser cards & PCI_e extension cables and somewhere to mount the GPU's since they wouldn't all fit in a normal chassis anymore. You'd basically have to buy a BTC miner GPU riser bench and run all the cables to the various cards. Wouldn't be pretty. Wouldn't be practical to move around. Overall inconvenient.

Unless you have a friend that has a huge CNC and tons of Acrylic/Plexi plates  All I need to do is to give him a CNC file and he has no problem throwing one plate into the machine and CNC it.

[Windows7ge]

56 minutes ago, BotDamian said:

Unless you have a friend that has a huge CNC and tons of Acrylic/Plexi plates  All I need to do is to give him a CNC file and he has no problem throwing one plate into the machine and CNC it.

If you have the means to get it done you can go for it. Bifurcation is presented in the BIOS differently depending on the manufacturer and I remember on Supermicro boards it's rather notoriously hidden in non-descriptive menus. So you'll have to do some serious digging.

 

With more modern, particularly high-end GPU's I wouldn't go lower than x8x8 but that should be plenty adequate. If you need to split them more than that you're better off investigating vGPU and just running multiple users off one GPU.

[BotDamian]

4 hours ago, Windows7ge said:

If you have the means to get it done you can go for it. Bifurcation is presented in the BIOS differently depending on the manufacturer and I remember on Supermicro boards it's rather notoriously hidden in non-descriptive menus. So you'll have to do some serious digging.

 

With more modern, particularly high-end GPU's I wouldn't go lower than x8x8 but that should be plenty adequate. If you need to split them more than that you're better off investigating vGPU and just running multiple users off one GPU.

vGPU was my plan and HyperV makes all of this really simple to do.

[BotDamian]

I have also another problem I haven't thought about, when running many clients, lets say 64 clients. I need some storage for all of them. The Mainboard has 8 Sata slots, and 1 M.2, i was thinking about getting some 2TB SSDs and using a m.2 as a cache. I'm not sure how big the NVME needs to be to run so many clients. Because if I use 4x 2TB SSDs it will run 16 users on 1 ssd, so the 500mb/s transfer will be split across all users assuming all need to access something so it's 32mb/s for each client. When using a 1TB NVME it would result with 15.6GB cache for each client and that's not a lot
So I would rather dump the whole NVME Cache idea, because it would make more sense to use more SSDs to increase the bandwidth. So 8x 1TB SSDs will run 8 clients on 1 ssd giving each client 64mb/s.

The VMs HyperV creates "disk image" files on the ssd, I could create a partition and put ONLY the disk image there, so it would be split evenly without worries that one SSD has more usage than another one.

Unless there is a better way to deal with this, I'm new to this whole VM thing, so I'm learning.

[BotDamian]

On 2/12/2022 at 5:33 PM, Windows7ge said:

Physically & virtually yes. It's called Bi-Furcation and is definitely supported by the motherboard.

 

The problem is this would require a convoluted series of riser cards & PCI_e extension cables and somewhere to mount the GPU's since they wouldn't all fit in a normal chassis anymore. You'd basically have to buy a BTC miner GPU riser bench and run all the cables to the various cards. Wouldn't be pretty. Wouldn't be practical to move around. Overall inconvenient.

I did some research but not really anything helpful, so the Supermicro has also PCIe x8 slots, I could actually use a extension cable and use a pciex8 to pciex16 adapter to plugin more GPUs? But wouldn't that mean that the GPU would be much slower because of the split GPU for VMs? If it's 5% across whole GPU on x8 vs x16 then i don't care at all. 

[Windows7ge]

3 hours ago, BotDamian said:

I did some research but not really anything helpful, so the Supermicro has also PCIe x8 slots, I could actually use a extension cable and use a pciex8 to pciex16 adapter to plugin more GPUs? But wouldn't that mean that the GPU would be much slower because of the split GPU for VMs? If it's 5% across whole GPU on x8 vs x16 then i don't care at all. 

I cannot say definitively. Traditionally in the past x8 was more than enough but with more modern GPUs needing more bandwidth I don't know if your cards would hit the cap. You'd have to pop one in a try it in both x8 & x16 slots to measure if there's a discernible difference.