Not sure which OS to use for Windows, Linux, and MacOS

happymarshmellow · March 8, 2022

Hello,

I'm building a Homelab server with the following parts: Ryzen 7 1700, 32 GB RAM, RTX 3060, R9 290, 4TB * 2 (raid 1), 1TB * 2 (raid 1), 240 GB SSD (boot drive), and a 850 watt psu from CoolerMaster. I'm trying to find an OS for my that I can create virtual machines for Windows 10, Windows 11, Linux (ubuntu), and MacOS (Monterey). I want to find an OS where I can play anti cheat games (like Lost Ark and Valorant). Reason is because I have a roommate and she has low end hardware and an old laptop thats not capable of playing games at a bearable frame rate. So instead of her buying parts, I figure I just share my hardware. The second hardest requirement is sharing the 3060 with the host, and the two windows virtual machine. Ideally I would also want to split the R9 290 gpu with my linux and macos vm, but if I can only passthrough to one vm I would choose my linux vm.

I have experimented with Proxmox, but it detected anti cheat on the windows virtual machines. And it didn't have the split gpu resources feature that I was looking for. I also experimented with using Ubuntu server, but I've read that Lost Ark isn't compatible with Linux. My third attempt is using Windows 10 Pro with Hyper - V. I was able to split my 3060 among my guest and windows vms via easy-gpu-pv. But I was not sure how to pass through my amd graphics card to 1-2 virtual machines and create a macos vm.

Any and all ideas are appreciated!

RONOTHAN## · March 8, 2022

If you want to split the 3060, your only option is Hyper-V. If you want to run a Mac OS VM or pass a GPU through to a Linux VM, your only option is Linux with KVM. It's a fundamental restriction for how they work. GPU-P works by sharing the driver between the host and the VM, so it's not possible to split the GPU if it would use different drivers as it would on Linux and Mac OS. Also don't believe anyone has successfully managed to virtualize Mac OS on Hyper-V since it isn't really that robust of a hypervisor. KVM (the backend for Proxmox and most other Linux based virtualization) is a lot more robust so it can handle GPU passthrough, running a Mac OS VM, and a lot more features, but it doesn't support sharing GPUs that aren't designed to be shared. There is a way to split the card if you have a 900 series card through 2000 series card using the vGPU unlock script, but it's currently not supported on 30 series so you pretty much can't unless you can figure out how to hack the vBIOS on 30 series cards.

The only real option I currently see is to maybe do nested virtualization. Run ProxMox on the host machine and run the Linux VM, the MacOS VM, and a single Windows VM. For that Windows VM, you will have to enable nested virtualization (I forget how to set that up, this is a very niche setup, I just know it's possible) and run the easy-gpu-pv script inside that VM. There are also some ways you can get anti-cheat to work in that Windows VM by passing things like device IDs through, but again, I forget how to set that up and there are a number of tutorials out there for how to get that working, but it's not always possible.

Tl;Dr: It's on the edge of possible, but it would take a lot of work to get this to work, you honestly might need more cores, and anti cheat might be possible to get working, but just barely.