Boot from 2nd GPU

[amardilo]

I built an Unraid server using an AMD Ryzen 2600, AMD RX 580 GPU and an ASUS TUF B450M-PLUS motherboard (updated to latest BIOS). So far it's been pretty good but I often have issues with my GPU in VMs (mostly around restarts of VMs). I have a spare ASUS Nvidia GT 1030 (from another build) laying around. Could I use that spare GPU in the bottom PCIe slot and have the PC boot using that GT 1030 leaving my more powerful RX 580 free to be used by the VM?

 

Having a look online some motherboards might allow this via the System Agent Configuration in the UEFI/BIOS, but I can't seem to find it in the Advanced setup. Can I not do this with my setup or am I missing something?

[Alex Atkin UK]

I know when you have an iGPU you can choose between that or PCIe, not sure about choosing a specific PCIe GPU if you have several though.

You could try setting the AMD card for PCI passthrough, it might still boot from it but it would hopefully switch over during boot.  I'm not sure the specifics of doing this on Unraid but there are instructions for Fedora here that should give you an idea. https://level1techs.com/article/ryzen-gpu-passthrough-setup-guide-fedora-26-windows-gaming-linux

[Mick Naughty]

If you have two monitors, just assign that monitor connected to that card as primary. Or install the 1030 first and boot so any card after that won’t be primary. 

[Alex Atkin UK]

The problem doing it that was it is will only affect the GUI, the console will go onto whichever GPU it detects first I believe.  Also if you are going to be using a GPU exclusively in VMs, its better to use IOMMU passthrough anyway as its the absolute lowest overhead thus maximum performance, assuming the VM you use supports it.

They have also migrated Linux towards plug and play so it likes to auto-detect everything, which I suspect would switch the default GPU once you plug the second one in as again its probably going to use them in the order they are detected.  Trying to get it to "stick" to the layout you want can be a royal PITA.

[amardilo]

8 hours ago, Alex Atkin UK said:

The problem doing it that was it is will only affect the GUI, the console will go onto whichever GPU it detects first I believe.  Also if you are going to be using a GPU exclusively in VMs, its better to use IOMMU passthrough anyway as its the absolute lowest overhead thus maximum performance, assuming the VM you use supports it.

They have also migrated Linux towards plug and play so it likes to auto-detect everything, which I suspect would switch the default GPU once you plug the second one in as again its probably going to use them in the order they are detected.  Trying to get it to "stick" to the layout you want can be a royal PITA.

 

Yeah it's getting pretty difficult. I have it setup to use IOMMU passthrough but I can never seem to restart my VM (such as after an update or if I just want to shut it down so I can use the CPU for another VM) with the GPU passthrough setup with a single card. It just crashes and will not restart without rebooting Unraid.

 

I also tried to move my main card but it takes up 2 slots and ends up blocking things like the USB 3.0 headers at the bottom of the motherboard.

 

What I was hoping to do was to give 1 VM full access to the RX 580 and have Unraid and whatever else use the spare GPU. Looks like I probably don't have the right hardware to do that.

[Alex Atkin UK]

I've read of that problem, I think its AMD specific (I could be wrong) but didn't check the workarounds due to being on nVidia.

Its so much easier to do with an iGPU due to the BIOS letting you choose.  The only wall I ran into was latency issues due to not being willing to block off CPU cores as 99% of the time I'm not running a VM and I do some video encoding, so losing cores would be a waste. Games will stutter more in a VM than native, but its useful for lighter games rather than having to always reboot into Windows.  Plus being able to test things while still in Linux, such as when helping out on here.

I was surprised that even switching from a GTX 1070 Ti to an RTX 2080, apart from having to assign the card as the PCI ID had changed, it just worked.  I'm still waiting for them to block the workaround that lets you use a desktop card in a VM though, with nVidia you just never know what they will do next.  But I didn't buy this setup to use a VM at all, it was just a bonus when I discovered how to do it.