GPU pass-though on linux??

[Super Tech Mike]

So first off, I have an RX 480 I run in my rig, I have a couple of quite old Nvidia and Amd cards lying around but non of them are really capable of "gaming". I am currently running windows, but I would really like to switch to linux (Manjaro specifically), thing is I would have to run windows in a virtual machine to use most of my software. I'm ok with doing that, except, things like games require a good gpu for the vm. So is it possible to have 2 gpus installed in my system on good one and a weaker one, and allocate the good one to my vm when its running, using the other one for the host system, and then when the vm is off, use the good gpu? All help is appreciated, sorry if that was a bit rambling. 

[Electronics Wizardy]

Whats your full system? You can do this pretty easily if you know your way around linux.

[BobVonBob]

Yes, although it's been a while and I don't remember if it will require a reboot. Check out this series from Level1Techs

 

 

Edit: Just checked and that series isn't actually finished. Whoops. Anyway, look up "linux vm graphics card passthrough" and you should find some stuff.

[Super Tech Mike]

 

4 minutes ago, Electronics Wizardy said:

Whats your full system? You can do this pretty easily if you know your way around linux.

I have a ryzen 1700, 32gb of ram and a rx 480

[Electronics Wizardy]

1 minute ago, Super Tech Mike said:

 

I have a ryzen 1700, 32gb of ram and a rx 480

What board? That won't make it easier.

 

Id probably just do a dual boot here if you want simple. Rzyen, esp first gen had some issues that make it more of a pain.

[Super Tech Mike]

https://www.asus.com/us/Motherboards/ROG-CROSSHAIR-VI-HERO-WI-FI-AC/ This is my board

[Electronics Wizardy]

Just now, Super Tech Mike said:

https://www.asus.com/us/Motherboards/ROG-CROSSHAIR-VI-HERO-WI-FI-AC/ This is my board

Go boot into linux and see what the iommu groups are. Id also put a second gpu in the second slot for this test.

[Sauron]

15 minutes ago, Super Tech Mike said:

is it possible to have 2 gpus installed in my system on good one and a weaker one, and allocate the good one to my vm when its running, using the other one for the host system,

Yes.

16 minutes ago, Super Tech Mike said:

and then when the vm is off, use the good gpu?

No. As far as I know this isn't possible. You can still use the "good" gpu for, say, rendering, but not to display your desktop. @Electronics Wizardy do you know something I don't?

[Super Tech Mike]

Do I need to install linux to test it, or can I do a live boot? Also, what would I look for and where?

[Electronics Wizardy]

Just now, Sauron said:

o. As far as I know this isn't possible. You can still use the "good" gpu for, say, rendering, but not to display your desktop. @Electronics Wizardy do you know something I don't? 

you can do this, its a pain though, tell x to stop using the gpu, give it to the vm, and viseversa.

[manikyath]

12 minutes ago, Super Tech Mike said:

windows in a virtual machine to use most of my software

from experience.. you'll be back on windows in no time..

if the majority of your software is on windows, you'll just end up in the virtual machine constantly.

it's a pain to set up, you'll need to shove more hardware into the same box to make it tick, you're working with piles of really specific software and configurations,  and so on..

 

also, manjaro linux is ghastly to use on a daily basis.. i did... and i went straight back to ubuntu derivatives because a church bench is more comfortabnle than an ant's nest.

 

if you're not doing this for the sake of being a linux enthousiast (as in, doing it more for the fun of setting it up than the actual usability) you may as well just dont.

[Electronics Wizardy]

Just now, Super Tech Mike said:

Do I need to install linux to test it, or can I do a live boot? Also, what would I look for and where?

live boot will be fine.

 

Look at the iommu groups(google it if you want exact commands0 to see if the second gpu is in a group with nothing else

 

[manikyath]

1 minute ago, Super Tech Mike said:

or can I do a live boot?

sorry man.. gonna be blatant here..

 

if you're not about ready to nuke your hard drive for science, this isnt the project for you to undertake.

[Super Tech Mike]

1 minute ago, manikyath said:

sorry man.. gonna be blatant here..

 

if you're not about ready to nuke your hard drive for science, this isnt the project for you to undertake.

Trust me, ive nuked my hardware quite a few times already. Im just trying to go into a bit more informed this time

[aezakmi]

Quote

 would really like to switch to linux (Manjaro specifically), thing is I would have to run windows in a virtual machine to use most of my software

what's the point of switching then?

 

The whole idea of Linux is that you use native programs, it'd be like buying a $3000 Mac just to use bootcamp

 

some games can run on linux but it's not an OS for gaming, it's more for workstation stuff, server management, etc

[datboi8192]

After watching the level 1 tech video on this I actually have been running Ubuntu some of the time on another drive. 

I was unlucky with the pass though though because my cpu/mobo combo treated all my pci slots as one single IOMMU group; thus passing it though would pass all of them though

 

I am able to play most of my steam games just fine, and lutris does work nicely most of the time for non steam games.

Windows is still my choice since the games are better and WSL runs most of my linux needs just fine.

[Wild Penquin]

Disclaimer: never used VMs (not had a strong enough use case to start learning them).

 

I'll point out that while theoretically* possible to detach a GPU from a running software (X.org) and move it to another (the VM in this case), it's not just the VM configuration (and having X.org let go of the GPU); IIRC It's been described that the GPUs need to boot, too, and are running their own microcode (or something). If you do not power them down (i.e. reboot) you may expect problems because of this, too.

 

Seems like quite a daunting thing to do, IMO. But if you, the OP, have a really strong use case for this situation ( 1) using Linux is mandatory, 2) dual-booting is not an option, 3) NAtive Linux games, which are getting more numeruous every day, are not enough) ... well, it might be worth the hassle, but there might definitely bee unseen roadblock on the way.

 

*) I'd be interested how that is done, though. Some kind of custom Bumblebee? Probably more practical to just restart the X.org, but that almost equals a reboot on a desktop computer.

[Guest]

6 hours ago, Wild Penquin said:

Disclaimer: never used VMs (not had a strong enough use case to start learning them).

 

I'll point out that while theoretically* possible to detach a GPU from a running software (X.org) and move it to another (the VM in this case), it's not just the VM configuration (and having X.org let go of the GPU); IIRC It's been described that the GPUs need to boot, too, and are running their own microcode (or something). If you do not power them down (i.e. reboot) you may expect problems because of this, too.

 

Seems like quite a daunting thing to do, IMO. But if you, the OP, have a really strong use case for this situation ( 1) using Linux is mandatory, 2) dual-booting is not an option, 3) NAtive Linux games, which are getting more numeruous every day, are not enough) ... well, it might be worth the hassle, but there might definitely bee unseen roadblock on the way.

 

*) I'd be interested how that is done, though. Some kind of custom Bumblebee? Probably more practical to just restart the X.org, but that almost equals a reboot on a desktop computer.

 

X.org Is just not technically capable of hotswapping graphic driver or GPU like windows, this will require all the X clients to shut down, there is no solution to this...

I don't know about Wayland but... It doesn't even support NVIDIA drivers

 

Bumblebbee doesn't switch driver, just runs a separate X.org fake server on another GPU driver, there is no hotswapping there, is just running on another driver and showing on another server so yes

 

the only solution is to make a script which automatically unbind vfio drivers and run the GPU driver, then restart X.org again

 

 

Edited February 3, 2019 by Guest

[wasab]

On 2/2/2019 at 3:28 PM, aezakmi said:

what's the point of switching then?

 

The whole idea of Linux is that you use native programs, it'd be like buying a $3000 Mac just to use bootcamp

 

some games can run on linux but it's not an OS for gaming, it's more for workstation stuff, server management, etc

I game on it perfectly fine. 

[aezakmi]

22 minutes ago, wasab said:

I game on it perfectly fine. 

fallout 4 can't run

hitman 2 can't run

not even the witcher 3 runs properly even with decent hardware

league of legends doesn't work 

battlefield 3 or 4 can't run because EA's origin is only for windows

 

and so on

those are the games I play and none of them runs on linux

[wasab]

29 minutes ago, aezakmi said:

fallout 4 can't run

hitman 2 can't run

not even the witcher 3 runs properly even with decent hardware

league of legends doesn't work 

battlefield 3 or 4 can't run because EA's origin is only for windows

 

and so on

those are the games I play and none of them runs on linux

The only games I play are Star trek online and company of heroes 2. The first which ran terrific through wine and 2nd has a native Linux port by feral. 

 

I would find other games if these tripple As aren't available. To me, games are never the reason for me to use certain hardwares or ecosystem, except maybe Pokemon. I am tempted to get a Nintendo switch everyday just for Pokemon... 

 

[Guest]

1 hour ago, aezakmi said:

fallout 4 can't run

hitman 2 can't run

not even the witcher 3 runs properly even with decent hardware

league of legends doesn't work 

battlefield 3 or 4 can't run because EA's origin is only for windows

 

and so on

those are the games I play and none of them runs on linux

I heard fallout 4 runs, and league of legends too since I played that months ago with no issues, as for battlefield the 3 doesn't just because of their stupid browser, but the single player worked fine for me and with decent performance, as for the Witcher 3 I read somewhere it has issues with Mesa drivers (Intel and Radeon cards) so you should be sure to update the drivers through repos in Ubuntu

let's say dxvk is still alpha software but I played nicely with it, it's everything around that made me quit gaming on Linux, but I suppose things will improve in the future

But an fps decrease (like 10,20) it normal for the API transition overhead at this point of dxvk production, still we should look at the future results

 

43 minutes ago, wasab said:

The only games I play are Star trek online and company of heroes 2. The first which ran terrific through wine and 2nd has a native Linux port by feral. 

 

I would find other games if these tripple As aren't available,.

 

there are a lot of other triple A titles working properly as well thanks to proton...

 

[Wild Penquin]

6 hours ago, Lukyp said:

X.org Is just not technically capable of hotswapping graphic driver or GPU like windows, this will require all the X clients to shut down, there is no solution to this...

I don't know about Wayland but... It doesn't even support NVIDIA drivers

 

Bumblebbee doesn't switch driver, just runs a separate X.org fake server on another GPU driver, there is no hotswapping there, is just running on another driver and showing on another server so yes

 

the only solution is to make a script which automatically unbind vfio drivers and run the GPU driver, then restart X.org again

  

 

FWIW, I noticed in a quick Google search earlier (after my post) that someone has made something like this with bumblebee.

 

Yes, X.Org can not hotswap a GPU. But if I've understod correctly, this can (and has been?) workaround somewhat like this: run main X.Org does not actually see the main GPU (at all) but a framebuffer (on the secondary GPU). A low-latency Looking Glass is used to copy the rendered content from the VM into the running X.Org. If the VM is shut down, a similar instance can be done on a natively running (separate) framebuffer (running native Linux X.Org applications). So, it is kind of nested X.Org instances???

 

Actually, I have no idea, just found this youtube clip with some explanation and some links to some scripts... but seems it is doable? Use your own judgement

 

EDIT: Looks like there are latency / input lag issues when running in a VM this way, judging from the comments in youtube

[Guest]

8 hours ago, Wild Penquin said:

FWIW, I noticed in a quick Google search earlier (after my post) that someone has made something like this with bumblebee.

 

Yes, X.Org can not hotswap a GPU. But if I've understod correctly, this can (and has been?) workaround somewhat like this: run main X.Org does not actually see the main GPU (at all) but a framebuffer (on the secondary GPU). A low-latency Looking Glass is used to copy the rendered content from the VM into the running X.Org. If the VM is shut down, a similar instance can be done on a natively running (separate) framebuffer (running native Linux X.Org applications). So, it is kind of nested X.Org instances???

 

Actually, I have no idea, just found this youtube clip with some explanation and some links to some scripts... but seems it is doable? Use your own judgement

 

EDIT: Looks like there are latency / input lag issues when running in a VM this way, judging from the comments in youtube

He is just using the intel iGPU as a primary GPU, and then running games separately in a different fake no-output X.org server after the vm shut down (unbinding the vfio driver from the GPU and attaching the nvidia one), then the frames are copied to the intel iGPU, as I said that's how bumblebee works, but it doesn't even support Vulkan (required for DXVK) unless you use nvidia-prime (which requires X.org shutdown) so not really the best choice anyway, and it's pointless since it's an actual physical GPU with video output and using prime with the intel is just useless unless you are lazy to switch off the video cable
He is basically emulating a laptop with an optimus GPU like environment in that way

The input lag is just Looking-glass related (if you don't want to use the GPU output), which today should be something different as it is an alpha software, or you could just be using the video output of the GPU

Edited February 4, 2019 by Guest