Need help with Steam Home Streaming in Ubuntu 18.04

[ReLink2013]

This is a slightly modified copy of a post I made on the Steam forums, I think it will get more attention here.

 

Hi all, I am running the following setup. (Skip down to "THE PROBLEM" if you don't want to read about my setup)

Hardware:
i7-7700k
16GB DDR4
Intel GPU Disabled in bios
PCI-E Slot 1 GeForce GTX 660TI
PCI-E Slot 2 GeForce GTX 980TI

Now I know, that looks weird but wait for the rest of the setup;

Main OS: Kubuntu 18.04 (Using custom ACS Override Kernel version 4.15.15)
Guest OS: Windows 10 Enterprise running in a KVM (Virtual Machine), with the 980TI pass-through to it.

Guest VM Specs:
4 Cores
6GB RAM
120GB VDisk on its own SSD

Basically the way I have things setup Windows 10 Runs in the background, I never even have to look at it unless I need to install something. Then I stream all my games through Steam, and run all my apps through RemoteApp using FreeRDP. Windows 10 has no idea its even running in a VM, and the 980TI is fully attached to the VM, to the point where I can even run MSI After burner and OC it.

I will start out by saying that I do not believe Windows 10 is the problem, everything in the VM seems to be running completely flawlessly. I am getting about 95% of the performance I would get on bare-metal, and Steam is reporting that it is in fact using hardware encoding when I am streaming.

THE PROBLEM:

The issue is that I cannot get Hardware accelerated decoding to work in Steam on the Linux Client. I currently have the 660TI dedicated for Linux to use, but I have also tried using a GTX 1050. I am running nvidia driver ver. 396.18, so I shouldn't have any issues. Yet every time I stream it says "Software decoding using 4 threads".

So please, can anyone shed some light on why I cant get Hardware decoding working? from everything I can see it should just work with the nvidia drivers installed.

Sorry for the long post, and thank you.

[limegorilla]

I may be wrong, but as your using the same CPU to code and decode the video it might be slightly confused. Could you not hook the Windows 10 Installation to a monitor / HDMI pass through and just switch to it when needed?

[jde3]

Pass through is a bitch man.. you also can't dedicate 4 cores to the guest if you only have 4 because each OS will believe they have full rights to that core and the threads will stomp all over each other. KVM will let you set core and thread count but.... the only way to keep the host OS off the guest's CPU's is to use cgroups.

 

To your actual problem.. Linux just may not support it (on that driver/card combo) even though the option is in steam.. try h264

[ReLink2013]

Quote

but as your using the same CPU to code and decode the video it might be slightly confused

The encoding is being handled 100% by the 980Ti.

Quote

Could you not hook the Windows 10 Installation to a monitor / HDMI pass through and just switch to it when needed?

Of course but that's neither challenging or fun. (at least to me)

Quote

you also can't dedicate 4 cores to the guest if you only have 4 because each OS will believe they have full rights to that core and the threads will stomp all over each other

I will definitely look into this, I hadn't heard that before. However I have had absolutely zero performance issues what so ever. Maybe I should have said threads instead of cores?

Quote

Linux just may not support it (on that driver/card combo) even though the option is in steam.. try h264

I can try switching back to the GTX 1050 instead of the 660Ti, however Valve states that you need the proprietary drivers, and a card that supports VDPAU, Either of the two cards I mentioned check both of those boxes. Also what do you mean "try h264"? Can you elaborate?

[ReLink2013]

Just to add some info, I launched steam from a terminal so I could see the output when the stream started. I noticed a few lines that grabbed my attention. Hope maybe they will be helpful.

 

ffmpeg verbose: Opened VA display via X11 display :0.
ffmpeg error: Failed to initialise VAAPI connection: -1 (unknown libva error).
CVAAPIAccel: av_hwdevice_ctx_create() failed
libavcodec software decoding with 4 threads

 

[rockerbacon]

A little late to the party, but I know the true source of your problem:

Ubuntu 18.04 ditched libva in favor of the new libva2, while Steam is still using libva.

Since libva is needed for hardware acceleration, you won't be able to use Steam Home Streaming with hardware acceleration on Ubuntu 18.04. Your options are:

• Use an older version of Ubuntu;
• Use another streaming program that has libva2 support (Parsec Gaming does);
• Wait for Steam to add libva2 support (which doesn't seem to be happening any time soon).

Facing the same problem here, I've been using Parsec and found it to be even better than Steam Home Streaming in many aspects.