Help with low latency desktop streaming on LAN

Banks4004 · December 9, 2019

Hi all! New to the forum but not to LTT (or finding tech answers via forum posts!). At a loss with this issue and wondering if anyone can help.

I am currently trying to stream my desktop to an Android unity game using this plugin: https://assetstore.unity.com/packages/tools/video/ump-android-ios-56044
But I cant get latency under about 4 seconds, even when playing the stream on the host PC! So most testing I'm doing is PC to PC. I would like to get the lowest possible latency even if quality suffers significantly.

Using ffmpeg for capture, encoding, and streaming:

.\ffmpeg -threads 8 -f gdigrab -framerate 60 -i desktop -c:v libx264 -refs 3 -g 60 -loglevel debug -pix_fmt yuv420p -slices 8 -an -preset ultrafast -tune zerolatency -f mpegts -listen 1 tcp://0.0.0.0:1234

The plugin supports native android media player and libVLC, both seem to have same latency. I have adjusted the resolution and framerate WAY lower without any change to latency, which makes me think its not encode/decode. Tried TCP and HTTP with similar latency (native android and/or this plugin doesn't support UDP). I've tuned the client network cache to what *seems* to be ideal (~400ms), but still way less that the latency I'm seeing. The latency doesn't seem to get any worse over time so maybe its the initial startup delay? I am totally unsure what to try from here.

If anyone notices a dumb mistake I'm making, has any ideas for trying different protocols, or ideas for hacking to get UDP to work I would love to hear them! Thanks all!