Encoding and streaming hardware/environment

Evg · May 24, 2020

Hi all,

This topic is related to different kinds of hardware.

My company develops FPGA acceleration boards. These boards are connected via PCIe and are used to accelerate different applications: from Neural networks and Network to Video Encoding. Our technology is new and has performance advantage over existing solutions. We consider to make hardware encoder solution. I don't have a lot of experience in this so I need to understand current situation and niches we can suit: from consumer solutions to business. My experience is only several streams with OBS.

I'd like to discuss Nvidia encoding, devices like Elgato and analogues, AWS Elemental Link, what are the parameters that affect the price and use cases (church/school/business_conference/gaming/event_multi_cameras streaming). Besides such on-premise solutions there might be server solutions (as PCIe boards) to encode many sources simultaneously at CDN side. We have possibility to use more powerful FPGA chips to suit required level of performance.

Our boards are accelerators and they accelerate snippets of code where execution might be highly parallelized, the other part of code is still running on CPU. So we depend on PC and board has to be connected to device with CPU (PC/Laptop). So one of the solutions is small device connected to Laptop via Thunderbolt, another is just usual PCIe board for PC. For server it's also PCIe board.

Supported codecs are defined only by software and different formats can have support over time. Considered formats are from AV1/HEVC to h.264 with 4k60 and lower quality.

We're looking for encoding software partners to adapt to our boards.

Regarding Nvidia I've seen that they have 2 encoders per card. Does this mean that 2 medias could be encoded in parallel? Do gamers need 2nd GPU or auxiliary hardware to stream with high quality? Are Elgato like devices still relevant?

The topic is quite wide and I appreciate everyone's opinion. DM me if you wish to talk

LogicalDrm · May 24, 2020

4 hours ago, Evg said:

Regarding Nvidia I've seen that they have 2 encoders per card. Does this mean that 2 medias could be encoded in parallel? Do gamers need 2nd GPU or auxiliary hardware to stream with high quality? Are Elgato like devices still relevant?

2 encoders means that one is encoding what is produced on screen and the other can be used to encode outgoing video stream. So end user only needs one GPU. Same could be done with Intels iGPU, but thats seems to be dropped out of Intels interests of deveploment.

Capture cards are still relevant for those who want to have something more than just webcam. Or have need to capture from consoles or secondary PCs. Since capture cards don't do encoding, one will still need hardware power for encoding video.

igormp · June 11, 2020

I know that I'm kinda of necrobumping this thread, but it's a nice topic and I wanted to correct some information here.

On 5/24/2020 at 8:53 AM, Evg said:

Regarding Nvidia I've seen that they have 2 encoders per card. Does this mean that 2 medias could be encoded in parallel? Do gamers need 2nd GPU or auxiliary hardware to stream with high quality? Are Elgato like devices still relevant?

You can encode as much media in parallel as you want, just keep in mind that the speed of each encode might slow down after the encoder hits 100% usage. Also, on geforce cards nvidia places an artificial limit of 3 encodes per card (you can mod the driver to get rid of that).

With high quality? Usually you need another PC or a powerful CPU since the hardware encoders in GPUs are usually meant for speed rather than quality. If you have a really powerful CPU or even another PC dedicated just to encode your stream in high quality, great.

If you don't mind about quality that much, or just want something good enough for a casual twitch stream, the latest NVENC has quality on par with ffmpeg's fast~medium preset. Older generation's nvenc, AMD's VCE/VCN and Intel's Quick Sync should be able to do the job with the most famous codecs with a not so shit quality.

Capture cards are important if you want to get the output of another device (such as a console or actual gaming PC) as LogicalDrm mentioned.

If you were thinking about making a product for the gamer market, I guess it'd be hard since there are sub $20 USB capture cards that can do 1080p30fps, and sub $100 ones that can do 60fps, and most of those people will already have a GPU or a dedicated PC for such encoding tasks.

A nice market for that kind of thing is actually in places that give remote classes, such as some universities. Usually those buy dedicated hardware encoders (such as the ones from epiphan and teradek) that can receive and video input (hdmi or whatever), and output a RTMP stream for a live show or something. Some can even handle HLS/DASH outputs nowadays. Building a SBC with an integrated FPGA to offload the encoding part would be an interesting thing to do.

As for server side stuff, I guess most just go with CPU-based encoding. At least twitch does. I guess that an fpga-based solution wouldn't be easy to scale nor to tune, along with latency problems and whatnot.

On 5/24/2020 at 2:11 PM, LogicalDrm said:

2 encoders means that one is encoding what is produced on screen and the other can be used to encode outgoing video stream. So end user only needs one GPU. Same could be done with Intels iGPU, but thats seems to be dropped out of Intels interests of deveploment.

That's wrong. Current Turing GPUs have only a single encoder but are still able to encode as much data as you want (ignoring nvidia's artificial cap). Intel's Quick Sync is still going strong and has been updated for the latest codecs in each new gen.