AMD's EPYC CPUs Push Netflix Server Bandwidth To 400 Gbps

[hysel]

Summary

Netflix has been serving up to 200 Gbps of TLS-encrypted video from a single server since 2020. Nonetheless, the company aims to double the bandwidth to 400 Gbps

 

Quotes

Quote

Netflix has been serving up to 200 Gbps of TLS-encrypted video from a single server since 2020. Nonetheless, the company aims to double the bandwidth to 400 Gbps. During his presentation at the EuroBSD 2021 conference (via HardwareLuxx), Andrew Gallatin, Senior Software Engineer at Netflix, detailed the challenges of pushing the bandwidth envelope on its FreeBSD-based servers.

 

Netflix turned to AMD's EPYC Rome processors to achieve its goal. The company equipped its server with the EPYC 7502P, which wields 32 Zen 2 cores with a 2.5 GHz base clock and 3.35 GHz boost clock. More importantly, the 32-core beast offers up to 128 PCIe 4.0 lanes, good for about 250 GBps of bandwidth or around 2 Tbps in networking units. Netflix paired the EPYC 7502P with 256GB of DDR4-3200 memory, with a total memory bandwidth of up to 150 GBps, or 1.2 Tbps in networking units.

 

For storage, Netflix's AMD-powered server utilizes 18 Western Digital WD SN720 2TB NVMe SSDs. It's also equipped with a pair of Nvidia's Mellanox ConnectX-6 Dx network adapters that communicate through a PCIe 4.0 x16 interface. Initially, Netflix was only getting 240 Gbps out of the server, primarily due to the limitation on the memory.

 

 

Netflix experimented with different NUMA (Non Uniform Memory Architecture) configurations to maximize the bandwidth. AMD's EPYC processors support different NUMA nodes per socket, which can either be 1, 2 or 4. Naturally, the processor dictates which modes are available or not. The EPYC 7502P, which is the SKU used in Netflix's server, supports all three NUMA modes. According to Gallatin's slide, a single NUMA node configuration delivers up to 240 Gbps, while a setup with four NUMA nodes bumps the value up to 280 Gbps.

In an attempt to optimize the performance and avoid hardware bottlenecks, Netflix tested offloading the TLS encryption to the Mellanox ConnectX-6 Dx, instead of the EPYC 7502P. With a bit of tinkering with the software and some firmware updates, Netflix managed to squeeze 190 Gbps per Mellanox ConnectX-6 Dx adapter or 380 Gbps with two network adapters. The encryption no longer passes through the processor, so it helps free up resources and cuts memory bandwidth by half. The results showed 50% processor utilization, with four NUMA nodes and around 60% without NUMA.

 

Netflix evaluated other processor options from Intel and Ampere, but AMD was clearly the superior option. For example, the EPYC 7502P offered 280 Gbps, while the Xeon Platinum 8352V (Ice Lake) and Altra Q80-30 delivered 230 Gbps and 180 Gbps, respectively. 

The memory was the bottlenecked on the Intel system, since the Xeon Platinum 8352V natively supports DDR4-2933 as opposed to the EPYC 7502P's DDR4-3200 support. Gallatin expects similar performance with the EPYC 7502P if the Ice Lake chip is paired with equivalent memory. While the Altra Q80-30 from Ampere does support DDR4-3200 memory, the chip is limited to 180 Gbps.

 

Nevertheless, the Altra Q80-30 was the closest competitor to the EPYC 7502P with the TLS offload. The system offered 240 Gbps, but Gallatin noted low processor utilization and many output drops, which could be a PCIe-specific problem. After enabling extended tags, the Altra Q80-30 system pumped out 320 Gbps, just 60 Gbps lower than the EPYC 7502P system. Apparently, the Xeon Platinum 8352V system had the PCIe relaxed ordering option locked out, so Gallatin wasn't able to assess the performance of the network adapter.

 

 

My thoughts

Another example how new technologies assist companies to increase their offering without increasing their server footprint. The TLS encryption part using the Mellanox ConnectX-6Dx was especially brilliant.

How about an LTT video trying to mimic their setup?

 

Sources

https://www.tomshardware.com/news/amd-epyc-cpus-netflix-bandwidth-400-gbps-per-server

 

[Quackers101]

DDR5 with ECC

but yes, any error is going to be costly. Get that ECC protection on you.

[ThreePinkApples]

The slides also mentioned an 800Gbps prototype that they would share more about "next year". Possibly an early sample of Zen 4 Epyc with DDR5?

[hysel]

2 hours ago, aerandir92 said:

The slides also mentioned an 800Gbps prototype that they would share more about "next year". Possibly be early sample of Zen 4 Epyc with DDR5?

And to think my old Dell r720 based NAS is graded for 1.5Mbps and it working fine

[igormp]

It's nice to see how they keep benchmarking and pushing the current hardware to its limits. They have done something similar 2 years ago too: https://www.phoronix.com/scan.php?page=news_item&px=Netflix-NUMA-FreeBSD-Optimized

[leadeater]

5 hours ago, Mr-G-Man said:

being a million dollar company, couldn't they afford 3600mhz ram?

Servers only run official JDEC ram speeds and the memory controller in AMD EPYC second gen is rated for 3200 so the fastest any ram will operate with those CPUs is 3200.

 

Wouldn't matter if 5200 ECC RDIMM existed, would still only run at 3200.

[Lightwreather]

So wait, netflix uses BSD?

Nice

[GDRRiley]

zen loves memory speed. across the line DDR5 will help zen. EPYC needs more memory bandwidth given their PICE amount

 

this sounds like the gear they use for deployment in IPSs

[StDragon]

"In an attempt to optimize the performance and avoid hardware bottlenecks, Netflix tested offloading the TLS encryption to the Mellanox ConnectX-6 Dx, instead of the EPYC 7502P. With a bit of tinkering with the software and some firmware updates, Netflix managed to squeeze 190 Gbps per Mellanox ConnectX-6 Dx adapter or 380 Gbps with two network adapters. The encryption no longer passes through the processor, so it helps free up resources and cuts memory bandwidth by half. The results showed 50% processor utilization, with four NUMA nodes and around 60% without NUMA."

 

Interesting. For a moment there, I was surprised AES-NI instruction set could keep up. Well that answers that question

[williamcll]

But would this increase my monthly costs?

[leadeater]

8 minutes ago, williamcll said:

But would this increase my monthly costs?

Should be lower as they would need half the servers for the same load. I assume though load year over year increases but not double. Overall should result in lower TCO for quite a number of years.

[WolframaticAlpha]

BSD? I would be surprised if they gave back these changes.

[igormp]

1 hour ago, WolframaticAlpha said:

BSD? I would be surprised if they gave back these changes.

*ahem*