10Gbe SMB Windows slow on Threadripper Gen 1+2

[havok2]

Hello dear community,
We have a full 10Gbe network in the office for our video production. Unfortunately I noticed that our systems with Threadripper 1950x as well as 2990wx only manage ~300 MB/s max in reading (writing is double @~700)+. Our NAS is a Synology FS3400 with 18x4TB SSDs. Connected to a Netgear M4300 x24-f24 via 4x10Gbe LACP. Our Windows Server 2016 with 4x10Gbe LACP is the only one that can nearly max out the full 10Gbe in both directions at CrystalDiskMark. Our Threaripper 3990x and all Intel systems also manage ~700mb here. Only the "older" Threadrippers do not. With iperf I don't get over 6Gbit/s in both directions (more than one thread). The bottleneck when reading is about the same as the performance with iperf with only one thread on the loopback. Could it have to do with the fact that the Aquantia agc107 which is installed onboard or also as a plug-in card in the form of an Asus XGC-100C only has 4x PCIe Gen 2.0? All PCs use this card, but the TR 1+2 Gen have deficits with it. I have now ordered a Mellanox Conect X 4 with RJ45 transceiver for testing. The card has PCIe Gen 3. I have also tried all settings in the network card. As well as other PCIe slots and drivers. I have also found two forums on the net where users describe this problem in connection with Windows 10 Pro 2004. For me, however, the problem exists with 1909 as well as 2004 and Windows 10 Pro for Workstation. I am open for ideas. Thanks a lot!

[Applefreak]

8 minutes ago, havok2 said:

Hello dear community,
We have a full 10Gbe network in the office for our video production. Unfortunately I noticed that our systems with Threadripper 1950x as well as 2990wx only manage ~300 MB/s max in reading (writing is double @~700)+. Our NAS is a Synology FS3400 with 18x4TB SSDs. Connected to a Netgear M4300 x24-f24 via 4x10Gbe LACP. Our Windows Server 2016 with 4x10Gbe LACP is the only one that can nearly max out the full 10Gbe in both directions at CrystalDiskMark. Our Threaripper 3990x and all Intel systems also manage ~700mb here. Only the "older" Threadrippers do not. With iperf I don't get over 6Gbit/s in both directions (more than one thread). The bottleneck when reading is about the same as the performance with iperf with only one thread on the loopback. Could it have to do with the fact that the Aquantia agc107 which is installed onboard or also as a plug-in card in the form of an Asus XGC-100C only has 4x PCIe Gen 2.0? All PCs use this card, but the TR 1+2 Gen have deficits with it. I have now ordered a Mellanox Conect X 4 with RJ45 transceiver for testing. The card has PCIe Gen 3. I have also tried all settings in the network card. As well as other PCIe slots and drivers. I have also found two forums on the net where users describe this problem in connection with Windows 10 Pro 2004. For me, however, the problem exists with 1909 as well as 2004 and Windows 10 Pro for Workstation. I am open for ideas. Thanks a lot!

Have you tried with Linux, could be an OS thing. Is you windows installation up to date, latest chipset drivers installed, latest firmware on the AQ107 cards? From my experience, there are some quirks with Zen and Zen+ in Windows 10 that have been resolved in Zen 2. Linux, in my case Ubuntu had no such problems. I have a 1920x in the office setup for remote access to allow staff to work on. The host is Ubuntu and the guest VMs are Windows 10 2004. Maybe try it out to rule out hardware deficiencies. 

[havok2]

10 hours ago, Applefreak said:

Have you tried with Linux, could be an OS thing. Is you windows installation up to date, latest chipset drivers installed, latest firmware on the AQ107 cards? From my experience, there are some quirks with Zen and Zen+ in Windows 10 that have been resolved in Zen 2. Linux, in my case Ubuntu had no such problems. I have a 1920x in the office setup for remote access to allow staff to work on. The host is Ubuntu and the guest VMs are Windows 10 2004. Maybe try it out to rule out hardware deficiencies. 

He thanks. Yeah I updated Chipset and Drivers as Well the Firmware of the AQC107 Networkchip. Nothing but 10BM/s more. Yeah I will try a clean Windows, Linux and maybe Server 2016 this week. I read that it is a „Windows Thing“ but we need it sadly for all of our tools we use. Maybe its realy the case that windows uses one core only. Maybe I can speed it up with faster speed per lane with an PCIe 3.0 Card. Maybe the newer card offers us some cpu offloading that helps too. I mean at all we can handle our RED 8K stuff very well, but there is always something that can be optimized. And latest when we build our Server 2022 with 2x Honey Badger and 100Gbit I don't want a OS barrier like that

 

Update:

I was able to find a driver from 2017 which supports Direct Cache Access. Why this feature was removed later I don't know. Also I changed from 4088 byte jumboframes to 9000 byte jumboframes. With this I was able to achieve 653MB/s read and 1234MB/s write on the Threadripper 1950x. However, the card could not boot under the Threadripper 2999wx with the driver. Thanks to the jumbos, I currently end up with 480MB/s Rx and 1230MB/s Tx. I am now waiting for the new NIC. It really seems to be a driver/Windows problem. I hope the new card is better on the way. I will get back to you.

[havok2]

Hello,
neither a ConnectX4 nor an X540-T1 brought more throughput. I am leaning more and more towards a CPU/BUS/OS problem. A 5860K or a 9900K can do 1234mb/s transmitting. But a 2990WX only <600mb/s... I am at my wit's end. According to powershell all NICs are connected correctly with PCIe lanes and Gen.