Trying to get 2GB/s on one PC out of 2x 10GbE ports

[WizPip]

Hey team!

 

I have two Win10 based machines, each with a twin 10GbE port Intel X550 adapter. These are all connected (4 wires total) to a 10GbE switch.

 

I fell down a huge number of rabbit holes before coming to the SMB Multichannel conclusion, which annoyingly mostly "just works". (For the love of god, avoid link aggregation if you just want speed!)

 

So, I'm at the point where I'm up to 1.3GB/s transfer speeds, but I want as close to the full 20Gbit (2.5GB theoretical maximum) possible. I have confirmed that during a file transfer both ports on both machines are all active and sending data at balanced speeds. Crystal disk mark says the M.2 in one machine and 5x 1TB SSDs in a simple Storage Space with 5 columns in the other are all capable of > 2GB/s read / write speeds... so what do I have left to poke to improve the speed?

 

Here are some pics which may answer some questions you may have...

NAS

 

PC

 

Transfer from NAS

 

RSS CPU config

 

PC during transfer

 

NAS during transfer

 

Some more SMB config below:

 

Switch

 

NIC config (all 4 identical) below:

[Jurrunio]

Z97 chipset only runs PCIe 2.0, if you write to a drive on that then you'll get a 2GB/s cap from there. Maybe something else is hogging transfer at the same time?

[svmlegacy]

4 minutes ago, Jurrunio said:

Z97 chipset only runs PCIe 2.0, if you write to a drive on that then you'll get a 2GB/s cap from there. Maybe something else is hogging transfer at the same time?

That being said, OP could try to run the network adapter in the single PCIe 3.0 x16 allocated, and either use integrated GPU or another PCIe x16 for graphics, to validate this theory.

[WizPip]

43 minutes ago, svmlegacy said:

That being said, OP could try to run the network adapter in the single PCIe 3.0 x16 allocated, and either use integrated GPU or another PCIe x16 for graphics, to validate this theory.

A good question, but both machines appear to be running from the PCIe 3.0 provision from the CPU rather than the chipset, and so are both running in PCIe 3.0 4x (3.8GB/s)

 

The NAS has no separate video card and is driven by the CPU.

 

PC

 

NAS

[Senzelian]

I'm a bit confused regarding the storage configuration of the NAS.

Which filesystem is it running and how are the drives exactly configured? Is it a RAID array?

 

Also what exactly is it that you tried to transfer from your NAS to your PC?

Is it just a game-folder or is it something like a .zip? 

[WizPip]

11 minutes ago, Senzelian said:

I'm a bit confused regarding the storage configuration of the NAS.

Which filesystem is it running and how are the drives exactly configured? Is it a RAID array?

 

Also what exactly is it that you tried to transfer from your NAS to your PC?

Is it just a game-folder or is it something like a .zip? 

NTFS and...

1 hour ago, WizPip said:

5x 1TB SSDs in a simple Storage Space with 5 columns

Note that the space in question is called "Fast space". 5 columns means the data is striped across all 5 drives, so this is effectively RAID 0, which explains the 2113MB/s read and 2482MB/s write sequential shown above.

 

What the GUI shows (note that you can't configure columns in the GUI so if you're having issues with a simple space's speed, this is probably why)

 

I have tried with various types of files, the screenshot in the OP shows a single large mkv file being transferred. Trying multiple at the same time does not affect this overall speed, and yet there would be more overhead for the disks doing this (yet task manager shows nothing maxed), so I don't believe we're at the disk limits.

[WizPip]

Just to confirm the speeds with only 1 cable I yanked one out during transfer, one from the PC and one from the NAS. It didn't stop transferring when I pulled one out but it got stuck for some time with one in each out. Then it took a few mins to start using both ports again when I replugged it all in.

 

I did confirm that the max speed with one cable in each is 1.08GB/s and the max I saw with two cables was 1.47GB/s. I guess I'm relatively happy with it overall, but if anyone at any time in the future notices an optimisation I've missed that can make it faster, let me know!