Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
michael_p

4x10Gb NIC performance

Recommended Posts

Posted · Original PosterOP

Hi,

 

I'm new to this forum and I have an issue.

I have point to point network, 4 clients connected to one server directly (no switch):

  • 1 server - i9-9900 with 4 port 10Gb NIC (intel x710),  windows10
  • 4 clients - xeon with 1 port 10Gb NIC, windows10

When I run iPerf I get the following results:

  1. 1 to 1 - 10Gb (doesn't meter which one)
  2. 2 to 1 - 10Gb x 2 (20Gb total)
  3. 3 to 1 - 8Gb x 3 (24Gb total)
  4. 4 to 1 - 6Gb x 4 (24Gb total)

The server is maxed out at ~25Gb/s

I tried to change the buffers, number of connection and nothing helps.

 

Please help,

Michael

Link to post
Share on other sites
20 minutes ago, michael_p said:

Hi,

 

I'm new to this forum and I have an issue.

I have point to point network, 4 clients connected to one server directly (no switch):

  • 1 server - i9-9900 with 4 port 10Gb NIC (intel x710),  windows10
  • 4 clients - xeon with 1 port 10Gb NIC, windows10 

When I run iPerf I get the following results:

  1. 1 to 1 - 10Gb (doesn't meter which one) 
  2. 2 to 1 - 10Gb x 2 (20Gb total) 
  3. 3 to 1 - 8Gb x 3 (24Gb total)
  4. 4 to 1 - 6Gb x 4 (24Gb total)

The server is maxed out at ~25Gb/s 

I tried to change the buffers, number of connection and nothing helps.

 

Please help,

Michael

Are you sure you aren't maxing out the storage you are using?   This is assuming you aren't using a ram disk or null to perform throughput testing.


If you are performing testing that is direct to storage, then you are likely limited by IO constraints.  Assuming you are using Windows and using somthing like NTtccp for testing, here are some additional options you will need enabling for the network config/driver;

 

Latest available Intel drivers

RSS (Recieve Side Scaling)
RSS Queues - Max the driver is capable of using

RSS Processors - Max threads on the system (logical processors)

MTU 9000 (Jumbo Frames) **this also NEEDS to be enabled on all your switching equipment**

Transmit Buffers - Max the driver is capable of using

Recieve Buffers - Max the driver is capable of using

 

If you can provide more details on how you are performing testing and what software you are using that would also be helpful.


Please quote or tag me if you need a reply

Link to post
Share on other sites

Is the data going out the network card or just flowing between ports?

 

If it has to get out the card, it could be an issue of pci-e bandwidth ... if the slot is pci-e 2.0 x8, then you have a max 8 x 500 MB = 4 GB/s or 32 gbps theoretical ... with overhead and stuff, you're looking at maybe 28 gbps maximum

Same if the slot is pci-e 3.0 x4 (slot may be x16 physically but only x4 electrically) ... a single 3.0 lane is ~ 970 MB/s so a x4 will be less than 4 GB/s.

 

Also keep in mind if the slot is created by PCH/chipset you have a DMI 3.0 connection to the cpu which has a maximum bandwidth of around 4 GB/s (at least it was that much on previous generations, not sure about i9 9900 )

Link to post
Share on other sites
Posted · Original PosterOP

Hi,

 

Thank you for replay.

  • I double check that I'm using full PCIe 3 x 8 minimum
  • All card/ports set to jumbo frame 9000
  • All RSS are maxed out
  • All buffers are maxed out

I perform the test using -

  • iPerf its network speed test that doesn't copy any thing to storage, only receiving data and discards it
  • My own software that basically does the same as iPerf

 

Is there something to do with windows?

 

Thanks

Link to post
Share on other sites

which version of windows do you use?

some older versions have an explorer with limited transfer speed(around 3-6GB/s) [like Linus showed in the "100GBit/s NICs"]

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×