10Gb Network Performance

[TheTwist]

We have a 10Gb network set up between a NAS and workstations on a Windows domain. However, when files are copied it acts like 1Gb connections.  The speed on average is 30Mb/s when a file is being copied from the NAS to a workstation. Some files are up to 25GB’s.

There are 2x 10Gb cards in NAS bonded and connected onto a 10Gb Switch which is LAG LACP. Then separate Cat6a ethernet cables from one port on the switch to each work station individually which has its own 10Gb card.

It could be the NAS hard drives but not sure  They are WD SATA 6Gb/s HDDs and run as RAID6

The work station are all SSD’s

[mariushm]

Gigabit connections will give you up to 125 MB/s .... 30 MB/s or mbps is awful.

Did you try without bonding ? Maybe that causes some conflict.

 

How's the speed through FTP for example, or some other protocol? Do you get more than 1 gbps when you transfer something using multiple parallel connections?

[Windows7ge]

Running an iperf test probably wouldn't hurt. It'll tell us if it's the network or something else.

[TheTwist]

18 hours ago, mariushm said:

Gigabit connections will give you up to 125 MB/s .... 30 MB/s or mbps is awful.

Did you try without bonding ? Maybe that causes some conflict.

 

How's the speed through FTP for example, or some other protocol? Do you get more than 1 gbps when you transfer something using multiple parallel connections?

 

Oops i meant 130 MB/s i did try with out bonding which is the same.  I have tried FTP and and it is a lot slower. 

18 hours ago, Windows7ge said:

Running an iperf test probably wouldn't hurt. It'll tell us if it's the network or something else.

 

Attached are the iperf results through the 10Gb connetion between workstation and NAS. First one is the NAS second is the workstation 

 

[Helly]

just wondering, did you add the 10Gb cards to the NAS yourself? If so, how old is this NAS. It's possible the CPU in it is just to slow for 10Gbit. Can you check the cpu usage on the NAS while copying a file?

[TheTwist]

46 minutes ago, Helly said:

just wondering, did you add the 10Gb cards to the NAS yourself? If so, how old is this NAS. It's possible the CPU in it is just to slow for 10Gbit. Can you check the cpu usage on the NAS while copying a file?

Our supplier added them in when the NAS was brought in 2017.    The CPU goes up to by about 10% copying things over. 

[Windows7ge]

2 hours ago, TheTwist said:

Attached are the iperf results through the 10Gb connetion between workstation and NAS. First one is the NAS second is the workstation

This tells us it's not the storage. When did the problem first propagate? Was this a new configuration or did it just stop doing what it was suppose to overnight?

[TheTwist]

15 minutes ago, Windows7ge said:

This tells us it's not the storage. When did the problem first propagate? Was this a new configuration or did it just stop doing what it was suppose to overnight?

We recently moved but everything is setup the same. Not settings changed on the switches or NAS. 

[Windows7ge]

3 minutes ago, TheTwist said:

We recently moved but everything is setup the same. Not settings changed on the switches or NAS. 

What cables are used for the 10Gbit network? Fibre? DAC? Cat#?

[TheTwist]

23 minutes ago, Windows7ge said:

What cables are used for the 10Gbit network? Fibre? DAC? Cat#?

All of them are Cat6a 

[Windows7ge]

2 minutes ago, TheTwist said:

All of them are Cat6a 

Do you have spare cables? Did you try switching them? If the problem started not long after a move it could be cable damage.

[TheTwist]

5 minutes ago, Windows7ge said:

Do you have spare cables? Did you try switching them? If the problem started not long after a move it could be cable damage.

There are 4 computers connected to the 10Gb.  And they are all the same. Aswell as they are on the same switch as the NAS

[Windows7ge]

1 minute ago, TheTwist said:

There are 4 computers connected to the 10Gb.  And they are all the same. Aswell as they are on the same switch as the NAS

If they all act the same it could be one of either cable going from the server to the switch. Iperf shows it's a network issue starting at the physical layer is the easiest place to begin the troubleshooting.

[TheTwist]

18 minutes ago, Windows7ge said:

If they all act the same it could be one of either cable going from the server to the switch. Iperf shows it's a network issue starting at the physical layer is the easiest place to begin the troubleshooting.

Yeah i'll take out all the cables from that 10Gb switch. Only allow 1 workstation and the synology and go from there. 

Thanks for your help. In the mean time, will post once I have done that.

[TheTwist]

Just isolated between the switch, workstation and NAS with nothing else connected to the Network.  Still getting the same Iperf results.

 

Could it be the NIC card in the work station? I have re-installed all the drivers.

[Windows7ge]

21 minutes ago, TheTwist said:

Could it be the NIC card in the work station? I have re-installed all the drivers.

If it worked before but all the clients dropped to this performance number at once it's incredibly unlikely to be the clients NIC. The servers NIC perhaps. What was the performance before this happened? What RJ-45 NICs are you using?

[Alex Atkin UK]

Have you tried doing iperf between the clients too?

 

If they are running Windows, it could also be worth trying between two running Linux off USB sticks, just to isolate OS updates as a potential issue.

 

I have to admit, I did have problems myself using LACP bonding two Gigabit ports which came out with a similar speed.  Its particularly unusual though as in your case it was working before.

[TheTwist]

Right so we actually have another server which is not connected to the 10Gig switch but with 10Gig NIC's. This also has given out the same speed using Iperf.

 

The NIC cards we are using for the Workstations are Intel X540-T2 I also ran Iperf between 2 work stations  which didnt give 

I am thinking it might be these cards which the settings are not correct? 

 

Here are the settings to the Workstation Nic's

 

Flow Control Rx & Tx Enabled 
Header Data Split Disabled 
Interrupt Moderation Enabled
Interrupt Moderation Rate Adaptive
IPsec Overload Auth Header & ESP Enable
IPV4 Checksum overload Rx & Tx Enabled 
jumbo packet disabled
Large send offload V2 Enabled
Local Administered Address 0
Local Link State Event Enabled
Low Latency Interrupt Ports NA
Low Latency Interrupt Ports Disabled
Maximum Number of RSS Queues 8
Packet Priority & VLAN Enabled
Received Buffers 512
Received Side Scaling   Enabled 
Speed & Duplex  Auto Negotiation
TCP Checksum Offload  RX & TX Enabled
Transmit Buffers 16384
UDP Checksum Offload Rx & Tx enabled