Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

SMB PC-PC - 350MB/s with 5Gbps connection, just right or AMD bottlenecking?

Hi everyone.

 

So I'm trying to get a 'simple' file transfer in place between my main PC and a Server PC. Both are running Windows 10, main PC has a 3900X, 32GB Ram, 1 x 1Gbps on board and a separate PCI-e with 2 x 2.5Gbps connections. Server PC has the same network setup, with a Ryzen 3600 and 32GB Ram.

 

I have the two PC's connected directly to each other via the two 2.5Gbps connections, each has it's own IP, and I read not to assign a gateway or DNS to this (contrary to Level1's advice on this, though I found it made no difference to transfer speeds). Both NIC's are RSS capable, and transferring a large .zip file or video file, I can see at both ends the throuughput is being shared equally between the two connections on each computer, so the SMB function must be working. However, with a total expected throughput of 5Gpbs, I was expected transfer speeds over 600MBps, but I'm only getting 300-350MBps for a few seconds, before it drops into the 200's and sometimes less. Looking on task manager, I can see both CPU's are maxing out on all cores, so I was wondering if the speed could be CPU bottlenecked (should add, there's nothing going through the 1Gbps connections, which are on a separate subnet). I'm also transferring from m.2 to m.2 and they are coping just fine in task manager. I wouldn't have thought so, watching the Quadruple your Network Speed Video from a few years back where they were using older processors, I would have thought a higher throughput would have been possible. I've tried everything I can think of and everything of mutliple sites and forums, and I still don't get close to the speeds I anticipated.

 

In anticipation of the 'why do you need that speed', simply put, I don't, but that's my latest little project so that's what I want to do. 🙂 

 

Any help or guidance would be appreciated!

Link to post
Share on other sites

If your CPUs are maxed out then that's the bottleneck, yes. I wonder, however, why would they be maxed out by a simple transfer over ethernet, even tho it's at a relatively high speed. I've seen even stronger CPUs than the 3900X max out when downloading games from steam at 10Gbps but that also includes decompression and installation of the data being transfered and it's at higher transfer speeds. Maybe the configuration could be optimized.

Link to post
Share on other sites

Make a 16-24 GB ram drive on the computer and save to the RAM drive and repeat the test. I use ImDisk Toolkit but there's other freeware applications that can do ram drives.

At least this would take out of the equation the storage on one side.

You shouldn't get you cpu saturated with this... try and see the cpu usage with just one link instead of two - could be the cpu usage is caused by the combination of data from both ports in memory or some other stuff related to using two connections.

 

Also try using something smarter than Windows Explorer, like for example FastCopy - https://fastcopy.jp/  -  or TeraCopy or even Total Commander's copy (you have options to configure buffer sizes when reading files, which can speed up transfers)

Link to post
Share on other sites

Thank you both for the replies.

 

1 hour ago, mariushm said:

Make a 16-24 GB ram drive on the computer and save to the RAM drive and repeat the test. I use ImDisk Toolkit but there's other freeware applications that can do ram drives.

 

Checking the disks being written from and to, both are barely going over 10% usage, so do you think the RAM disk is necessary. I put an SSD in my server to ask as a cache for transfers, but AMD's software is worse than temporamental and wouldn't install, so I'll use it as a manual cache if needed. 

1 hour ago, mariushm said:

Also try using something smarter than Windows Explorer, like for example FastCopy - https://fastcopy.jp/  -  or TeraCopy or even Total Commander's copy (you have options to configure buffer sizes when reading files, which can speed up transfers)

I also tried FastCopy and TeraCopy and both struggled with transfer speeds above 200MB/s. I tried adjusting settings, but neither would go higher. Interestingly, the CPU's were still maxed out on with these speeds. Not sure what is happening here, but something doesn't feel right.

Link to post
Share on other sites

Have you looked to see if your cards supports Interrupt Moderation?

 

I had to disable it on my 10Gbit cards to get performance up which is kinda funny as the whole point of it is to reduce CPU usage, but something just wasn't quite right with how it was functioning.

 

Interestingly performance as fine going Linux to Linux, the problem was going from the Linux NAS to Windows 10 and yes even Windows 10 to Windows 10 is impacted.  I've found there is something fundamentally broken in Windows 10 networking lately.

Router: i5-7200U appliance running pfSense WiFi: Honor Router 3 (~850Mbit peak throughput)
ISPs: Zen VDSL (66Mbit) + Plusnet VDSL (66Mbit) + Three 5G (~600Mbit)

Link to post
Share on other sites
3 hours ago, Alex Atkin UK said:

Have you looked to see if your cards supports Interrupt Moderation?

 

I had to disable it on my 10Gbit cards to get performance up which is kinda funny as the whole point of it is to reduce CPU usage, but something just wasn't quite right with how it was functioning.

 

Hi, thanks for the reply. So both cards support interrupt moderation, I disabled on all 4 connections, and still no luck - initial peak at 350MB/s then a drop to sub 200MB/s. 🤔

Link to post
Share on other sites

Have tried a number of 'make your network faster' sites, most were a complete flop, unsurprisingly, not as a criticisim but I accept each issue is likely to be unique. I had a quick thought, as a test, to try an internal SSD-SSD transfer, and after an initial transfer speed peak of over 1GB/s, I dropped to a consistent 400-420MB/s - could this be a processor/SSD issue, not a hardware one?

Link to post
Share on other sites

So quick update, after leaving the settings, I'm getting consistent 380MB/s for the first 2gb or so of transfer before it drops. Still not quite sure what to try next. Smb is clearly working, so where is the throttling occurring? 

Link to post
Share on other sites
32 minutes ago, scottiecb19 said:

So quick update, after leaving the settings, I'm getting consistent 380MB/s for the first 2gb or so of transfer before it drops. Still not quite sure what to try next. Smb is clearly working, so where is the throttling occurring? 

Could it be the drives themselves running out of cache? What happens when you do a file transfer between two drives on the same machine? Just a shot in the dark...

Link to post
Share on other sites

What does iperf say?

 

I would say that if your SSDs can sustain writing about 200MB/sec indefinitely, you can't complain.

 

Perhaps try it with a single 2.5G connection so as not to overwhelm your CPU.

 

Link to post
Share on other sites
17 hours ago, Blue4130 said:

Could it be the drives themselves running out of cache? What happens when you do a file transfer between two drives on the same machine? Just a shot in the dark...

Quote
On 9/19/2021 at 3:01 PM, scottiecb19 said:

Have tried a number of 'make your network faster' sites, most were a complete flop, unsurprisingly, not as a criticisim but I accept each issue is likely to be unique. I had a quick thought, as a test, to try an internal SSD-SSD transfer, and after an initial transfer speed peak of over 1GB/s, I dropped to a consistent 400-420MB/s - could this be a processor/SSD issue, not a hardware one?

 

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×