File Copies In Windows Stops and Starts; V Shaped Ramps In Transfer Speed

JohnSmith2 · January 9

I have two Windows 10 computers directly networked together, but when I copy a single, huge file (20GB) between them, I get performance that looks like this:

For some reason, after a certain period of time, the copy speed falls off a cliff and then ramps back up. I can't really tell what's happening. When I copy the file over a gigabit or 2.5 gigabit connection, I never observe this behavior. But when the connection is 750 MB/sec (first example) or 2 GB/sec (second example), I see this.

I guess it could be the SSD write caches filling up, but I thought that would look different, something like this for example:

Given that's not what I'm seeing, and considering I see the same behavior no matter which pair of SSDs I'm copying between (Samsung 980 Pro, Samsung 970 Evo +, TeamGroup A440 Pro, Samsung MZVLQ512HBLU), I suspect it's something else. So maybe it's a Windows problem, or maybe it's related to how my network adapters are configured. I'm not sure.

I've seen this with two networking setups; two Intel XL710-XDA2 cards connected with a QSFP+ cable, and an OWC 10G Ethernet card connected to a Mellanox Connect-X2 card using an SFP+ to ethernet adapter. Both setups gave me file copies with V shaped dips like that.

Maybe there is a similarity in how the adapters are configured by default. Maybe there's a certain setting that could cause this? Any of these, maybe?

One computer has a Ryzen 7950X CPU and the other has an Intel i7-9750H CPU, in case that's relevant. Both are running Windows 10 21H2 (build 19044).

Any ideas what causes this behavior, and what I should try changing to fix it?

EDIT:

I tested copying the same 20GB file directly between the SSDs on the same system (so not over the network). They copied at a constant 1.8 to 2.2 GB/sec (depending on which pair of SSDs I was testing) the entire time, over the entire transfer. So I'm pretty sure this is networking related, and not SSD cache related.

Alex Atkin UK · January 9

6 hours ago, JohnSmith2 said:

I have two Windows 10 computers directly networked together, but when I copy a single, huge file (20GB) between them, I get performance that looks like this:

For some reason, after a certain period of time, the copy speed falls off a cliff and then ramps back up. I can't really tell what's happening. When I copy the file over a gigabit or 2.5 gigabit connection, I never observe this behavior. But when the connection is 750 MB/sec (first example) or 2 GB/sec (second example), I see this.

I guess it could be the SSD write caches filling up, but I thought that would look different, something like this for example:

Given that's not what I'm seeing, and considering I see the same behavior no matter which pair of SSDs I'm copying between (Samsung 980 Pro, Samsung 970 Evo +, TeamGroup A440 Pro, Samsung MZVLQ512HBLU), I suspect it's something else. So maybe it's a Windows problem, or maybe it's related to how my network adapters are configured. I'm not sure.

I've seen this with two networking setups; two Intel XL710-XDA2 cards connected with a QSFP+ cable, and an OWC 10G Ethernet card connected to a Mellanox Connect-X2 card using an SFP+ to ethernet adapter. Both setups gave me file copies with V shaped dips like that.

Maybe there is a similarity in how the adapters are configured by default. Maybe there's a certain setting that could cause this? Any of these, maybe?

One computer has a Ryzen 7950X CPU and the other has an Intel i7-9750H CPU, in case that's relevant. Both are running Windows 10 21H2 (build 19044).

Any ideas what causes this behavior, and what I should try changing to fix it?

EDIT:

I tested copying the same 20GB file directly between the SSDs on the same system (so not over the network). They copied at a constant 1.8 to 2.2 GB/sec (depending on which pair of SSDs I was testing) the entire time, over the entire transfer. So I'm pretty sure this is networking related, and not SSD cache related.

As they are networked directly to each other, have you tried using jumbo frames? Looks like its called Jumbo Packet in that list, increase it to something a lot higher than the default on both machines (must be identical). It may have a predefined list of packet sizes or let you freely enter a number. I'd try something like 9000.