Eliminating the file transfer slowdown

[dbG33K]

When I transfer a 20GB+ file from my SSD over to a traditional spinning HDD, I get terrific speeds for the first few seconds (2 or 4 GB maybe?), but then the speeds drop to like 10% of those high speeds for the remainder of the transfer. My understanding was that this is due to a high speed cache on one of the drives, I'd assume the SSD. Is this correct?

 

My bigger question then is this: is there a way to reduce or eliminate this extreme slowdown? I've heard of creating SSD Caches. Is this a possible solution to the slowdown issue on a Windows 10 (non-NAS) OS? Or am I way off?

 

TIA!

 

EDIT: For context, my OS drive is a Samsung NVMe drive, and I have an SSD I use for games and other downloads, and I use my Windows 10 machine as my Plex Media Server, with my media housed in ~25TB of HDDs.

[Princess Luna]

HDDs are slow, there's really no way around it... only way to make things faster is replace the HDDs for SSDs, even cheap ones like the Crucial BX500 would be very superior on data transfer.

[dbG33K]

7 minutes ago, Princess Luna said:

HDDs are slow, there's really no way around it... only way to make things faster is replace the HDDs for SSDs, even cheap ones like the Crucial BX500 would be very superior on data transfer.

Yes, true, but even still on SSDs, there's a brief period of time where the transfer speed is what it "should" be, followed by reduced speeds. Is there a way to reduce that?

[Princess Luna]

Just now, dbG33K said:

Yes, true, but even still on SSDs, there's a brief period of time where the transfer speed is what it "should" be, followed by reduced speeds. Is there a way to reduce that?

That's normal behavior.

[Lenovo1984]

20 minutes ago, dbG33K said:

Yes, true, but even still on SSDs, there's a brief period of time where the transfer speed is what it "should" be, followed by reduced speeds. Is there a way to reduce that?

No, that is completely normal and expected. 

[mariushm]

Windows (through OS and SATA controller) will cache some amount of data in RAM while starting transfer towards the drive.

So a few GB of content will seem to be transferred super fast... once the maximum amount of data fills the ram, your speed will be limited by the disk throughput.

So let's say you have 20GB and you transfer from SSD to HDD ... the SSD reads with up to 300-500 MB/s (let's say 400 MB/s sustained) and the hard drive writes with 200 MB/s

 

1s ... SSD read 400 MB, HDD wrote 200 MB , still in RAM 200 MB - speed reported 400 MB/s

2s ... SSD read 400 MB, HDD wrote 200 MB ,, still in RAM 400 MB - speed reported 400 MB/s

...

5s .. SSD read 400 MB, HDD wrote 200 MB ... still in ram 1000 MB - speed still reported 400 MB/s but maximum ram cache reached

6s .. SSD read 200 MB, HDD wrote 200 MB (from ram), data from SSD fills back the 1 GB ram cache, windows reports (5s x 400 + 1 x 200 ) / 6s = 366 MB/s

7s .. SSD read 200 MB, HDD wrote 200 MB (from ram), data from SSD fills back the 1 GB ram cache, windows reports (5s x 400 + 2 x 200 ) / 7s = 340 MB/s

... and so on... something like that.

 

The mechanical hard drive speed will also vary as it has to seek... if your mechanical drive is fragmented, windows constantly has to find areas that are as continuous as possible and empty and write as big as possible chunk of data of those 20 GB in that area, so the drive will have to move its heads to new locations often so the speed of a mechanical drive will often vary depending on where the drive heads are.

The ntfs driver has a "bitmap", keeps track of the biggest continuous strings of empty sectors, to reduce fragmentation.. .for example if an application says it's gonna write a 100 MB, it will try to begin writing the file in an area where there's more than 100 MB of continuous free space. It won't start writing in an area where there's only let's say 10 MB of free disk space between two other files.

 

[dbG33K]

Can I somehow delegate more RAM for transfer cacheing?

[hojnikb]

SSDs nowadays all use some sort of SLC write caching, which means that for the first few GBs (depending on the drive and size) will write very quickly, but after that cache is filled, writes will be much slower (can be as slow as 50-100MBs on cheap TLC/QLC drives). You cannot get around that, unless you buy a different SSD, that has no caching.

 

HDDs on the other hand have a different issue. Because of their mechanical nature, data on the edge of the magnetic disc will be read or written to the fastest, but the slowest when reaching the inner circle (thats due to constant rpms of the disc).