file transfert speed

[asheenlevrai]

Hi

I always hear (read) that when transferring files (from a PC to another or from a drive to another on the same machine), it is slower if many small files are transferred compared to few large files.

 

What I mean is that I understood that if I need to transfer 100GB, it will take longer if I need to transfer 10'000 files of 10MB each rather than 10 files of 10GB each, right?

I am not sure I understand 100% why, but this seems to be the consensus. Let me know if I'm wrong here...

 

Now what if I would take those 10MB files and zip them into a single 100GB file (let's assume the compression rate is 0% for the sake of this example)? Would that large zip file be transferred faster than the sum of all the files it contains?

 

Thank you very much in advance for your help and explanations.

 

Best,

-a-

[Eigenvektor]

Yes, in most cases the single large ZIP file would be transferred faster. The reason is simply that there is some overhead when transferring a file:

• find the file (e.g. especially on a HDD seek time)
• open it for reading
• reserve space on the remote machine (check if there's enough)
• create a file
• open it for writing

With a single large file you have that overhead once, and the rest is transfer. With 10,000 files you have that overhead 10,000 times + the actual transfer.

[asheenlevrai]

Thank you so much @Eigenvektor

[LogicalDrm]

-> Moved to Storage Devices