Steam Downloads Games Faster on Ubuntu/Linux than it does on Windows

[DigiThax]

TL; DR;
I've setup my PC to dual boot and steam downloads (not installs, I'm talking about consistent high speed download rates) the same games much faster on Linux than they do on Windows.
Has anyone else noticed the same?

 

Here's some information:

I have a gigabit ethernet connection and I recently had to format my PC, I use the same free version of Windows 10 (might this be the issue?) and the latest LTS version of Ubuntu
(forgot the version number but I setup the bootstick only a month back)

Both OSs are on the same Gen3 NVME SSD and yes I was using the same ethernet connection in both instances. When picking my games to download on Linux, the download
rates were consistently above 80MB/s, something which Windows only does during the first few seconds of the download then drops down to below 30MB/s and stays there.

I also ensured both the Steam apps used the same Download Region when I noticed the difference.

 

Is there a way to get the same download speeds on Windows? And what is slowing it down?

[Kilrah]

If the games are uncompressing to a lot of small files that's something linux does a lot better at. 

[DigiThax]

3 hours ago, Kilrah said:

If the games are uncompressing to a lot of small files that's something linux does a lot better at. 

If a game has support for both OSs, yes that would be true. But these are windows only games that I'm running with proton and they're packaged the same way regardless if I download it on Windows or Linux right?

[LloydLynx]

13 hours ago, DigiThax said:

If a game has support for both OSs, yes that would be true. But these are windows only games that I'm running with proton and they're packaged the same way regardless if I download it on Windows or Linux right?

Downloading and decompressing the game data has nothing to do with running the game. Downloading and decompressing speed has to do with how the OS processes stuff on the CPU and how the file system (Windows NTFS vs Linux EXT4) writes and arranges data. Linux just happens to be better at parallel processing and EXT4 tends to be faster than NTFS. 

[Kilrah]

23 hours ago, DigiThax said:

But these are windows only games that I'm running with proton and they're packaged the same way regardless if I download it on Windows or Linux right?

Yes they're packaged the same way but that's the point, they both have to do the same thing and Windows is extremely slow at handling tons of small files compared to linux. 

One reason is Windows defender scanning each file or viruses as it gets accessed which doesn't happen on linux, suspending Defender during the process tends to help a bit but that's not the only reason. 

[DigiThax]

1 hour ago, Kilrah said:

Yes they're packaged the same way but that's the point, they both have to do the same thing and Windows is extremely slow at handling tons of small files compared to linux. 

One reason is Windows defender scanning each file or viruses as it gets accessed which doesn't happen on linux, suspending Defender during the process tends to help a bit but that's not the only reason. 

Yes windows defender might have something to do with it. I'll experiment without it the next time thanks!

[DigiThax]

12 hours ago, LloydLynx said:

Downloading and decompressing the game data has nothing to do with running the game. Downloading and decompressing speed has to do with how the OS processes stuff on the CPU and how the file system (Windows NTFS vs Linux EXT4) writes and arranges data. Linux just happens to be better at parallel processing and EXT4 tends to be faster than NTFS. 

Thats the thing, we're not talking about speeds that HDD/SDDs can go to, we're dealing with speeds that any old hardware can sustain.

If Windows could never attain 100MB/s download rates that would be fine, but when I start the download it starts high and then slows down. Only occasionally going back above 80MB/s which too only lasts for a short time.

My question was why cant it sustain above 80MB/s on Windows when clearly downloads can reach that speeds for short bursts

[LloydLynx]

4 hours ago, DigiThax said:

If Windows could never attain 100MB/s download rates that would be fine, but when I start the download it starts high and then slows down. Only occasionally going back above 80MB/s which too only lasts for a short time.

I think because of write caching. Windows enables it by default on most, if not all, drives. 

 

Lets say we have 100MB of data to transfer onto the drive. Windows will write that data to RAM to make it seem like an instant transfer, but will then proceed to write that data to the drive in the background. This makes the system feel more snappy than it really is. And yes, this creates potential for data loss. The option to enable/disable it even warns you. You can find the options for this in Device Manager by right clicking on your drive and going to the tab for write caching. 

[DigiThax]

22 hours ago, LloydLynx said:

I think because of write caching. Windows enables it by default on most, if not all, drives. 

 

Lets say we have 100MB of data to transfer onto the drive. Windows will write that data to RAM to make it seem like an instant transfer, but will then proceed to write that data to the drive in the background. This makes the system feel more snappy than it really is. And yes, this creates potential for data loss. The option to enable/disable it even warns you. You can find the options for this in Device Manager by right clicking on your drive and going to the tab for write caching. 

Ah okay I dont know about that, its definitely something I will try. Thanks for the details!