How does steam work?

[Jason1232]

How does Steam download updates for games without re-downloading all the files?

 

My complete guess would be some kind of checksum...check... then update based on that but i have no idea how and its kinda got me curious.

[kelvinhall05]

It just downloads the new files. You're overthinking it.

[Jason1232]

So, whats stooping it from over-writing files that don't need to be updated, and allows it to over-write the ones that do?

[Mira Yurizaki]

1 minute ago, Jason1232 said:

So, whats stooping it from over-writing files that don't need to be updated, and allows it to over-write the ones that do?

There's probably something on the server that contains the deltas of what changed and downloads only that.

[QuantumBit]

1 minute ago, Jason1232 said:

So, whats stooping it from over-writing files that don't need to be updated, and allows it to over-write the ones that do?

Quote people or else they wont see it.

Steam just knows not to update those files? Like @kelvinhall05 said you're overthinking this.

[kelvinhall05]

3 minutes ago, Jason1232 said:

So, whats stooping it from over-writing files that don't need to be updated, and allows it to over-write the ones that do?

It only downloads the required files. If it's a bug fix, it'll overwrite the files that have been specified in the update, but otherwise, nothing gets overwritten.

[Jason1232]

4 minutes ago, QuantumBit said:

Quote people or else they wont see it.

Steam just knows not to update those files? Like @kelvinhall05 said you're overthinking this.

I'm not overthinking it, I hate to sound like a dick but i am asking a how does something work, and your response is it just does, stop thinking about it.

 

 

[Mira Yurizaki]

1 minute ago, Jason1232 said:

I'm not overthinking it, I hate to sound like a dick but i am asking a how does something work, and your response is it just does, stop thinking about it.

It probably uses MD5s. And if anything it probably uses a system similar to Subversion or Git for handling files.

[Jason1232]

5 minutes ago, M.Yurizaki said:

It probably uses MD5s. And if anything it probably uses a system similar to Subversion or Git for handling files.

Now, that's useful, that gives me something i can go look into further, so thanks.

[Tabs]

Just now, Jason1232 said:

I'm not overthinking it, I hate to sound like a dick but i am asking a how does something work, and your response is it just does, stop thinking about it.

 

 

The version of the game you have installed is known by steam as a "manifest". Different versions of a game, for example, the English and the French version of the same game, are called "depots", and the game as a whole is referenced by it's "appid".

 

When steam checks for updates, it sends your appid+depotid+manifest version to steams servers. It gets a response telling steam how many - if any - new "manifests" have been released. The manifest itself tells steam exactly what files have CHANGED between manifests, so when an update is released, steam only downloads the files changed between the current manifest, and the newest manifest.

 

You can use steamdb.info to view individual changes between manifests - here for example is the main data file for Fallout 4, showing the full filetree, and the manifests it's included in.

 

I hope this information helps.

[Tabs]

2 minutes ago, Jason1232 said:

Now, that's useful, that gives me something i can go look into further, so thanks.

This is partially right; steam hashes files to verify their integrity, but it does not use them for the purpose of checking versions. Any changed files between manifests get completely re-downloaded, rather than the use of deltas or other patching mechanisms.

[Jason1232]

4 minutes ago, Tabs said:

The version of the game you have installed is known by steam as a "manifest". Different versions of a game, for example, the English and the French version of the same game, are called "depots", and the game as a whole is referenced by it's "appid".

 

When steam checks for updates, it sends your appid+depotid+manifest version to steams servers. It gets a response telling steam how many - if any - new "manifests" have been released. The manifest itself tells steam exactly what files have CHANGED between manifests, so when an update is released, steam only downloads the files changed between the current manifest, and the newest manifest.

 

You can use steamdb.info to view individual changes between manifests - here for example is the main data file for Fallout 4, showing the full filetree, and the manifests it's included in.

 

I hope this information helps.

Now that was just what i was looking for!

 

Thanks!

[Jason1232]

17 minutes ago, Tabs said:

This is partially right; steam hashes files to verify their integrity, but it does not use them for the purpose of checking versions. Any changed files between manifests get completely re-downloaded, rather than the use of deltas or other patching mechanisms.

I know its a bit crude, but was just trying to figure it out, somthing like this?

https://gyazo.com/c33cf8f0be809d0a7926200de5eec4c7

[Tabs]

 

30 minutes ago, Jason1232 said:

I know its a bit crude, but was just trying to figure it out, somthing like this?

https://gyazo.com/c33cf8f0be809d0a7926200de5eec4c7

 

Almost right - Everything is perfect except the appid parent. It'd go straight from steam to Fallout 4 (appid 377163), etc since they all have separate appID's. We see the "nice name" but steam treats Fallout 4 as simply a bundle with AppID 377163.

 

 

And, when it comes to patching, effectively your machine would get a list of versions and download anything from the updates that is new.

 

So, for example:

 

App V1.0 (installed) > App V1.1 (updates to app.exe and app.dat) > App V1.2 (updates to app.dat and new file app2.dat)

 

In this scenario, Steam would download app.exe from V1.1 and the app.dat and app2.dat files form V1.2, since deltas are not used and those are the "effective" changes from V1.0 to V1.2. I know you didn't ask for clarity on this part but I thought I'd elaborate.

[Jason1232]

19 minutes ago, Tabs said:

 

 

Almost right - Everything is perfect except the appid parent. It'd go straight from steam to Fallout 4 (appid 377163), etc since they all have separate appID's. We see the "nice name" but steam treats Fallout 4 as simply a bundle with AppID 377163.

 

 

And, when it comes to patching, effectively your machine would get a list of versions and download anything from the updates that is new.

 

So, for example:

 

App V1.0 (installed) > App V1.1 (updates to app.exe and app.dat) > App V1.2 (updates to app.dat and new file app2.dat)

 

In this scenario, Steam would download app.exe from V1.1 and the app.dat and app2.dat files form V1.2, since deltas are not used and those are the "effective" changes from V1.0 to V1.2. I know you didn't ask for clarity on this part but I thought I'd elaborate.

I see so rename "nice names" to just 123456 (or somthing like that).

[Tabs]

Just now, Jason1232 said:

I see so rename "nice names" to just 123456 (or somthing like that).

Yep, pretty much. For completeness sake, underneath each of V1, V1.1 etc would be a filetree, but for the purposes of demonstrating how the system works, that pretty much nails it.