Windows 10 Pro, integrated tiered storage?

Skyyblaze · August 1, 2020

Long story short for the past 20 days I have been using Primocache with a spare 120gb SSD on my gaming PC and I saw a great benefit in load-times in nearly all my games. Now that the 30 day trial period is coming to an end I would have no qualms paying the 30 or so bucks for a license but before I do so, I remembered Linus talking about Windows 10 having its own tiered storage system integrated in a video, I think it was in the Petabyte Project one? Looking into it it's surprisingly hard to find something about it online though so I wondered, do I misremember what Linus meant or misheard something, is it only available on Windows 10 Enterprise or Server? I'm running Windows 10 Pro.

Like said I don't mind paying for Primocache but if I can do the same for free with Windows itself I don't mind getting my hands dirty All I want to do is having my 120gb spare SSD accelerate my two game HDDs.

Thanks for help in advance!

GoodBytes · August 2, 2020

The only system that Windows 10 has related to what you mentioned is SuperFetch (introduced back in Vista). Now called: SysMain, this feature load commonly used libraries and application components into memory before they are required based on some fancy system analytics it performs based on your usage.

Personally, I don't know how much storage space you have and actually need, but I would say, nothing beats pure SSD setup. And I would put the 30$ in your savings to get an appropriate high performance SSD later on. SSD prices drops rapidly (ok well, now with the current situation everything is a mess in pricing and depending on your region things might have skyrocketed, but ignoring this situation as things should be back to normal by the end of the year in terms of manufacturing and product demands).

That said, an SSD won't be as fast as RAM, if you are using that feature. The latest and fastest consumer SSD (PCI-E 4.0 M.2 NVMe drive), matches the speed of your typical DDR2 ~400MHz RAM, if my calculations were correct.

Personally, I am not a fan of RAM Cache, due to lack of reliability (power loss -> data loss), and the fact that you need to wait for it to move data from disk to RAM in any case, so you pass through the load time in any case... well, I guess it is hidden a bit as you get to do other things meanwhile and not wait. Typically games load time don't benefit much from faster storage beyond a certain point, like an SSD, as big portion is passing texture from RAM to the GPU memory, loading meshes/models and shader compiling, and not disk speed. To clarify: there is a notable difference between HDD and SSD, and SSD will accelerate object/texture load time during runtime of a game session (affects mostly open world games). But a decently fast SSD to the fastest SSD given the same computer specs, the margins are typically negligible.

But if your game(s) benefits from RAM speed, then sure, great!

Skyyblaze · August 4, 2020

On 8/2/2020 at 4:37 AM, GoodBytes said:

The only system that Windows 10 has related to what you mentioned is SuperFetch (introduced back in Vista). Now called: SysMain, this feature load commonly used libraries and application components into memory before they are required based on some fancy system analytics it performs based on your usage.

Personally, I don't know how much storage space you have and actually need, but I would say, nothing beats pure SSD setup. And I would put the 30$ in your savings to get an appropriate high performance SSD later on. SSD prices drops rapidly (ok well, now with the current situation everything is a mess in pricing and depending on your region things might have skyrocketed, but ignoring this situation as things should be back to normal by the end of the year in terms of manufacturing and product demands).

That said, an SSD won't be as fast as RAM, if you are using that feature. The latest and fastest consumer SSD (PCI-E 4.0 M.2 NVMe drive), matches the speed of your typical DDR2 ~400MHz RAM, if my calculations were correct.

Personally, I am not a fan of RAM Cache, due to lack of reliability (power loss -> data loss), and the fact that you need to wait for it to move data from disk to RAM in any case, so you pass through the load time in any case... well, I guess it is hidden a bit as you get to do other things meanwhile and not wait. Typically games load time don't benefit much from faster storage beyond a certain point, like an SSD, as big portion is passing texture from RAM to the GPU memory, loading meshes/models and shader compiling, and not disk speed. To clarify: there is a notable difference between HDD and SSD, and SSD will accelerate object/texture load time during runtime of a game session (affects mostly open world games). But a decently fast SSD to the fastest SSD given the same computer specs, the margins are typically negligible.

But if your game(s) benefits from RAM speed, then sure, great!

Thanks for the reply! I actually found the feature now, it's called Windows Storage Spaces and is able to use a SSD sort of like Intel Optane to boost performance of a mechanical drive with a Solid State Cache. This feature is also available in Windows 10 Pro but from what I got you only get the GUI to configure it in Windows 10 Server, in all other editions you have to go with PowerShell commands. Still Storage Spaces is no option for me in the end because it requires all related drives to be formatted and formatting my 6tb of gaming storage would be a logistical nightmare for me so I rather plonk down the 30€

You are right that nothing beats a pure SSD setup and realistically I could do with just 2-3TB of storage for games but from past experience I always buy double the space I need no matter if it's PCs or phones, that way I don't have to manage space and always have reserves. Though getting 6tb of SSD storage still isn't all that cheap, especially not considering that I save money for a Playstation 5 and a new GPU for my PC as my GTX 1070, while still performing great, shows its age in terms of features, I really want to get on the DLSS train too.

I actually found that using a small SSD as a cache for a HDD works very well, in the trial month of me using Primocache my load-times in games I play often, often halved if not more and technically it's a far more efficient use of SSD space. Let's take Metal Gear Rising or Final Fantasy XIII as examples, I don't have the exact sizes on hand but these games are around 10gb actual game files and 30gb BluRay quality video-cutscenes. If I install these games completely on a SSD I essentially waste 20-30gb because loading video files from a SSD will have no benefit for the game.

Sure I could now manually manage things and do Junctions but that's a hassle and here methods like Windows Storage Spaces or Primocache come in, they analyze the most frequently accessed files on a drive you assign and then mirror these files to the SSD cache and tell the game to load them from there in the background. That way you don't have to install a game completely on a SSD and save space but still get the majority of the speed benefit.

I think my biggest improvement was actually Anno 1800 where loading times are now a quarter of that when loading a save-game compared to when I wasn't using Primocache yet.

Here is a video that showcases the improvements in some other games:

(it's not my video but it gets the point across :P)

GoodBytes · August 4, 2020

3 hours ago, Skyyblaze said:

Thanks for the reply! I actually found the feature now, it's called Windows Storage Spaces and is able to use a SSD sort of like Intel Optane to boost performance of a mechanical drive with a Solid State Cache. This feature is also available in Windows 10 Pro but from what I got you only get the GUI to configure it in Windows 10 Server, in all other editions you have to go with PowerShell commands. Still Storage Spaces is no option for me in the end because it requires all related drives to be formatted and formatting my 6tb of gaming storage would be a logistical nightmare for me so I rather plonk down the 30€

No that is not what it does. You have no control of Storage Space, and it has no goal to act like a cache. What you experience is just a coincidence. You also have performance loss from using what is essentially software RAID in JBOD configuration.

Quote

You are right that nothing beats a pure SSD setup and realistically I could do with just 2-3TB of storage for games but from past experience I always buy double the space I need no matter if it's PCs or phones, that way I don't have to manage space and always have reserves. Though getting 6tb of SSD storage still isn't all that cheap,

Why are you keeping games you don't play anymore? And your logic on storage doubling is flawed as well, as your free space is exponential.

You don't need to backup Steam. They have their own backups.

Quote

I actually found that using a small SSD as a cache for a HDD works very well, in the trial month of me using Primocache my load-times in games I play often, often halved if not more and technically it's a far more efficient use of SSD space. Let's take Metal Gear Rising or Final Fantasy XIII as examples, I don't have the exact sizes on hand but these games are around 10gb actual game files and 30gb BluRay quality video-cutscenes. If I install these games completely on a SSD I essentially waste 20-30gb because loading video files from a SSD will have no benefit for the game.

So? Once you finish the game you...uninstall it.