Jump to content

Using an SSD as a cache for my 6TB Hard Drive

JumiElazul

Hey all,

 

So I've got a 2TB PCIe 4.0 SSD as my main boot drive, as well as a 6TB Western Digital Black for all my mechanical storage needs (mostly backlog games).  Just now have I considered the possibility of getting another SSD to use as a cache for this mechanical drive, but I've never done this before, so I'm wondering the best way to go about this.

 

I've got a Ryzen 9 3900x on the x570 platform so my understanding is I can use StoreMI to use a new SSD I purchase as the cache for my Western Digital drive.  My main questions are:

 

Does it matter how big the SSD is?  Will a 250 GB SSD be enough to cache a 6TB hard drive or will I need something bigger? 

Will there be any appreciable performance difference between a 2.5 inch SATA SSD and an NVMe SSD being used as the cache?

Link to comment
Share on other sites

Link to post
Share on other sites

StoreMI will move data between the HD and cache disk, so whatever remains on the cache disk will be higher performing. Depending on use case if 250GB is enough. As for performance, there will be a difference. Either are much faster than a HD, but if you care about the performance difference is again a user choice.

Gaming system: R7 7800X3D, Asus ROG Strix B650E-F Gaming Wifi, Thermalright Phantom Spirit 120 SE ARGB, Corsair Vengeance 2x 32GB 6000C30, RTX 4070, MSI MPG A850G, Fractal Design North, Samsung 990 Pro 2TB, Acer Predator XB241YU 24" 1440p 144Hz G-Sync + HP LP2475w 24" 1200p 60Hz wide gamut
Productivity system: i9-7980XE, Asus X299 TUF mark 2, Noctua D15, 64GB ram (mixed), RTX 3070, NZXT E850, GameMax Abyss, Samsung 980 Pro 2TB, random 1080p + 720p displays.
Gaming laptop: Lenovo Legion 5, 5800H, RTX 3070, Kingston DDR4 3200C22 2x16GB 2Rx8, Kingston Fury Renegade 1TB + Crucial P1 1TB SSD, 165 Hz IPS 1080p G-Sync Compatible

Link to comment
Share on other sites

Link to post
Share on other sites

If it's for games you play every once in a while, it won't really be worth it. Cache stores the files that are used most frequently...so if you're not using any of those items regularly, then nothing will really be cached properly. It would make more sense to just get an SSD and load the games you plan on playing directly to it.

CPU: Ryzen 9 5900 Cooler: EVGA CLC280 Motherboard: Gigabyte B550i Pro AX RAM: Kingston Hyper X 32GB 3200mhz

Storage: WD 750 SE 500GB, WD 730 SE 1TB GPU: EVGA RTX 3070 Ti PSU: Corsair SF750 Case: Streacom DA2

Monitor: LG 27GL83B Mouse: Razer Basilisk V2 Keyboard: G.Skill KM780 Cherry MX Red Speakers: Mackie CR5BT

 

MiniPC - Sold for $100 Profit

Spoiler

CPU: Intel i3 4160 Cooler: Integrated Motherboard: Integrated

RAM: G.Skill RipJaws 16GB DDR3 Storage: Transcend MSA370 128GB GPU: Intel 4400 Graphics

PSU: Integrated Case: Shuttle XPC Slim

Monitor: LG 29WK500 Mouse: G.Skill MX780 Keyboard: G.Skill KM780 Cherry MX Red

 

Budget Rig 1 - Sold For $750 Profit

Spoiler

CPU: Intel i5 7600k Cooler: CryOrig H7 Motherboard: MSI Z270 M5

RAM: Crucial LPX 16GB DDR4 Storage: Intel S3510 800GB GPU: Nvidia GTX 980

PSU: Corsair CX650M Case: EVGA DG73

Monitor: LG 29WK500 Mouse: G.Skill MX780 Keyboard: G.Skill KM780 Cherry MX Red

 

OG Gaming Rig - Gone

Spoiler

 

CPU: Intel i5 4690k Cooler: Corsair H100i V2 Motherboard: MSI Z97i AC ITX

RAM: Crucial Ballistix 16GB DDR3 Storage: Kingston Fury 240GB GPU: Asus Strix GTX 970

PSU: Thermaltake TR2 Case: Phanteks Enthoo Evolv ITX

Monitor: Dell P2214H x2 Mouse: Logitech MX Master Keyboard: G.Skill KM780 Cherry MX Red

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

Ah I see.

 

The reason I thought to get a smaller SSD as a cache drive instead of another full size SSD was I have some games on the mechanical drive that are very large (100+ GB for a single game) and that even though I don't play them often, when I do play them I will play them for a couple of weeks at a time.  In this way I thought getting a cache will make more sense because I can keep my large amount of storage and get a reasonable SSD for about $80.00 to $100.00 rather than paying a couple of hundred for more space.  I do use the drive frequently, I tend to just not put my "high importance" games on there to not waste my main SSD space if that makes sense.

 

Does this line of thinking make sense?  Or is it still better to just fork more money out for another 1 or 2 TB NVMe drive.

Link to comment
Share on other sites

Link to post
Share on other sites

If you're on Windows you can use Storage Spaces. This is tiering (not the same as caching) whereby a heat map is created and the most-accessed files will be put in the faster SSD tier while less-accessed will go to cold storage (HDD tier).

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, NewMaxx said:

If you're on Windows you can use Storage Spaces. This is tiering (not the same as caching) whereby a heat map is created and the most-accessed files will be put in the faster SSD tier while less-accessed will go to cold storage (HDD tier).

Just curious, I have been researching this solution but do you need to format the drives before adding them to the 'pool'? Every guide I've found so far assumes these are brand new unformatted drives...

 

I have a ssd, used for Windows and commonly used games, a HDD and an external HDD...I'm wanting to combine all 3 drives and use the SSD as a 'cache' drive. I do currently have a lot of applications and games I don't want to have to reinstall though if I do have to format...

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, steelo said:

Just curious, I have been researching this solution but do you need to format the drives before adding them to the 'pool'? Every guide I've found so far assumes these are brand new unformatted drives...

I have the same question now :D

Link to comment
Share on other sites

Link to post
Share on other sites

13 minutes ago, steelo said:

Just curious, I have been researching this solution but do you need to format the drives before adding them to the 'pool'? Every guide I've found so far assumes these are brand new unformatted drives...

 

I have a ssd, used for Windows and commonly used games, a HDD and an external HDD...I'm wanting to combine all 3 drives and use the SSD as a 'cache' drive. I do currently have a lot of applications and games I don't want to have to reinstall though if I do have to format...

Creation of the pool will format the drives.

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, JumiElazul said:

Hey all,

 

So I've got a 2TB PCIe 4.0 SSD as my main boot drive, as well as a 6TB Western Digital Black for all my mechanical storage needs (mostly backlog games).  Just now have I considered the possibility of getting another SSD to use as a cache for this mechanical drive, but I've never done this before, so I'm wondering the best way to go about this.

 

I've got a Ryzen 9 3900x on the x570 platform so my understanding is I can use StoreMI to use a new SSD I purchase as the cache for my Western Digital drive.  My main questions are:

 

Does it matter how big the SSD is?  Will a 250 GB SSD be enough to cache a 6TB hard drive or will I need something bigger? 

Will there be any appreciable performance difference between a 2.5 inch SATA SSD and an NVMe SSD being used as the cache?

Use PrimoCache. I've personally done it before with it and it's amazing. Zero migration, zero data loss on cache drive failure, super configurable, very affordable and I have very good experience with their support as well. 250GB SSD cache should generally be enough, but given how cheap SSD's are these days, I'd go with at least 512GB. Or 1TB if price isn't too much already for you. The thing is, PrimoCache caches everything. It'll also cache game files, large textures and stuff which, if you play several huge games at once, it'll populate the cache. And if it runs out of cache, it'll start discharging oldest cached data that hasn't been accessed for a while. So it's smart to strike a good balance between cache size and actual storage drive. I've used 256GB SSD with 2TB HDD and it worked well.

 

I'd avoid StoreIM or Intel's RST solutions because they are volatile and when things go wrong you're losing whole drives of data. That's the last thing you want. With PrimoCache, there are no such problems and you can discard caching at any time. You also have full control over caching and how it's done, how aggressive it is, how it's performed etc.

 

Another option is eBoostr. Old app meant for Windows XP to replace ReadyBoost, but I found it can work as a caching app for SSD+HDD combos. And last time I checked it, it worked even under Windows 10 64bit despite not claiming it does. It also works in a different way. PrimoCache works on a block level where eBoostr works on file level. Meaning you actually have full control over what it is allowed to cache and what isn't. You can set it to exclude certain file types from caching, like movies which are usually pretty big and don't need to be cached to be usable. PrimoCache doesn't distinguish between files, it just registers what blocks on HDD are accessed regularly and it'll cache that. Or you can set what apps you want it to forcibly cache no matter what.

 

Both apps offer trial versions so check both and see which one works for you. Cost wise, they are similar, around 30€ or $.

Link to comment
Share on other sites

Link to post
Share on other sites

30 minutes ago, RejZoR said:

Use PrimoCache. I've personally done it before with it and it's amazing. Zero migration, zero data loss on cache drive failure, super configurable, very affordable and I have very good experience with their support as well. 250GB SSD cache should generally be enough, but given how cheap SSD's are these days, I'd go with at least 512GB. Or 1TB if price isn't too much already for you. The thing is, PrimoCache caches everything. It'll also cache game files, large textures and stuff which, if you play several huge games at once, it'll populate the cache. And if it runs out of cache, it'll start discharging oldest cached data that hasn't been accessed for a while. So it's smart to strike a good balance between cache size and actual storage drive. I've used 256GB SSD with 2TB HDD and it worked well.

That sounds like the exact solution I need!  I love that you mention texture caching, because some of my Hard Drive games are 100 GB+ specifically because of high res texture mods.

 

I'm definitely going to look into PrimoCache.  Thanks!

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, JumiElazul said:

That sounds like the exact solution I need!  I love that you mention texture caching, because some of my Hard Drive games are 100 GB+ specifically because of high res texture mods.

 

I'm definitely going to look into PrimoCache.  Thanks!

Although total texture size might be 100+ GB, it doesn't mean it'll cache them all at once however. Texture files are usually accessed by games in blocks too. Texture file might have 5GB, but if game only needs a 150MB of textures from it regularly, PrimoCache will only cache those blocks.

 

You'll notice PrimoCache working if your HDD is loud and you can clearly hear its grinding now. With caching, it'll gradually become almost as quiet as SSD because most of regularly accessed stuff will be fetched from SSD most of the time. This is usually most noticeable during Windows boot, but you'll also see it in games and programs. I've set it up for my cousin and his OS partition had 90% hit ratio last time I've checked it. That means 90% of all read data was actually fetched from SSD and not HDD. His partition with games had over 70% hit ratio.

 

There was also one interesting instance back with NFS 2015. A lot of users had problems of falling through the map because HDD's allegedly couldn't fetch map data fast enough for game to render it, causing players falling through map parts that did't exist because HDD wasn't able to fetch them fast enough. I never had that issue because I was using SSD caching.

 

I've been on 2TB SSD exclusively for several years and now I've stick 8TB HDD again in my system for bulk data. And I'm actually thinking of doing hybrid SSD+HDD and have 8TB of storage that basically behaves like SSD again. Maybe... Anyway, yeah, it's a really great way to have massive storage with really high speeds and it's one of most underrated and ignored methods.

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, RejZoR said:

Although total texture size might be 100+ GB, it doesn't mean it'll cache them all at once however. Texture files are usually accessed by games in blocks too. Texture file might have 5GB, but if game only needs a 150MB of textures from it regularly, PrimoCache will only cache those blocks.

 

You'll notice PrimoCache working if your HDD is loud and you can clearly hear its grinding now. With caching, it'll gradually become almost as quiet as SSD because most of regularly accessed stuff will be fetched from SSD most of the time. This is usually most noticeable during Windows boot, but you'll also see it in games and programs. I've set it up for my cousin and his OS partition had 90% hit ratio last time I've checked it. That means 90% of all read data was actually fetched from SSD and not HDD. His partition with games had over 70% hit ratio.

 

There was also one interesting instance back with NFS 2015. A lot of users had problems of falling through the map because HDD's allegedly couldn't fetch map data fast enough for game to render it, causing players falling through map parts that did't exist because HDD wasn't able to fetch them fast enough. I never had that issue because I was using SSD caching.

 

I've been on 2TB SSD exclusively for several years and now I've stick 8TB HDD again in my system for bulk data. And I'm actually thinking of doing hybrid SSD+HDD and have 8TB of storage that basically behaves like SSD again. Maybe... Anyway, yeah, it's a really great way to have massive storage with really high speeds and it's one of most underrated and ignored methods.

Dont mean to hijack this thread and I hope it helps the OP...Do you have to format your drives initially to use it? That's the big problem I have...I have tons of data I dont want to have to migrate. Also,  I have a small 240gb ssd, mainly as a boot drive. I also have a 500gb hdd and a 1 TB external hdd. Will this method of drive caching substantially increase boot time? Thanks

Link to comment
Share on other sites

Link to post
Share on other sites

4 minutes ago, steelo said:

Dont mean to hijack this thread and I hope it helps th OP...Do you have to format your drives initially to use it? That's the big problem I have...I have tons of data I dont want to have to migrate. Also,  I have a small 240gb ssd, mainly as a boot drive. I also have a 500gb hdd and a 1 TB external hdd. Will this method substantially increase boot time? Thanks

According to their webpage:

 

"No need to reinstall Windows or applications, no need to migrate documents or data, no need to change computer infrastructure..."

 

Haven't started using it yet, but it appears it will work without needing to move or re-install stuff!

Link to comment
Share on other sites

Link to post
Share on other sites

4 minutes ago, JumiElazul said:

According to their webpage:

 

"No need to reinstall Windows or applications, no need to migrate documents or data, no need to change computer infrastructure..."

 

Haven't started using it yet, but it appears it will work without needing to move or re-install stuff!

Awesome! I may just try this tomorrow but honestly, I would consider backing up very important files first! Hopefully one is able to revert back if they dont like it.

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, steelo said:

Dont mean to hijack this thread and I hope it helps the OP...Do you have to format your drives initially to use it? That's the big problem I have...I have tons of data I dont want to have to migrate. Also,  I have a small 240gb ssd, mainly as a boot drive. I also have a 500gb hdd and a 1 TB external hdd. Will this method of drive caching substantially increase boot time? Thanks

Not at all. Ask all you're interested in. Hybrid storage is very cool, it's just unfortunate a lot of people don't know it or avoid it because of rubbish scare misinformation people spread about it. And crappy SSHD's that had useless 8GB of cache that helped very little. If SSHD's had 32GB caches back in the day and evolved to 256GB today on bigger drives, they'd be awesome. Instead vendors basically killed them themselves because they made badly designed SSHD's with underwhelming caches.

 

Back to your question, PrimoCache can be added or removed at ANY time without having to format or manipulate your main drive. Which makes it ideal for upgrade of existing system. So you have only 4TB HDD in your system? No probem. Buy PrimoCache, buy SSD, install the SSD, install PrimoCache, set which drive you want to cache using what SSD (if you have more of them), decide how much of SSD you want to use and then you just use your system like you would normally. PrimoCache will do the magic under the hood without any of your input, constantly adapting to your system usage. What you run regularly will be cached, what you don't use will be discarded from cache. The bigger your SSD cache is, the more it'll be able to cache without having to discard anything. Your system will become quieter and it'll start to feel more like you're using SSD than using HDD.

 

If SSD fails for whatever reason, your system will just continue to behave like it has only HDD again. You won't lose any data. You can also remove PrimoCache for whatever reason at any point without having to do anything to HDD like formatting it or unpairing it like you have to with StoreMI and Intel RST. You just remove the SSD cache in PrimoCache and system will instantly fall back into its normal HDD only operation. It's why I love PrimoCache so much. It makes things WAAAAAY faster and is also convenient at the same time. And doesn't really have any serious annoyances like other such apps that do require you to manipulate your main storage drive or don't allow you to customize how they behave. PrimoCache gives you almost full control over all that.

 

As for your exact situation, I'd pair that 240GB SSD to your 500GB internal drive and use 1TB external drive as cold storage for things like movies and stuff that you probably already use for. Since you already use SSD for boot drive, you'll have to format it and install Windows on your 500GB HDD unfortunately. Then install PrimoCache on HDD too and set it up to use that 240GB SSD for caching. And that's it. PrimoCache will do the magic by itself after that. It's a bit easier if you already only have Windows installed on HDD and you just add empty SSD later for caching. But I can tell you, it'll be worth it. You'll have same boot times as on SSD and all the apps you use regularly like browser and stuff will fire up instantly like on SSD. But you'll have 500GB to use and it'll seamlessly behave like you have 500GB SSD. Though I'd suggest you upgrade to a larger HDD at this point so you really gain some space and speed at the same time.

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, RejZoR said:

Not at all. Ask all you're interested in. Hybrid storage is very cool, it's just unfortunate a lot of people don't know it or avoid it because of rubbish scare misinformation people spread about it. And crappy SSHD's that had useless 8GB of cache that helped very little. If SSHD's had 32GB caches back in the day and evolved to 256GB today on bigger drives, they'd be awesome. Instead vendors basically killed them themselves because they made badly designed SSHD's with underwhelming caches.

 

Back to your question, PrimoCache can be added or removed at ANY time without having to format or manipulate your main drive. Which makes it ideal for upgrade of existing system. So you have only 4TB HDD in your system? No probem. Buy PrimoCache, buy SSD, install the SSD, install PrimoCache, set which drive you want to cache using what SSD (if you have more of them), decide how much of SSD you want to use and then you just use your system like you would normally. PrimoCache will do the magic under the hood without any of your input, constantly adapting to your system usage. What you run regularly will be cached, what you don't use will be discarded from cache. The bigger your SSD cache is, the more it'll be able to cache without having to discard anything. Your system will become quieter and it'll start to feel more like you're using SSD than using HDD.

 

If SSD fails for whatever reason, your system will just continue to behave like it has only HDD again. You won't lose any data. You can also remove PrimoCache for whatever reason at any point without having to do anything to HDD like formatting it or unpairing it like you have to with StoreMI and Intel RST. You just remove the SSD cache in PrimoCache and system will instantly fall back into its normal HDD only operation. It's why I love PrimoCache so much. It makes things WAAAAAY faster and is also convenient at the same time. And doesn't really have any serious annoyances like other such apps that do require you to manipulate your main storage drive or don't allow you to customize how they behave. PrimoCache gives you almost full control over all that.

 

As for your exact situation, I'd pair that 240GB SSD to your 500GB internal drive and use 1TB external drive as cold storage for things like movies and stuff that you probably already use for. Since you already use SSD for boot drive, you'll have to format it and install Windows on your 500GB HDD unfortunately. Then install PrimoCache on HDD too and set it up to use that 240GB SSD for caching. And that's it. PrimoCache will do the magic by itself after that. It's a bit easier if you already only have Windows installed on HDD and you just add empty SSD later for caching. But I can tell you, it'll be worth it. You'll have same boot times as on SSD and all the apps you use regularly like browser and stuff will fire up instantly like on SSD. But you'll have 500GB to use and it'll seamlessly behave like you have 500GB SSD. Though I'd suggest you upgrade to a larger HDD at this point so you really gain some space and speed at the same time.

Awesome...I may just do that this weekend. I'm also considering 'converting' the external HDD to an internal HDD. I assume it will behave the same as any other HDD at that point.

Link to comment
Share on other sites

Link to post
Share on other sites

@steelo

It depends. Years ago, they just used regular SATA drives with extra USB interface stuck on them. Nowadays, a lot of drives have USB interface as integral part of the drive (probably coz it's cheaper than having SATA and USB together, meaning you can't take it out of the enclosure and stick it in a PC as internal SATA drive.

 

I had such idea to cannibalize external drive and get huge HDD on the cheap since external ones are way cheaper. But then I watched a repair video of one WD external drive and when the guy took it out, it had no SATA interface, just microUSB sticking directly out of it. You'd have to carefully open it and if it doesn't have SATA, put it back together.

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, RejZoR said:

@steelo

It depends. Years ago, they just used regular SATA drives with extra USB interface stuck on them. Nowadays, a lot of drives have USB interface as integral part of the drive (probably coz it's cheaper than having SATA and USB together, meaning you can't take it out of the enclosure and stick it in a PC as internal SATA drive.

 

I had such idea to cannibalize external drive and get huge HDD on the cheap since external ones are way cheaper. But then I watched a repair video of one WD external drive and when the guy took it out, it had no SATA interface, just microUSB sticking directly out of it. You'd have to carefully open it and if it doesn't have SATA, put it back together.

I cannibalized an external HDD a few years ago and it wasn't difficult. However, like you said I have no idea whether this one has a SATA connector or not. I might just do this the right way and buy a 1 TB HDD...I do love the idea of consolidating the 3-4 drives into one and using my SSD as a cache drive.

Link to comment
Share on other sites

Link to post
Share on other sites

@RejZoR

 

Alright.  So today I went out and grabbed myself a shiny new WD Black SN750 1 TB SSD and a license for PrimoCache.  I have set everything up and it seems to work amazing.  Now I'm a little hazy on some of these settings and how they will be affecting performance.  I figure since you've used it before I would ask for your input on some settings :D.

 

So it's a 1TB SSD and a 6TB Hard drive that I am accelerating, with 32GB of System RAM.  I've currently set it up to improve Read/Write, with Defer-Write off (as in the research I've done it seems unsafe for my data without a fail safe on my power supply).

 

How does the level-1 Cache setting using system RAM affect performance?  What would be an optimal settings for 32GB of RAM?  I'm unsure as to what will change if I use 4GB of my RAM for Caching vs something much higher like 16GB.

 

Furthermore, how does the Block Size affect the cache?  It defaulted to 128KB and I've left it as such, but I'm wondering if a lower or higher setting may be better.

 

I've tried to do some reading to figure out what I should be setting these at but I'm a little out of my depth here.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

@JumiElazul

 

Awesome :)

 

Like you've guessed it, Deferred Writing is best kept off or if you do use it, set it in such a way that it writes data immediately, but with modifiers (you can even force it to write data in a faster manner than Windows write caching does). What it does is change the method of writing to HDD. Because SSD caching only works for read operations, they offer additional "boosting" functions for writing too. This only really comes in use if you're doing really heavy writing on your disk so it quickly grabs the write requests and stores them in RAM and gives HDD some breathing space even if it can't write with such speed. Program that requested writing will think data was already written, but in reality it'll be waiting in RAM to be actually written and is going to be managed by PrimoCache.

 

As for Level-1 caching, I don't think it's needed. I had it enabled for a while and couldn't really notice any difference. Basically what Level-1 cache does is act as another layer of caching. When read request is done, PrimoCache first checks Level-1 cache (RAM cache) and if it finds required data, it reads it from there). If it's not found there, it checks Level-2 cache (SSD cache) and reads it from there. If data is still not found, it reads it from Level-3 pool which is the regular HDD. Since you have 32GB of RAM and you probably don't ever use all of it, you can dedicate like 8GB of it easily and it'll use these 8GB for ultra fast caching on top of SSD caching. You can play with it, but for me it didn't really make a difference because I had super fast SSD's as Level-2 cache and they served all the data super fast anyway. I guess this can be used for realyl cheap SSD's that are faster than HDD, but still not optimal fast like high end SSD's. But even really cheap SATA SSD's today are saturating SATA6 easily so it's not really an issue.

 

As for block size, while it affects speed somewhat, it affects the overhead the most. As you're adjusting it, PrimoCache calculates the overhead for Level-1 and Level-2 cache. Check those values. What they mean is at small block sizes it'll have a more granular caching, but will create a lot of overhead, meaning you'll actually have less usable cache space. With large blocks, PrimoCache will have less granular control over cache, but will have less overhead an more cache space to use. I don't remember exactly, but as you go through block sizes, there are like two values that are sort of a sweet spot where you don't create much overhead, but still have relatively small block size. I did some testing with ATTO back then and it did show some difference in speed, but the overhead factor played a bigger role (overhead essentially wastes your cache capacity and setting 100GB SSD cache isn't really 100GB in the end, it'll be less than that, depending on block size you pick).

 

Hope it helps :) Btw, as you'll be using the system, PrimoCache will automatically decide what to cache and you can see the hit ratio graph in its main program window. The higher the hit rate, the more system is reading from the SSD cache instead of slower HDD. Hit rates of over 75% are not really unusual which means 3/4 of the data is being read from SSD instead of HDD. Which means the system will be freaking fast :D It's really a shame people refuse to accept and use hybrid storage setups because they are quite affordable way of having super fast and super large storage. You'll now essentially have a 6TB SSD. SSD's this big don't even exist for consumers and if they did, they'd cost around 800€. So spending some cash on SSD for caching and PrimoCache will set you back like 200€ and that's it. Quite a difference and you still get the speed.

 

Let me know over some time how it behaves and if you like it :)

Link to comment
Share on other sites

Link to post
Share on other sites

Thanks, that really helps a lot!  I appreciate all the help and detail you've put into explaining, I think you've given me the perfect solution for my setup.

 

Time to play with the settings a bit now that I understand a bit more what I'm aiming for :D

Link to comment
Share on other sites

Link to post
Share on other sites

I'll only talk about block size. There's two lines of thinking. First, you want to go for the typical 4KB block size, because this matches general cluster size and also how SSD mapping is arranged. That is to say, 32-bit (4B) addressing for 4KB clusters, therefore 1B of overhead per 4KB. Yes, a lot of times sector size is 512B ("512e"), but generally you're looking at 4KB ("4Kn") for optimal performance. On the other hand, with many HDD systems you would tend to stripe to 128KB for cold storage. Of course in reality your TLC SSD probably writes 16KB pages (16x3 = 48KB requests in TLC). So anywhere in the 4KB-128KB range is fine, although for best performance you would want to match cluster size (4KB). Your SSD does its own mapping and the OS does filesystem-level caching so this is more about how you want to balance the performance.

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 years later...

I like PrimoCacle, I just wish we had an OpenSource alternative to it, so Linux average-consumer users could also use it, or a cheaper version which allows for only one L1 and one L2 cache at a time, which more people could afford.

 

What I think most people tend to overlook, is the possibility to use this with a desktop or with a laptop with multiple free PCI lanes to connect a bunch of SSDs in Raid0, then use the resulted storage for caching other drives. For example, let's say you have a 4TB HDD, and a 512 GB SSD with your system on it, then you raid-0 some 3-4 SSDs with 256GB each. You can then use that 256GB, which now has read speeds almost equal to the sum of the read speeds of each of those 3-4 drives, to cache both the SSD and the HDD, which will result in the cached parts being read at speeds almost equal to that of the sum of the read speed of the raid-0 cache and the read speed of the drive the files originate from (i.e. the 512SSD or 4TB HDD). 
 

Honestly, laptop manufacturers could use a 256GB or 512GB SSD for the OS, and another SSD of 256GB or 128GB or even 64GB (since those should be cheaper) to have the pagefile on as well as to cache the other SSD, to make the whole PC feel faster, while also expanding the pagefile to make the system behave better if it needs to use more RAM than it has (because a pagefile larger than the amount of physical RAM means the difference is Virtual (RAM) Memory). So, with 4GB of real/physical RAM, 512GB usable storage for Windows and everything else, and a 64GB or 32GB SSD (split into 20% unallocated to increase the lifetime of the SSD and maintain it's peak speeds, 16GB of Virtual RAM partition for the pagefile, and the rest used to cache the other SSD) to speed up the system, then give this kind of setup a fancy name, and most people will only benefit from that. Bonus points for user-swappable caching SSDs (without opening the main part of the laptop or desktop), with a driver which detects the size of the SSD, and lets you choose how much of the SSD to use as Virtual RAM and allocates the rest up to 80% of the SSD to caching the other SSD.

Now, this pushes things a bit, but a 256GB SSD 20% (51.2 GB) unallocated and another 20% used to cache the HDD, then the rest of the SSD used for the Windows partition with a pagefile which to expand the 4GB of physical/real RAM into 16GB of total ram which means 12GB of Virtual RAM, then optionally moving the user files (where people download things by default), both program files folders, the programdata folder, and the appdata folder, onto the 512GB or 1TB HDD, and bamboozle people with an 768GB SSD+ sticker or 1.25 TB SSD+ sticker, and most people won't have a problem with that. 

And on ultra-cheap devices, having a relatively small eMMC drive (i.e. 64GB) or used to hardware-level cache an array of SD cards (which are much cheaper) in raid (just not raid-0, so people can swap one, if needed, ideally). For example, an eMMC of 64GB costs roughly $50, but for the same price you could get 5 microSDXC cards of 64GB 100MB/s Class 10 each, resulting in a raid configuration with 1 parity drive having 400MB/s, which is the same as the eMMC, with the advantage that manufacturers can get them cheaper and also fit the option of swapping the SD cards in the future, to increase the storage and/or speed.

 

Also, there's also the option of having an eMMC hardware-level read-only caching the rest of the drives, and call it a SSD Accelerator feature of the laptop.

 

Leaving that aside, why isn't there an open-source alternative to PrimoCache yet, for L2 disk caching?

Edited by Sapioit
Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×