SSD Software caching of RAID 5 array?

xBlizzDevious · February 27, 2015

Hi everyone,

Basically, I have a RAID 5 array on my RAID card and shortly (some time this evening, when I've built my new server), I'll have two spare 64GB SSDs.

- Is there a way of caching the RAID 5 array through software? If so - what do you recommend?

- Is it worth it? I store all sorts of stuff on there such as media, games (friends use my server to play games on) and VMs.

Thanks for your input!

Note - I'll be using Windows 7 x64 for compatibility.

MrBucket101 · February 27, 2015

Hi everyone,

Basically, I have a RAID 5 array on my RAID card and shortly (some time this evening, when I've built my new server), I'll have two spare 64GB SSDs.

- Is there a way of caching the RAID 5 array through software? If so - what do you recommend?

- Is it worth it? I store all sorts of stuff on there such as media, games (friends use my server to play games on) and VMs.

Thanks for your input!

Note - I'll be using Windows 7 x64 for compatibility.

Depends on your definition of software, I have SSD caching enabled on my array. LSI licenses it as "software" but the feature is included in the drive firmware, but disabled unless a valid key is provided. I bought a cheap "hardware key" for cachecade off eBay.

The speed increases I saw after turning on caching where incredible. Before my SSD (with write-back+BBU enabled), I was getting 800MB/s read, 600MB/s write, and now I'm getting 1.7GB/s read, and 1.2GB/s write. I'm not entirely sure how the trickery of it all works, especially given the SSD's I used topped out at like 450MB/s each. But the tests I ran with hdparm and dd, kept returning consistent results. So I'll take it. If you'd like more info, checkout the server link in my sig, and jump to updates 2 and 3.

You have to be careful with cache drives though, if a cache drive dies with data on it that hasn't been written to disk, then you've lost that data for good. RAID 1 pretty much eliminates that risk. Just some food for thought.

@AidanJMessenger recently posted this write up in the 10TB+ Storage thread. He is using FreeNAS, and ZFS raid to manage his array, and has a 64gb SSD enabled for caching. He is using WD Green drives, bit of a no-no as far as RAID goes, I don't imagine his speeds even with the SSD will be blazingly fast. But perhaps he can elaborate more on how he set everything up.

xBlizzDevious · February 27, 2015

Depends on your definition of software, I have SSD caching enabled on my array. LSI licenses it as "software" but the feature is included in the drive firmware, but disabled unless a valid key is provided. I bought a cheap "hardware key" for cachecade off eBay.

The speed increases I saw after turning on caching where incredible. Before my SSD (with write-back+BBU enabled), I was getting 800MB/s read, 600MB/s write, and now I'm getting 1.7GB/s read, and 1.2GB/s write. I'm not entirely sure how the trickery of it all works, especially given the SSD's I used topped out at like 450MB/s each. But the tests I ran with hdparm and dd, kept returning consistent results. So I'll take it. If you'd like more info, checkout the server link in my sig, and jump to updates 2 and 3.

You have to be careful with cache drives though, if a cache drive dies with data on it that hasn't been written to disk, then you've lost that data for good. RAID 1 pretty much eliminates that risk. Just some food for thought.

@AidanJMessenger recently posted this write up in the 10TB+ Storage thread. He is using FreeNAS, and ZFS raid to manage his array, and has a 64gb SSD enabled for caching. He is using WD Green drives, bit of a no-no as far as RAID goes, I don't imagine his speeds even with the SSD will be blazingly fast. But perhaps he can elaborate more on how he set everything up.

Thanks. The card I have doesn't have caching on board - as far as I'm aware (Highpoint RocketRAID 2720SGL) - so I would mean software within Windows. But thanks for the detailed help. I'll check out that thread and see what was set up.

AidanJMessenger · February 28, 2015

The way I've set up my caching is through freenas. Freenas offers the option to add an SSD or stripe of SSDs, or a write cache only SSD or read cache only SSD etc, to any of your Arrays. So that's what I chose to do. Given that you aren't using freenas, this isn't a viable option for you. I'm also obviously using Raid-Z, not Raid 5.

I also stated I don't know whether it gives me much more performance. I mean, I only have gigabit networking anyway, and the server easily saturates that. But I've always had the SSD cache, so I can't be sure of whether the SSD cache was helping out with that, and whether it would still saturate it with only my drives. But when I was resilvering the drives, it was doing so at 300MB/s, so it probably would.

One thing I know it probably is helping with though, is with the RAM cache. The way freenas works, is it often writes things to memory, then moves them onto the hard drive. This is obviously easy to lose in a power loss situation, and the fact I'm not using ECC memory can also be a problem, and of course this requires large amounts of ram to work properly. By using an SSD cache, this overrides Freenas' usual tendency to use RAM, (Though it still does take advantage of it, just not as much), so that would've definitely helped out my sister.

But anyway, that's just how I've sorted it and whatnot.

To answer your actual questions:

Is it worth it? Probably not. If you've only got gigabit networking, then you're limited by that 125MB/s anyway, which a RAID 5 can likely saturate anyway, given that you're using a RAID card. Now if you were doing a software RAID setup, which are notoriously slower, then yes it would likely help a lot.

When it comes to doing a software SSD cache of a hardware RAID setup, that could potentially complicate things, but I'm not sure.

On my sister's laptop, it has a 500GB hard drive, and a 24GB msata SSD which was configured as a cache drive from the factory. When I wiped the computer to install Windows 8 Enterprise on, I needed to somehow reconfigure the cache, which couldn't be done the way it was done stock for some reason. SO I had to find some software. I decided to use FancyCache. It worked very well.

Overall, you probably won't see a huge improvement, but if they're just spare SSDs, and you're not actually paying to add this SSD to your system, then you might as well use one as a cache, to gain a small amount of performance. I mean, browsing commonly used, smaller files could be a little quicker. But who knows.

Hopefully something in the large amount of writing was of some use

xBlizzDevious · February 28, 2015

The way I've set up my caching is through freenas. Freenas offers the option to add an SSD or stripe of SSDs, or a write cache only SSD or read cache only SSD etc, to any of your Arrays. So that's what I chose to do. Given that you aren't using freenas, this isn't a viable option for you. I'm also obviously using Raid-Z, not Raid 5.

I also stated I don't know whether it gives me much more performance. I mean, I only have gigabit networking anyway, and the server easily saturates that. But I've always had the SSD cache, so I can't be sure of whether the SSD cache was helping out with that, and whether it would still saturate it with only my drives. But when I was resilvering the drives, it was doing so at 300MB/s, so it probably would.

One thing I know it probably is helping with though, is with the RAM cache. The way freenas works, is it often writes things to memory, then moves them onto the hard drive. This is obviously easy to lose in a power loss situation, and the fact I'm not using ECC memory can also be a problem, and of course this requires large amounts of ram to work properly. By using an SSD cache, this overrides Freenas' usual tendency to use RAM, (Though it still does take advantage of it, just not as much), so that would've definitely helped out my sister.

But anyway, that's just how I've sorted it and whatnot.

To answer your actual questions:

Is it worth it? Probably not. If you've only got gigabit networking, then you're limited by that 125MB/s anyway, which a RAID 5 can likely saturate anyway, given that you're using a RAID card. Now if you were doing a software RAID setup, which are notoriously slower, then yes it would likely help a lot.

When it comes to doing a software SSD cache of a hardware RAID setup, that could potentially complicate things, but I'm not sure.

On my sister's laptop, it has a 500GB hard drive, and a 24GB msata SSD which was configured as a cache drive from the factory. When I wiped the computer to install Windows 8 Enterprise on, I needed to somehow reconfigure the cache, which couldn't be done the way it was done stock for some reason. SO I had to find some software. I decided to use FancyCache. It worked very well.

Overall, you probably won't see a huge improvement, but if they're just spare SSDs, and you're not actually paying to add this SSD to your system, then you might as well use one as a cache, to gain a small amount of performance. I mean, browsing commonly used, smaller files could be a little quicker. But who knows.

Hopefully something in the large amount of writing was of some use

Thanks. I'll give that FancyCache a go. And yeah, they're just spare SSDs as I've built a new server to house the RAID 5 array and the OS SSDs from my old server are no longer needed. I'll either sell them or use them.