Raid 5 expandability in windows

[idiocracy]

While i am reading online that you shouldn't use raid 5, it however appears to be suitable for my use case. Sacrificing 1 drive for a bit of redundancy is about as much as i'd want to sacrifice.

Running 2x raid0 where one is a ssd cache drive for the other raid0, data isn't important, speed is. If i replace the 6 drives for 5x ssd, i won't need the cache drive any more.

 

But can i expand the storage capacity later on without doing anything else than install the drive, and add it to the pool, without losing any data?

[Xjph]

Quote

But can i expand the storage capacity later on without doing anything else than install the drive, and add it to the pool, without losing any data?

 

That is entirely dependent on what you're using to manage the array. Some RAID5 setups can be easily expanded, some can't.

 

Quote

Running 2x raid0 where one is a ssd cache drive for the other raid0, data isn't important, speed is.

Can you elaborate on what you mean by this? There is no "cache drive" in a RAID0 setup. Do you have RAID0, or a dual drive hybrid? Or... do you mean you have a RAID0 of SSDs which is serving as the cache for a RAID0 of HDDs in a double-RAID0 hybrid setup?

 

[idiocracy]

7 hours ago, Xjph said:

 

That is entirely dependent on what you're using to manage the array. Some RAID5 setups can be easily expanded, some can't.

 

Can you elaborate on what you mean by this? There is no "cache drive" in a RAID0 setup. Do you have RAID0, or a dual drive hybrid? Or... do you mean you have a RAID0 of SSDs which is serving as the cache for a RAID0 of HDDs in a double-RAID0 hybrid setup?

 

 

It's a double raid0 as you mention. I manage the array through windows, both raid0 are entirely windows run, this has worked fine for years. The hdd raid0 limit is they are not handling lots of 4k random read very well. So i am thinking about switching for only ssd. And was thinking raid5.

[cmndr]

What is your actual use case?
Is it playing games? If it is primocache, an optane stick (or even just your existing SSDs) and RAID 5 on your HDDs is probably fine.

 

Expanding RAID5 is generally not a thing on windows, or at the minimum it's finicky. If needed you can just make a second RAID array or consider dumping your data somewhere else and then expanding/rebuilding the array. 

 

 

One thing to be aware of if you're trying to do SSD caching.... you generally don't need much cache (assuming you're not running VMs or databases and your set up has a good prioritization system - primocache does, so does ZFS). HDD performance falls apart when reading/writing lots of small blocks of data. These small blocks in practice might only take up a few dozen GB of data size. A lot of people think that you need WAY more cache than you actually do. Only a very small chunk of data is "hot" and performance sensitive. 

 

 

One other thing to consider - get a NAS. My own set up is 4x4TB HDDs (RAID Z1, similar to RAID 5), 118GB of optane as L2 ARC (cache) and 32GB RAM as cache. Virtually all reads from the array come from the RAM (first 32GB of cache) and a very tiny chunk come from the next 118GB of cache. It's generally snappier than an internal nvme drive.

 

 

55 minutes ago, idiocracy said:

It's a double raid0 as you mention. I manage the array through windows, both raid0 are entirely windows run, this has worked fine for years. The hdd raid0 limit is they are not handling lots of 4k random read very well. So i am thinking about switching for only ssd. And was thinking raid5.

What's the use case? Is a large chunk of your data "cold" that's unlikely to be read for a long time? Can you group your data so that anything needing speed just lives on its own SSD and low performance stuff lives in a slower pool?

 

 

[idiocracy]

1 hour ago, cmndr said:

What is your actual use case?
Is it playing games? If it is primocache, an optane stick (or even just your existing SSDs) and RAID 5 on your HDDs is probably fine.

 

Expanding RAID5 is generally not a thing on windows, or at the minimum it's finicky. If needed you can just make a second RAID array or consider dumping your data somewhere else and then expanding/rebuilding the array. 

 

 

One thing to be aware of if you're trying to do SSD caching.... you generally don't need much cache (assuming you're not running VMs or databases and your set up has a good prioritization system - primocache does, so does ZFS). HDD performance falls apart when reading/writing lots of small blocks of data. These small blocks in practice might only take up a few dozen GB of data size. A lot of people think that you need WAY more cache than you actually do. Only a very small chunk of data is "hot" and performance sensitive. 

 

 

One other thing to consider - get a NAS. My own set up is 4x4TB HDDs (RAID Z1, similar to RAID 5), 118GB of optane as L2 ARC (cache) and 32GB RAM as cache. Virtually all reads from the array come from the RAM (first 32GB of cache) and a very tiny chunk come from the next 118GB of cache. It's generally snappier than an internal nvme drive.

 

 

What's the use case? Is a large chunk of your data "cold" that's unlikely to be read for a long time? Can you group your data so that anything needing speed just lives on its own SSD and low performance stuff lives in a slower pool?

 

 

 

It's a homebuild NAS kinda machine, the families central media hub i suppose. It's a 2400G with 16gb ram, I have 1 ssd for OS, Then 4x 2TB hdd for storage, and 2x250gb ssd as cache for the hdd. Because the hdd just dies instantly if the ssd's aren't there.

It does file hosting, media hosting, streaming, conversion, transcoding, basically a bunch of on demand stuff. Which is why a read cache doesn't really work as many files aren't repeat access, and it'd need to be very big.

 

Most work fine as is, the performance only tanks during smaller reads which there's also a bunch of.
And that's why i'm thinking about replacing the current 6 drives, with 5/6x 2TB ssd drives.
The reason i was interested in raid5 was because i was hoping i could expand without loosing data. If that is not the case i think i'll settle for a 5x ssd raid0, that should sort any performance issues i've had. Yes i lose data if a drive dies, but that is manageable.

 

Besides that i'm also thinking about upgrading to a 5300G if i can get my hands on one.

[cmndr]

What kind of caching is being used for that? Is it AMD's STOREMI 1.0 where there's a tiering system? Is it something else that's more janky?

 

My 4 core Excavator (about half as fast CPU wise) NAS with TRUENAS SCALE, 32GB RAM, etc. is pretty much solid all the way. Even on rarely/never before accessed data, it's getting most of the data out of the ARC. I have some posts/tests/benchmarks here - 

 

Quote

The reason i was interested in raid5 was because i was hoping i could expand without loosing data. If that is not the case i think i'll settle for a 5x ssd raid0,

So ZFS does support RAID Z expansion. It's not ideal.
https://arstechnica.com/gadgets/2021/06/raidz-expansion-code-lands-in-openzfs-master/

https://freebsdfoundation.org/blog/raid-z-expansion-feature-for-zfs/

This would require installing something other than Windows. Though I suspect that if you were using something a bit "smarter" for your NAS you wouldn't have performance issues. 

The other consideration... Just go simpler. Think 2 drives RAID1 (or in ZFS terms, a mirror). Start with maybe 2x 6TB drives. When that hits ~70% capacity, get another pair and expand the pool... maybe move around some files so that they get evenly redistributed. 

If you can, you want to avoid having like 50 small drives. High failure rate, lots of jankiness and they electricity costs ADD up (think $100 a year to run 24/7). 

 

Quote

Besides that i'm also thinking about upgrading to a 5300G if i can get my hands on one.

This probably won't matter unless you're really limited by transcoding (could you just send the movies at full quality over the home network?). More RAM, faster networking and better disks is usually the way to go. 
 

 

Quote

Which is why a read cache doesn't really work as many files aren't repeat access, and it'd need to be very big.

So this makes it sound like the cache is just really "dumb". Is it FIFO? LRU? Something else?

My parent's QNAP NAS has a 240GB dumb cache on it that gets a lower hit rate than the ~25GB of RAM that I'm using for ARC on my NAS. 
Both ZFS (free, built into TRUENAS) and Primocache ($30ish, runs on Windows) do a great job of loading metadata (info about where blocks on the HDDs are stored) into ARC (adaptive replacement cache). VERY different level of snappiness, since all the super commonly used stuff is literally in RAM.