RAID 0 vs 5 vs COR parity for home use

[Jarsky]

At the cost of 1 extra disk, theres no reason personally to not to go RAID5 over RAID0. A drive doesn't have to be dead for it to destroy all your data either, even just unallocated sector count creep will do it. Restoring backups is a pain in the ass, and backing up large media collections would be expensive as hell. My NAS is hosting my media as well as the primary backup for my personal files for my PC (as well as a data store for my VM's application data - my VM server has ssd's which only host the primary VM partitions). 

 

Running many types of raids, including hardware RAID which I use, also allow you to expand the array easily adding in more disks. 

If you add a new disk to the array thats DoA and errors (which has happened to me twice) then you can recalculate parity without having to hold a massive backup of all that data. 

 

My NAS is over 40TB - I couldn't imagine spending ~$3000 on more drives to create a complete mirror backup of it all, or the amount of time and effort it would take to replace all of that media - when for a few hundred extra I have parity. 

 

I guess it comes down too, the size of your storage, and what your use case is.  

In my opinion, if you don't want to have parity - then you're safer to do drive spanning than RAID0. 

 

[brwainer]

1 hour ago, scottyseng said:

I remember for me to build the initial primary array (6 x 4TB at the start), was around 12 days 24/7. To rebuild one drive took around 4-5 days. Also have to consider the time it would take to move the data from the backup to the new array as well, and I have a lot of data at 20TB (Though I'd say about 1/3 is critical data...rest I can rerip from the physical media I own or download from where I bought digital media from).

Is that type of initializing used on RAID 0 and 1 setups? The only hardware controller I've used for RAID 0 and RAID1 (an HP something-or-other with a RAM cache and BBU) did them instantly, just writing the header I assume. In fact I've been able to destroy and rebuild the array on that controller and the filesystem within the array was 100% intact still - meaning that it didn't initialize anything other than the header. I don't know if that's a common setup or not, but I think that the initialization you are talking about wouldn't affect the type of system that @SCHISCHKA is talking about at all.

 

 

[scottyseng]

1 minute ago, brwainer said:

-snip-

Indeed you are correct, you can fast initialize hardware RAID 0 / 1 and the other levels. I'd personally recommend full initialization to make sure the RAID card goes through all of the drives first. I have a LSI MegaRAID 9260 8i.

 

Yeah, I was mostly replying to the OP's post (about wanting redundancy) and haven't read the full thread.

[SCHISCHKA]

2 hours ago, brwainer said:

Is that type of initializing used on RAID 0 and 1 setups? The only hardware controller I've used for RAID 0 and RAID1 (an HP something-or-other with a RAM cache and BBU) did them instantly, just writing the header I assume. In fact I've been able to destroy and rebuild the array on that controller and the filesystem within the array was 100% intact still - meaning that it didn't initialize anything other than the header. I don't know if that's a common setup or not, but I think that the initialization you are talking about wouldn't affect the type of system that @SCHISCHKA is talking about at all.

 

 

I estimate transferring 20TB would take about two to three days over gigabit or using single drive speed. I think that is what they meant.

[brwainer]

32 minutes ago, SCHISCHKA said:

I estimate transferring 20TB would take about two to three days over gigabit or using single drive speed. I think that is what they meant.

No when you initialize a RAID 5 or 6 it writes to all the disks completely, before you are even able to start putting data on the array (or some do let you start using it but it is very slow)

[leadeater]

27 minutes ago, brwainer said:

No when you initialize a RAID 5 or 6 it writes to all the disks completely, before you are even able to start putting data on the array (or some do let you start using it but it is very slow)

Unless you have a card that supports background initialization, which tbh is something you can't do without now days considering disk sizes. 

[SCHISCHKA]

48 minutes ago, brwainer said:

No when you initialize a RAID 5 or 6 it writes to all the disks completely, before you are even able to start putting data on the array (or some do let you start using it but it is very slow)

with mdadm you can start any RAID and tell the system that the drives are empty and do not require initialization. It does not affect performance as the drives are empty

[Jarsky]

56 minutes ago, brwainer said:

No when you initialize a RAID 5 or 6 it writes to all the disks completely, before you are even able to start putting data on the array (or some do let you start using it but it is very slow)

 

Background initialization is a great thing, and on hardware raids the speed hit is negligible for home use.

[ChalkChalkson]

12 hours ago, brwainer said:

For many of the people on this forum, and for the ones who have thus far commented on this thread, our home NASs have way more than 2 drives worth of data. You cannot categorically say that my system is not a home NAS. What you are saying makes sense and is true for small home NASs, but the way you are arguing against redundancy does not universally make sense for all "home NAS" units.

Yep, my NAS mainly used to hold my DVD collection, with an average of 5GB per DVD, and the 4TB I started with, I could hold about 800 DVDs, my collection is decently big, but I do not have 800 (that I though were worth ripping). With Blue-Rays however it looks a bit different, I made the switch relatively recently (since I just recently got a nice enough TV that it matters), but already my Blue-Rays take up more space than my DVDs, and I probably need to add a new 3TB drive every 1-3 years (depending on whether more 2015 like years come along). So before long, my NAS will be a bit too large to justify not spending the money on 1 additional drive to not spend about 1% of the running time of the NAS rebuilding from backup

[ChalkChalkson]

 

12 hours ago, brwainer said:

 For ones that I see in the future, I will still advocate for some level of redundancy just because it is more convenient to repair an array than to have to recover it.

I will likely do the same (as I do atm), but my reason is more profound: many people (against all the advise we can give) have either terrible backup habits, or no backups at all, so a drive failure could mean data loss for them... Also I don't want to be the guy who strongly advised against unraid parity or RAID 1/5/6/10 and having OP come back with a ton of lost data.

But just as you do, if there is someone who is saying he just wants 1TB of NAS that keeps a local copy of his onedrive, I will gladly point out, that RAID 1 is overkill

[ChalkChalkson]

12 hours ago, brwainer said:

 And I will point to the Storage Rankings thread on this forum that has 120 systems with an average of 31.4TB Raw disk space per server. Most of the active participants of these sections have systems on this list.

Well that arrangement is a little weak, but your conclusion is probably true, since there are 2 levels of selection bias here:

1: People who chill out in a NAS subforum probably like storage solutions, so they probably own storage solution, maybe even overkill ones.

2: People who enter a ranking list probably have lots of storage, even relative to the subforums baseline

 

I'd say I own pretty typical consumer grade home built NAS systems, and I didn't enter because I feel they are pretty boring compared to most NAS solutions on there. So given my small sample size, I would assume there are a lot of people not participating, and they are probably below the average 

[ChalkChalkson]

11 hours ago, scottyseng said:

Also have to consider the time it would take to move the data from the backup to the new array as well

Very valid point I didn't even consider up to this point (since I never had to do it).

My backup sits on a secondary NAS at my parents place, and they don't have enterprise grade internet... Their upload should be around 5Mbit/s, so even rebuilding a single Terrabyte would take more than 18 days and I can't really afford another full backup or a dedicated solution with good upload.

[Jarsky]

1 hour ago, ChalkChalkson said:

 I didn't enter because I feel they are pretty boring compared to most NAS solutions on there. So given my small sample size,

 

Ahh....I see you didnt see my old ghetto setup

[ChalkChalkson]

35 minutes ago, Jarsky said:

 

Ahh....I see you didnt see my old ghetto setup

Still more space than my NASs  

BTW big fan of the Phenom II line, ran on an X6 way too long  

[scottyseng]

6 hours ago, ChalkChalkson said:

Very valid point I didn't even consider up to this point (since I never had to do it).

My backup sits on a secondary NAS at my parents place, and they don't have enterprise grade internet... Their upload should be around 5Mbit/s, so even rebuilding a single Terrabyte would take more than 18 days and I can't really afford another full backup or a dedicated solution with good upload.

Yeah, I remember...originally I only had one array (4 x 4TB in RAID0) and I had to move the data off of the drives to a temp RAID0 array of two 4TB drives to store my stuff (I was going to change to RAID6)...it took 3 days to transfer, 24/7. And it wasn't a smooth process either, sometimes windows explorer flat out crashed due to the amount of data. I tried Teracopy but that also crashed...I had to do it in smaller chunks and then it went alright.

5 hours ago, Jarsky said:

Ahh....I see you didnt see my old ghetto setup

Brings back memories of when I had hard drives of my PC on my desk (before I had a good case / was using a prebuilt HP as a PC) because there wasn't enough space inside of the case....

[ChalkChalkson]

32 minutes ago, scottyseng said:

Yeah, I remember...originally I only had one array (4 x 4TB in RAID0) and I had to move the data off of the drives to a temp RAID0 array of two 4TB drives to store my stuff (I was going to change to RAID6)...it took 3 days to transfer, 24/7. And it wasn't a smooth process either, sometimes windows explorer flat out crashed due to the amount of data. I tried Teracopy but that also crashed...I had to do it in smaller chunks and then it went alright.

Brings back memories of when I had hard drives of my PC on my desk (before I had a good case / was using a prebuilt HP as a PC) because there wasn't enough space inside of the case....

Just today (funnily enough) I was forced to rebuild my array from parity and rebuild my backup (no fun...) the parity rebuilt took 4h, the backup is still running, even though I literally drove to my parents place to eliminate the network bottleneck. It will probably about 14 more hours to build it

[Helly]

On 11/06/2017 at 6:54 AM, scottyseng said:

I remember for me to build the initial primary array (6 x 4TB at the start), was around 12 days 24/7. To rebuild one drive took around 4-5 days. Also have to consider the time it would take to move the data from the backup to the new array as well, and I have a lot of data at 20TB (Though I'd say about 1/3 is critical data...rest I can rerip from the physical media I own or download from where I bought digital media from).

12 days seems extremely long to me. My 8x 5TB was initialized after 28 hours. Is my card(areca 1880i) just fast or is yours just slow?

 

For me its parity over backup. I'm not spending twice the money to have an offline backup of everything. I feel thats more of a waste of money since i've never even needed my backup in all the years i've had RAID-6's running. I do have a backup of the more important stuff on smaller drives but even those i don't keep synced with the server. They're sometimes months behind. But i guess thats my own fault really. I've only had to rebuild my current raid-6 twice due to the PSU not being sufficient and dropping 2 drives. It was done rebuilding after 17 hours. WAY less then the probably days it would have taken to restore the 12TB of data i got on there from a backup. Which has been said before is a huge pain in the ass i rather avoid.

[scottyseng]

Just now, Helly said:

12 days seems extremely long to me. My 8x 5TB was initialized after 28 hours. Is my card(areca 1880i) just fast or is yours just slow?

It slows down hard when you have another array apparently. I do remember my original array only taking 9ish hours (4 x 4TB though). However, once that array was made, I had to make my second array and that's when the slow down happened. I'm not sure why, but the controller was bogged down having to run one array while building the either.

 

Yeah, I agree, it's very hard for me to even attempt trying to make a clone of my NAS...the drive cost alone would be a few grand at minimum. That's not including having to build another server / get another UPS unit for it.

 

Oh wow, that's interesting that your PSU wasn't sufficient, but indeed, the surge power on start of a lot of hard drives does add up. I know my system will pull 500W to get started but then idle down to 200W on normal operation.

 

Dang, your controller seems to be really quick. I wish my rebuild times were that fast...