What is the best practical home use for RAID?

[TheGermanEngie]

What would be the best RAID for home users? Like say I wanted to store 500GB worth of videos, movies, pictures etc for home use. Should I use a RAID 1 for that (2x 500GB laptop drives) or are other RAIDS more practical and cost saving for the number of drives I have/cost of each drive? It would all be set up through software and not a card or anything like that since I'm a broke boi and on a budget.

[Enderman]

Don't use raid and just use a second drive for regular backups.

[ARikozuM]

If you have two, use RAID 1. If you have three, use RAID 5.

 

Redundancy is more important than speed for most home users unless you have a few hundred to spend on data recovery. 

 

Edit: Forgot to mention that you can use Storage Spaces and shares within Windows and your network, respectively. 

 

The other option is to add a SATA data+power cable for a more permanent single-PC backup either internally or externally. 

[TheGermanEngie]

1 hour ago, Enderman said:

Don't use raid and just use a second drive for regular backups.

not backups per se, more like a NAS is what i'm going after...

[79wjd]

1 minute ago, TheGermanEngie said:

not backups per se, more like a NAS is what i'm going after...

RAID 0 or JBOD if you want more capacity.

RAID 1 if you want some redundancy or shitty (unreliable) backups.

Actual backups to an external drive if you want reliable backups.

[Enderman]

5 minutes ago, TheGermanEngie said:

not backups per se, more like a NAS is what i'm going after...

Well just back up the nas, it will be more reliable than doing raid.

[beyonddc]

2 hours ago, TheGermanEngie said:

not backups per se, more like a NAS is what i'm going after...

NAS is just a network-attached storage.  There's no rule that says you need to setup RAID in your NAS.

 

I think your first step is to set a budget and the storage size you need for your NAS.

 

If you are looking for a low budget NAS then here's couple possible options

• A DIY Raspberry Pi 3 Model B NAS
• A 2nd hand computer desktop on Craigslist and re-purpose it to be a dedicated NAS
• Western Digital My Cloud series, the single hard drive one is affordable

Once you get into buying a dedicated NAS device with 4 bays and above, the price will start to add up.  A dedicated NAS device can be $300+ USD and then you have to buy hard drives for it.

[TheGermanEngie]

On 5/28/2018 at 12:31 AM, beyonddc said:

NAS is just a network-attached storage.  There's no rule that says you need to setup RAID in your NAS.

 

I think your first step is to set a budget and the storage size you need for your NAS.

 

If you are looking for a low budget NAS then here's couple possible options

• A DIY Raspberry Pi 3 Model B NAS
• A 2nd hand computer desktop on Craigslist and re-purpose it to be a dedicated NAS
• Western Digital My Cloud series, the single hard drive one is affordable

Once you get into buying a dedicated NAS device with 4 bays and above, the price will start to add up.  A dedicated NAS device can be $300+ USD and then you have to buy hard drives for it.

I think people are mistaking my intentions.. it's not a cold storage backup or a NAS intended for super fast anything like RAID 0. It's a NAS with the intention of storing my personal and family files in a RAID for redundancy. Using RAID 1. Or 5, if I add another drive. Of course RAIDs fail, I'll be very aware of that. And yes, I'll use an old computer I have lying around. Going to always be on and running 24/7. Anyways I know the best setup, I'm going for a RAID 1. Thank you all for your suggestions.

[Lady Fitzgerald]

3 hours ago, TheGermanEngie said:

I think people are mistaking my intentions.. it's not a cold storage backup or a NAS intended for super fast anything like RAID 0. It's a NAS with the intention of storing my personal and family files in a RAID for redundancy. Using RAID 1. Or 5, if I add another drive. Of course RAIDs fail, I'll be very aware of that. And yes, I'll use an old computer I have lying around. Going to always be on and running 24/7. Anyways I know the best setup, I'm going for a RAID 1. Thank you all for your suggestions.

I'm getting the impression you are confusing redundancy with backups. All RAIDs abov 0 do is give you redundancy (RAID 0 doesn't even give you that). All redundancy does is protect you from drive failure (up to aa point). It will not protect you from viruses and other malware, user error, data loss due to other hardware failure, theft, natural and unnatural disasters, etc. For that you need actual backups that are powered up and connected to the computer only when updating the backups but are otherwise kept powered down, disconnected from the computer, and stored away from the computer. For the best protection, you should have both onsite and offsite backups.

 

A RAID can be part of a backup but, alone, it is not a backup.

[TheGermanEngie]

2 hours ago, Lady Fitzgerald said:

I'm getting the impression you are confusing redundancy with backups. All RAIDs abov 0 do is give you redundancy (RAID 0 doesn't even give you that). All redundancy does is protect you from drive failure (up to aa point). It will not protect you from viruses and other malware, user error, data loss due to other hardware failure, theft, natural and unnatural disasters, etc. For that you need actual backups that are powered up and connected to the computer only when updating the backups but are otherwise kept powered down, disconnected from the computer, and stored away from the computer. For the best protection, you should have both onsite and offsite backups.

 

A RAID can be part of a backup but, alone, it is not a backup.

no I know. I was focusing primarily on the home media server and redundancy aspect of it, backups of the actual server is a totally different subject than what I was initially asking about. And yeah RAID doesn't make your data and hard drives invincible. Though I should probably consider a backup solution once I get the server up and running. 

[Lady Fitzgerald]

4 hours ago, TheGermanEngie said:

no I know. I was focusing primarily on the home media server and redundancy aspect of it, backups of the actual server is a totally different subject than what I was initially asking about. And yeah RAID doesn't make your data and hard drives invincible. Though I should probably consider a backup solution once I get the server up and running. 

Then you are ahead of most people. Cool!

[Razor Blade]

The best RAID for home users?  The answer is yes. It really depends on what you want to do. These could be use cases...

 

(RAID 0) Do you just want to go fast and have the max amount of capacity where risk of failure may not be your top concern? 

(RAID 1) Do you have the family PC that you don't necessarily need speed or space, just want a little insurance against a drive failure?

(RAID 5 or 6) Are you running a media server that you need drive space but don't want to take it offline and completely reinstall everything if a drive ever fails? 

(RAID 10, or 01) Are you running a workstation PC that you edit a bunch of videos on that you absolutely need all the speed you can get but can't tolerate a single drive failure?

 

RAID or Redundant Array of Inexpensive Disks simply means it is a way to assemble multiple hard drives or SSDs into an array you use for...."blah"... where "blah" is your use case.

 

• RAID 0 or data striping is wonderful for raw speed and capacity needs. Having a couple SSDs in RAID 0 makes for a really fast scratch disk array you can dump media to almost as fast as you can drag it over. Though RAID 0 has no redundancy so if one disk fails, you lose all of your data.

 

• RAID 1 or data mirroring gives you an element of redundancy. While this in theory means you have a more robust dataset, it is still not a substitute for a backup plan. Think of it as a good way to keep you running instead of to keep your data safe. Downside is you're literally paying twice for the same amount of usable disk space

 

• RAID 5 introduces the idea of parity. One drive's worth of capacity will be used for parity. So for example if you have five 2TB drives in a RAID 5 you will only be able to use 8TB and not 10TB. This means if any one drive out of the array fails, parts of the data that was on that drive are scattered across the other drives. This makes it possible to replace the disk and rebuild the array. The downside to this array is not only is it going to be slower than a RAID 10 but also you will stress the other drives in the array during a rebuild process...which could lead to another drive failing...however unlikely that is.

 

• RAID 6, like RAID 5, introduces parity but now in the form of two drives. So RAID 6 allows up to two drive failures before the array is lost. The disadvantage to this RAID is not only is it much slower...it also eats up another entire drive just for parity. So for the same five 2 TB drive example I used, you would only get 6TB available in the array.

 

Next may not be as common for home uses...

• RAID 01 is a mirror of stripes. Not to be confused with RAID 10, RAID 01 literally means you are mirroring striped arrays. While you are getting an element of redundancy in that if a disk fails you still have the other array, if a disk from both sides of the mirror fail, you're SOL. The advantage is you still get great speeds and with more capacity than you would with a RAID 10

 

• RAID 10 is a stripe of mirrors. This means you are striping together mirrored disks. While you get a lower capacity with this solution, you are getting a bit more robust data set. You still get speed, but will sacrifice capacity vs a RAID 01.

[jde3]

I think the disk manufactures tried to change the "I" from Inexpensive to Independent.. but the idea is failure is expected and assured really so usually having more spares is better than having more expensive drives.

 

Lot of good info on this but I'll add to this topic that a second drive (removable) isn't really that great of a backup. (it's better than nothing sure) but a second drive can't protect you from theft, it can't protect you from disaster like fire or flood etc.

 

You need to use offsite for a good reliable backup, this is for data that can not be lost ever.. (encryption keys, document and title scans, creative works and scripts, baby pictures idk.. people value data differently you get the idea though.) I tend to prefer loading encrypted backups on to Amazon S3 for this as there is little chance of them going out of business (:cough: crashplan) but pick your poison with a provider.

[AbsoluteFool]

On 28.5.2018 at 2:30 AM, TheGermanEngie said:

What would be the best RAID for home users? Like say I wanted to store 500GB worth of videos, movies, pictures etc for home use. Should I use a RAID 1 for that (2x 500GB laptop drives) or are other RAIDS more practical and cost saving for the number of drives I have/cost of each drive? It would all be set up through software and not a card or anything like that since I'm a broke boi and on a budget.

I see there are many diffrent thoughts here, but the RAID level you use depends on the use case. Like, if you want to store videos (Just store and not anything else) RAID5/6 is fine for up to 3 drives and more. Going over 6 drives you'll want to consider a RAID 50/60. If you want to edit you might go for speed over redurancy because you have a diffrent backup. Then RAID 0/1/10 will be better depending on your need.

 

Let's say i was gona go just for speed i wouldn't use anything at all and pump all drives into a RAID 0 and gain drive preformance. Would be great for streaming and Video on demand content. But horrible for storage.

 

So what RAID you use and what is best, depends on what you are using it for. There is no magical answer that will always be right.

[leadeater]

4 hours ago, AbsoluteFool said:

Going over 6 drives you'll want to consider a RAID 50/60.

That's not really good practice at all nor that efficient either. 24 disk RAID 6 is totally fine, or better 22 disk RAID 6 + 2 hot spare (idle spin down). That second config is actually Netapp recommended practice, every disk shelf is configured with a dual parity two hot spare RAID group then combined in to what they call an aggregate (pool basically).

 

Any time you introduce a stripe you're massively increasing failure risk, don't do it unless you need to and that's normally only for speed reasons. Not that I've ever seen a RAID 6 array on a good controller go slow, you'll easily push upwards of 1GB/s read and 600MB/s+ write.

 

Edit:

For RAID 50/60 stripe across failure boundaries where it won't really increase failure risk as much like disk shelves or drive cages, if a shelf fails it's not going to matter what RAID level you pick losing that many drives will always take the system down (bar RAID 10 with mirroring across shelves).

[jde3]

Raid 60 is better because it's a bigger number.

 

An old rule of thumb was

3+ drives = Raid Z1

6+ drives = Raid Z2

9+ drives = Raid Z3

Past that you might want to consider groups of mirrors or a stripe depending on performance but be careful.. (Perhaps go Raidz1 on a pool of mirrors, 5+1) for home Raid Z3 with a few spares should suit you quite well.

 

Mirrors also provide good performance (on reads at least with ZFS because it's smart enough to use the bandwidth of all the drives) if that is what your after.

 

[AbsoluteFool]

14 hours ago, leadeater said:

That's not really good practice at all nor that efficient either. 24 disk RAID 6 is totally fine, or better 22 disk RAID 6 + 2 hot spare (idle spin down). That second config is actually Netapp recommended practice, every disk shelf is configured with a dual parity two hot spare RAID group then combined in to what they call an aggregate (pool basically).

 

Any time you introduce a stripe you're massively increasing failure risk, don't do it unless you need to and that's normally only for speed reasons. Not that I've ever seen a RAID 6 array on a good controller go slow, you'll easily push upwards of 1GB/s read and 600MB/s+ write.

 

Edit:

For RAID 50/60 stripe across failure boundaries where it won't really increase failure risk as much like disk shelves or drive cages, if a shelf fails it's not going to matter what RAID level you pick losing that many drives will always take the system down (bar RAID 10 with mirroring across shelves).

Yeah, it was just an example. I believe people are way to hung up in getting "The best" when there really is no "right" answer without knowing the use case for the RAID array.

[Razor Blade]

RAID 50 and 60 mean you sacrifice even more disks for speed. For example if you have 8 drives in a RAID 50 array of 4+4 array you are going to use 2 disks for parity. In a RAID 60 with the same 8 disks, HALF will be used for parity! Now I was giving a "single card" example of this. You could have a RAID 50 spanned across 24 drives then the motherboard goes wrong and corrupts the firmware on a RAID card....like what happened to poor Whonnock... All the parity in the world would not have prevented that failure.

[leadeater]

4 hours ago, Razor Blade said:

RAID 50 and 60 mean you sacrifice even more disks for speed. For example if you have 8 drives in a RAID 50 array of 4+4 array you are going to use 2 disks for parity. In a RAID 60 with the same 8 disks, HALF will be used for parity! Now I was giving a "single card" example of this. You could have a RAID 50 spanned across 24 drives then the motherboard goes wrong and corrupts the firmware on a RAID card....like what happened to poor Whonnock... All the parity in the world would not have prevented that failure.

Honestly I've never done a RAID 50 or 60 because that type of thing scares me to death and an actually good RAID card with large cache will basically do anything you ask it to. If someone is thinking about doing RAID 50 or 60 it's probably time to move over to a software solution or enterprise array.

[dalekphalm]

On 5/27/2018 at 8:30 PM, TheGermanEngie said:

What would be the best RAID for home users? Like say I wanted to store 500GB worth of videos, movies, pictures etc for home use. Should I use a RAID 1 for that (2x 500GB laptop drives) or are other RAIDS more practical and cost saving for the number of drives I have/cost of each drive? It would all be set up through software and not a card or anything like that since I'm a broke boi and on a budget.

There are two practical reasons to use RAID, in my opinion:

 

1. To take multiple smaller drives, and combine them into a single "virtual" drive - thus avoiding having to manage different drives for the same content. (Does not apply to RAID1 obviously)

2. To give some redundancy against hardware failure. If a drive fails, you replace the dead drive with a new one, without having to redo the setup, and minimizing down time. (the level of redundancy depends on which RAID level is used - furthermore, there is no redundancy with RAID0)

 

One thing to remember is that RAID IS NOT A BACKUP. RAID will not protect you against malware, accidental file deletion, or many other ways of losing data. RAID is, at it's heart, simply about lessening the amount of time your system is offline if a drive fails.

 

If this is for personal use, I would actually recommend you taking those 500GB drives and doing the following:

1. Configure each drive as it's own drive - no RAID at all

2. Use the first drive as your intended storage drive (for music, videos, whatever)

3. Use the built-in backup software of your OS (or a third party backup software of your choice) to configure a scheduled regular backup (differential is best, but there are many methods) of the first drive, with the second drive as the "destination" for the backups.

 

This will simplify your setup and give you backups in case you lose a file (or the drive dies).

[jde3]

With ZFS you have the ability to test out a raid layout before you use it.

 

Here is a little script to create you 24 file backed "drives" on FreeBSD. (Or FreeNAS but the web UI may not be aware of the pool, not sure)

#!/bin/csh
set i = 1
while ( $i <= 24 )
dd if=/dev/zero of=./disk$i bs=1M count=128
mdconfig -a -t vnode -f ./disk$i
echo "disk $i done"
@ i++
end

The drives will be listed as /dev/md0-23

 

Then you can add them to a pool with..

zpool create mirror md0 md1 mirror md2 md3 .. etc

 

There is also the -n switch to only print what it would have done. You can practice destroying and resilvering disks this way also. You can remove devices with mdconfig -d -u 0 for instance.

[Pixel5]

23 hours ago, dalekphalm said:

There are two practical reasons to use RAID, in my opinion:

 

1. To take multiple smaller drives, and combine them into a single "virtual" drive - thus avoiding having to manage different drives for the same content. (Does not apply to RAID1 obviously)

2. To give some redundancy against hardware failure. If a drive fails, you replace the dead drive with a new one, without having to redo the setup, and minimizing down time. (the level of redundancy depends on which RAID level is used - furthermore, there is no redundancy with RAID0)

 

One thing to remember is that RAID IS NOT A BACKUP. RAID will not protect you against malware, accidental file deletion, or many other ways of losing data. RAID is, at it's heart, simply about lessening the amount of time your system is offline if a drive fails.

 

while it is correct that raid is not a backup i would still say having one disk redundancy is more than most people have and already a good protection of your data.

combine this with snapshots and you are already pretty safe against most things which is more than enough for the average user.

 

If you want to go all in you obviously want offsite backups but realistically the normal user does not need this beyond a few documents or pcitures which you could sync to google drive or amazon.

 

[dalekphalm]

19 minutes ago, Pixel5 said:

 

while it is correct that raid is not a backup i would still say having one disk redundancy is more than most people have and already a good protection of your data.

combine this with snapshots and you are already pretty safe against most things which is more than enough for the average user.

 

If you want to go all in you obviously want offsite backups but realistically the normal user does not need this beyond a few documents or pcitures which you could sync to google drive or amazon.

 

I'm sorry but I strongly disagree.

 

I would even go so far as to say that RAID is worse than a backup, because it gives a false sense of security.

 

If the average user is doing nothing? Sure, RAID is slightly better than "nothing". But here's the thing - instead of doing RAID, they should just setup backups first instead.

 

Offsite backups, etc, are definitely overkill for most users. That we can agree on. But you don't need that for a simple backup solution.

 

For most users, just getting an external USB Drive + Windows (or mac) built-in backup utility is all you need.

 

The problem is that people get this idea that "RAID is enough - I don't need a backup" because of posts on the forum that encourage it. When, in most cases, these users don't even need RAID at all.

 

I will always remind any newbie discussing RAID that RAID is not a backup. That will not change. They can decide for themselves if it's worth the risk, after I've given them the information needed to make an informed decision.

[TheGermanEngie]

21 hours ago, dalekphalm said:

I'm sorry but I strongly disagree.

 

I would even go so far as to say that RAID is worse than a backup, because it gives a false sense of security.

 

If the average user is doing nothing? Sure, RAID is slightly better than "nothing". But here's the thing - instead of doing RAID, they should just setup backups first instead.

 

Offsite backups, etc, are definitely overkill for most users. That we can agree on. But you don't need that for a simple backup solution.

 

For most users, just getting an external USB Drive + Windows (or mac) built-in backup utility is all you need.

 

The problem is that people get this idea that "RAID is enough - I don't need a backup" because of posts on the forum that encourage it. When, in most cases, these users don't even need RAID at all.

 

I will always remind any newbie discussing RAID that RAID is not a backup. That will not change. They can decide for themselves if it's worth the risk, after I've given them the information needed to make an informed decision.

I'm not using RAID as a backup, though... and I'm fully aware it's not a backup as well. It's being used for my home media server, to have my media be accessible 24/7. I have a cold storage offsite backup of all the data on the server. Better safe than sorry...

[dalekphalm]

1 hour ago, TheGermanEngie said:

I'm not using RAID as a backup, though... and I'm fully aware it's not a backup as well. It's being used for my home media server, to have my media be accessible 24/7. I have a cold storage offsite backup of all the data on the server. Better safe than sorry...

That's awesome. You are better than most users when it comes to that.

 

Anyway, my comment wasn't pointed at you. I was just correcting what I consider to be bad advice.