Afraid of storing family data on DIY NAS

[AlphaPixel]

Hi guys.

I made a NAS with 4 drives 1TB each in Raid 5 with FreeNAS. The point is that I have been losing family photos very often using external hardrives, and I wanted to make a safe place to store all my data.

Is it safe for me to put everything in the NAS?

Can you guys give me tips and solutions to make a "bulletproof" way to store my data?

[iamdarkyoshi]

Back up to a cloud based service, my family uses idrive

[Trefonix]

Yeah, you pretty much can't get any safer than cloud storage. Failing that, you can setup a raid array with some redundancy I suppose?

[Gamerguy207]

Why would it go wrong it's just going to transfer data that's all think about it

[BlueChinchillaEatingDorito]

5 minutes ago, Trefonix said:

Yeah, you pretty much can't get any safer than cloud storage. Failing that, you can setup a raid array with some redundancy I suppose?

Or buy a bunch of 128GB USB flash drives and back up your important data on those. They're like 30 bucks a pop nowadays. Can't imagine a family with a TB worth of photos. 

[Mira Yurizaki]

A snippet from a guide I wrote:

 

Quote

1. The 3-2-1 Rule for Backing Up Files
This is more of a guideline, but a good one none the less because it's simple and covers a wide range of what you should do when backing up files. The 3-2-1 rule breaks down into this:

• Make at least three copies of your data. While having two is sufficient, three or more increases the survivability of your data greatly.
   
• Have your data on at least two different media/drives. There's no point in making those three copies if they all live on the same hard drive. Make sure your copies are spread across at least two different media or drives, like an HDD, SSD, or optical disc.
   
• Have one copy of your data on at least one place offsite. What if you had five copies spread out on different drives, but they were in the same house and the house burned down? Now you have nothing. One copy of your data should live somewhere offsite, be it with a trusted person, a safe deposit box, or in the cloud.

 

[Doramius]

You have a redundancy setup with the RAID.  Your alternative would be doing something like RAID 10, RAID 6, or some other RAID combination.  For home use, RAID 5 should be plenty and provide appropriate redundancy for safety.

 

Depending on how much data you have, cloud storage might be fine.  I'd still keep a physical copy somewhere.  

[vanished]

4 minutes ago, Doramius said:

You have a redundancy setup with the RAID.  Your alternative would be doing something like RAID 10, RAID 6, or some other RAID combination.  For home use, RAID 5 should be plenty and provide appropriate redundancy for safety.

 

Depending on how much data you have, cloud storage might be fine.  I'd still keep a physical copy somewhere.  

RAID is not a backup, it protects you against drive failure, not other forms of data loss, like the house burning down, accidentally deleting them yourself, etc.

For the vast majority of home users RAID is completely pointless in fact, and they would be much better served by an external drive backup that they run on command or on a schedule.

[AlphaPixel]

I think I'm going M.Yurizaki route, the second question is: what if a drive fails?

Is my NAS build good for redundancy or am I going to have a second drive fail when I switch the bad drive?

[MrKickkiller]

1 minute ago, AlphaPixel said:

I think I'm going M.Yurizaki route, the second question is: what if a drive fails?

Is my NAS build good for redundancy or am I going to have a second drive fail when I switch the bad drive?

RAID 5 (or Actually RAIDZ1) means that you can lose 1 drive from the vdev/pool (correct terminology escapes my head ftm). Assume drive 2 somehow fails. Your Nas box will continue to operate. If you replace the broken drive 2 with a drive (that has atleast the same capacity), you can tell the nas to reconstruct the data based on the other 3 drives.

 

Your decision of going with Freenas also secures you from other dangers: Bitrot / Corruption.

In your case, you got 3 drives worth of space, for your 4 drives. The other drive is used for parity. Parity is basically a small and easy to calculate piece of data that descibes how the OTHER data (your photo's, documents etc) relates to each other.

Eg: Drive A, Drive B, Drive C, Drive P

      1111       0001      1000      0110

The P part stores the XOR outcome of all other parts.

If you know 3 out of 4 operands in a mathematical process, you can calculate the 4th.

Eg: A+3B = P. Say you lose the value of A, but still retain the knowledge of value B and P.. You can calculate what A used to be. 

 

ZFS (the filesystem behind Freenas) will periodically verify that ALL data on drives is what it originally was, by running the maths on each and every bit/byte. You'll be protected against 1 bit going bad in a drive. With RAIDZ2 - RAIDZ3, you would be protected against 2 bits going bad. But, the chances of 2 bits going bad in your case will be astronomically small. 

[AlphaPixel]

What if the NAS OS fails? 

[AlphaPixel]

I'm a Software Engineer so I don't have much knowledge about servers, but I know about parity and how to code it.

[Mikensan]

RaidZ pools can be imported into a new install very very very easily.

 

Make sure copies that go onto the FreeNAS reside elsewhere (crashplan / google / dropbox / personal computer) and you're fine. Likelehood of both entirely dying at the same time is minimal.

 

What happened to your external drives that you lost data on? May still be able to recover data.

[Doramius]

1 hour ago, Ryan_Vickers said:

RAID is not a backup, it protects you against drive failure, not other forms of data loss, like the house burning down, accidentally deleting them yourself, etc.

For the vast majority of home users RAID is completely pointless in fact, and they would be much better served by an external drive backup that they run on command or on a schedule.

If using RAID as your only storage option, it is an idiot move waiting for failure.  I think you're looking into my statement incorrectly.  

First, a RAID "CAN' be used as a backup of another data source.  As stated, the OP had external drives that have failed, the RAID would be a backup of that data.  The 'redundancy of the RAID' IS NOT a backup to itself.  It is only a precautionary or preventative to data loss of the array, itself. Because of this, RAIDs are actually becoming quite popular for home storage, especially when run as a NAS.  Even Popular Electronics and other major publications has acknowledged the growing popularity of consumer RAID setups due to the increase in home based NAS devices for data storage and home surveillance.  

I also mentioned cloud services, which are also gaining in popularity for home users.  If you make a copy of a file to be archived in a separate place, it is a backup.  The level to which one goes for the protection and location of that backup, is another story.  It is a backup, nonetheless.

 

Again, not exactly sure how you were understanding my reply.

[vanished]

2 minutes ago, Doramius said:

Again, not exactly sure how you were understanding my reply.

I think I may have misunderstood how you intended it to be used, that's fair  

 

The main thing I wanted to get across is that it protects you from drive failure - in fact it protects you probably better than anything else can - but, that's all it does.  There's a ton of other ways you can lose data and it does nothing for any of them.

[Doramius]

6 minutes ago, Ryan_Vickers said:

I think I may have misunderstood how you intended it to be used, that's fair  

 

The main thing I wanted to get across is that it protects you from drive failure - in fact it protects you probably better than anything else can - but, that's all it does.  There's a ton of other ways you can lose data and it does nothing for any of them.

I had a sense that's where you were thinking, but no harm done.  I had to read my statement again after you made your post, and made the clarification in case someone else may have interpreted it that way.  It's late on a Thursday.  If we were all thinking clearly, we'd probably be at the bar asking for strong drinks to remedy that issue.

[AlphaPixel]

Hummmm so I guess I'm going to make multiple NAS with RAID5.

[Daniel644]

16 minutes ago, AlphaPixel said:

Hummmm so I guess I'm going to make multiple NAS with RAID5.

NO, your RAID array is fine for your home use, but you ALSO need to have an external drive located off site (AWAY FROM YOUR HOUSE, ideally not even in the same town if possible) in case you get flooded or a Fire or a twister or one of many other natural disasters strike you don't want the backup to get destroyed with the original. there is no reason to spend the money on a second RAID array for a backups backup, thats just being paranoid over drive failure and by the time you spend that much money you can have decades of online data storage for all your photos that already employ there own RAID arrays on a much larger scale to prevent data loss.

[AlphaPixel]

Money is not the problem, I get free samples of hardware to test. That's why I wanted to make the RAID (it's almost free).

So i'm going to have one RAID at my house and the second at my other house.

[Trefonix]

On 20/07/2017 at 7:34 PM, BlueChinchillaEatingDorito said:

Or buy a bunch of 128GB USB flash drives and back up your important data on those. They're like 30 bucks a pop nowadays. Can't imagine a family with a TB worth of photos. 

You must have never met my grandmother. She literally takes 10 photos of everything and has been doing so for the last 14 years. This adds up to multiple terabytes of photos

[BlueChinchillaEatingDorito]

4 hours ago, Trefonix said:

You must have never met my grandmother. She literally takes 10 photos of everything and has been doing so for the last 14 years. This adds up to multiple terabytes of photos

Was she like that during the film days too? Damn that would've been expensive.

[vanished]

On 7/20/2017 at 0:34 PM, BlueChinchillaEatingDorito said:

Or buy a bunch of 128GB USB flash drives and back up your important data on those. They're like 30 bucks a pop nowadays. Can't imagine a family with a TB worth of photos. 

Actually it adds up pretty quickly at 30 MB a pop.