To RAID or Not To RAID

[Supersonicwolfe]

I am wondering if I should get a mobo with RAID 1 for my personal NAS / "data center" or just use a software to automatically mirror the disks. (e.g. use Windows Storge Spaces or a 3rd party software if you have some suggestions). I am considering to use RAID 1 to prevent hard drive (such as Iron wolf or WD red) failing, but I also heard if the RAID controller failed, it's game over unless I can find another mobo with same chipset (Since the raid controller is part of ICH10R chipset).

 

Oh, and I also want this setup to be somewhat transplantable, not sure if that reason alone can kill RAID setups.(I am using a Q9550 been sitting in a closet for years, safe to say it is good for at least some years even if I OC it, but one day it will die and I don't want to spend hours copying files from backups and rebuilding the whole setup)

 

So here will be what my drive setup will look like.

SATA 1 OS (SSD)

SATA 2 Games and Programs (WD Black)

SATA 3 DATA 

SATA 4 DATA (or RAID 1 with SATA 3)

SATA 5 (or External USB 3.0)  Windows Image Backup drive

SATA 6 BD-ROM (Super important files will get burned onto BD or M-Disc)

USB 3.0 (TBD) External image back up of SATA 3 drive (Maybe offsite incase of fire/flood/hurricane/tornado etc.)

Ethernet Online back up of SATA 1,2,3

 

FYI: I already have a Asus P5QL Pro (P43+ICH10  chipset w/ 6 SATA ports) but I am eyeing Gigabyte GA-EP45-UD3R (P45+ICH10R chipset 6+2 SATA ports), mostly because it is a bigger, nicer-looking, better-performing, less-awkward-ATX-power-location board and last but not least ICH10R chipset as well as Gigabit's own RAID controller.

 

Do let me know if you think my back up scheme is a bit too extreme and if you have any suggestion on building a home server/media PC

[SupaKomputa]

q9550 is an overkill for just home nas, it's power hungry and hot.

i won't use it for home server.

i might use a low powered atom / celeron solution.

as for raid use raid 1 for 2 drives or raid 5 for 3 drives.

[Supersonicwolfe]

16 minutes ago, SupaKomputa said:

q9550 is an overkill for just home nas, it's power hungry and hot.

i won't use it for home server.

i might use a low powered atom / celeron solution.

as for raid use raid 1 for 2 drives or raid 5 for 3 drives.

Well I am working with what I have (either a E8500 or Q9550, Q9550 is my first choice as I know it is younger and I do use that machine to play OLD games from time to time.)

 

Aside from that, I do want to know more about the importance of the raid controller. Like if I can built a RAID 1 or RAID 5, can I transplant the data drives to a newer mobo (such as a LGA 1151) when the 775 mobo died?

[kirashi]

On 9/28/2018 at 10:38 PM, Supersonicwolfe said:

Oh, and I also want this setup to be somewhat transplantable, not sure if that reason alone can kill RAID setups.(I am using a Q9550 been sitting in a closet for years, safe to say it is good for at least some years even if I OC it, but one day it will die and I don't want to spend hours copying files from backups and rebuilding the whole setup)

On 9/28/2018 at 11:26 PM, Supersonicwolfe said:

Aside from that, I do want to know more about the importance of the raid controller. Like if I can built a RAID 1 or RAID 5, can I transplant the data drives to a newer mobo (such as a LGA 1151) when the 775 mobo died?

In theory, one can move the RAID controller PCI card between machines along with the HDD's too and use them without losing data. However, if the RAID controller dies and you don't have a spare one of the EXACT SAME hardware revision running the EXACT SAME firmware, your array could be lost forever. SCSI or SAS based RAID setups are a bit different, as these enterprise grade options are usually designed with more resliliance & can be moved between SAS controllers, however, they usually come at a much higher cost per capacity.

 

Let's clear something up first though: RAID IS NOT A BACKUP. RAID. IS NOT. A BACKUP. Please, be smart and keep copies of any important files you cannot re-create in at least 3 different places, preferably on 2 different mediums, 1 of which is in a different geographical location, at all times. This way if your house burns down and you get locked out of Dropbox / Google Drive / OneDrive, you'll probably have a chance to recover data from the HDD kept at your relatives house 3-5 hours away from your location.

 

Instead, research Disk Pooling software that's easy to migrate between machines. This way, if a piece of hardware dies (aside from the HDD's themselves, of course) it's super easy to just transplant the HDD's into another PC, install the Drive Pooling software, and bam - you're up and running again. RAID has its' purposes in a properly managed IT environment with the (large) budget to support recovering / rebuilding an array, but I don't advise it for residential or small business use as it's just not worth the time you'd spend trying to rebuild the array.

 

I use StableBit's Drive Pool software because it's literally install, setup, and forget, AND supports simply moving all your drives to another computer. One can even detach a drive from a pool to say read the files on another computer WITHOUT the use of the software, then reconnect that drive to re-sync it to its' original pool. (Of course, this depends on how you have your pools configured, as you might not want to write any data to the drive with it detached from the pool to prevent it from being inconsistent.)

 

I have 2 drive pools setup in a "RAID" 0 READ / 1 WRITE configuration, meaning that my data is duplicated for redundancy across 2x SATA SSD's and 2x HDD's, but also doubles my read speeds from both pools because it reads from both SSD's or both HDD's paired in each pool. It's partly what made me hold off on buying an NVMe drive, since I can get a combined read rate of around 1100 MB/s from 2x SSD's, and around 350 MB/s read from 2x HDD's.

 

Full disclosure: I am NOT paid a dime to talk about their software - I'm just an extremely happy user who's bought and paid for it after doing research into other Drive Pool solutions. It's written by a bunch of data storage geeks who went looking for a replacement to Windows Home Server 2011 as it became end of life. There's an old comparison post on reddit linked below discussing Drive Bender and FlexRAID as well, if you wanted to check those out.

 

Edited October 1, 2018 by kirashi
added bit about SAS / SCSI RAID

[Helly]

On 9/29/2018 at 9:06 AM, kirashi said:

In theory, one can move the RAID controller PCI card between machines along with the HDD's too and use them without losing data. However, if the RAID controller dies and you don't have a spare one of the EXACT SAME hardware revision running the EXACT SAME firmware, your array could be lost forever.

Not entirely true for ALL RAID controllers. I know for instance that you can safely move array's between different SAS controllers of Areca. for instance you can create an array on an 1880 and move it to an 1882 then to an 1883 and to the latest 1884 without any problems. firmware and hardware revisions don't matter in that case at all.

Moving a controller between different machines is 100% possible without data loss with ANY brand of controller. The mobo/cpu or anything else has NOTHING to do with the array in that case.

 

In theory, you MIGHT be able to move array's between onboard intel controllers as well. I've never tried it though, so if you have different intel onboard controllers, i suggest trying to move them between systems before putting anything important on it.

I have moved an array from an AMD asus 990fx sabertooth to a gigabyte 990fx-ud7 without any data loss. But that is the same controller so it makes sense.

[GlobalPentahedron Corp]

only use raid if you need your computer to run uninterrupted, otherwise you are better off using that second drive as an offline/external backup. You should have at least two back up disks. I don't think a home server needs redundant disks. If your drive fails, turn off the server, swap the dead drive for one of your back ups, and order a new drive. That is ~20 minute of downtime that a business cannot afford, but a home user can.

[kirashi]

7 hours ago, Helly said:

Not entirely true for ALL RAID controllers. I know for instance that you can safely move array's between different SAS controllers of Areca.

--SNIP--

Correct. Enterprise RAID (such as SCSI aka SAS drives & RAID setups) is designed much better from the ground up. It can also be a lot more expensive, since SAS drives (10K & 15K RPM, usually) have a higher cost per capacity, although I'm sure you can adapt regular SATA drives to SCSI with some fandangled adapter cable of some sort. I'll go back and edit my post, as I was referring to regular SATA-based RAID that most consumers would be using, since most consumer & even small business oriented motherboards & computers don't handle SAS or SCSI drives.

[Lady Fitzgerald]

On 9/29/2018 at 12:06 AM, kirashi said:

Let's clear something up first though: RAID IS NOT A BACKUP. RAID. IS NOT. A BACKUP. Please, be smart and keep copies of any important files you cannot re-create in at least 3 different places, preferably on 2 different mediums, 1 of which is in a different geographical location, at all times. This way if your house burns down and you get locked out of Dropbox / Google Drive / OneDrive, you'll probably have a chance to recover data from the HDD kept at your relatives house 3-5 hours away from your location.

I completely agree except two or more kinds of media is not necessary as long as you use current media and migrate to newer media before the older media becomes obsolete.

[Lady Fitzgerald]

12 minutes ago, GlobalPentahedron Corp said:

only use raid if you need your computer to run uninterrupted, otherwise you are better off using that second drive as an offline/external backup. You should have at least two back up disks. I don't think a home server needs redundant disks. If your drive fails, turn off the server, swap the dead drive for one of your back ups, and order a new drive. That is ~20 minute of downtime that a business cannot afford, but a home user can.

I totally agree except I would clone or sync a backup drive to the replacement drive.

[GlobalPentahedron Corp]

Just now, Lady Fitzgerald said:

I totally agree except I would clone or sync a backup drive to the replacement drive.

Two back up drives. Rotate them everyday.

One in a safe place.

One plugged into a dock syncing all day as you work.

That is a bare minimum.

I keep another drive at my parents that I rotate when I can.

[Razor Blade]

Periodic backups is a must for any data deemed important. RAID and drive pools are nice for storage expansion and up time. The redundancy protects against many types of failure but as mentioned above, it should never be treated as a means for backup.

 

That said I think there is still plenty of reason for a home user to utilize RAID. But using a RAID controller to manage your array is a bit unnecessary now a days. You can import an array to a different controller but there will always be risk of it not being able to. Software RAID seems to be a bit more forgiving...at least in my experience.

 

As far as backup media? Sure more is better. But don't over complicate it. If you loose 40TB of..err movies it wouldn't be as devastating as losing photos of your child's baby photos or a wedding gallery or important documents like will, tax, or other stuff. That in mind backing up everything is nice but make sure you don't spend all your money buying super expensive drives just to double back up movies you could always re rip or software you could re download.

[Lady Fitzgerald]

34 minutes ago, GlobalPentahedron Corp said:

Two back up drives. Rotate them everyday.

One in a safe place.

One plugged into a dock syncing all day as you work.

That is a bare minimum.

I keep another drive at my parents that I rotate when I can.

Very bad idea! If you leave a drive syncing all day, that is not a backup! By leaving the "backup drive" attached to the computer, you are exposing it to infection from viruses, etc. If the source drive gets infected or otherwise corrupted, the synced drive will also become corrupted. If you accidentally delete a file, It will also be deleted from the synced drive.

 

For a drive to be a true backup drive, it must be kept disconnected from the computer and powered down except while updating the backup. For data drives, folder/file syncing works best but you need to make sure there are no infections or corruption first (running security scans just before syncing will take care of that).

 

For data to be reasonably safe, it must be kept in three, separate places. For most people, this means the computer, an onsite drive, and an offsite drive. Any drive connected to the computer, internally or external, is considered part of the computer. 

14 minutes ago, Razor Blade said:

That in mind backing up everything is nice but make sure you don't spend all your money buying super expensive drives just to double back up movies you could always re rip or software you could re download.

Considering how long it takes to rip CDs, DVDs, and BDs, plus any manual tagging that has to be done (especially for classical music CDs), it can be worthwhile to keep backups of them if one should lose everything, say, in a house fire.

 

I have had CDs (especially burned ones from indie artists) and DVDs go bad on me in storage and replacing them may not always be possible. I have almost 700 CDs and around as many DVDs and BDs, many dating back for decades. I estimate half are no longer available. The remainder would take weeks to rerip and, in the case of most of my CDs, correctly tagging them would take an almost identical amount of time.  It's well worth it to me to keep them backed up.

 

Usually, you can download only the latest versions of software. The latest versions are not always the best, They could be buggy, bloated, or just redesigned in an unacceptable way. Also, software can be discontinued and if you don't have a backup, you're SOL.

 

One thing I learned when I was still in the workforce was to keep copies of everything. Doing so bailed out my butt many times. The same is true of backups. 

[GlobalPentahedron Corp]

4 minutes ago, Lady Fitzgerald said:

Very bad idea! If you leave a drive syncing all day, that is not a backup! By leaving the "backup drive" attached to the computer, you are exposing it to infection from viruses, etc.

no its ok i rotate them daily. If I get ransomware on wednesday, I only lose wednesday's work. Which is ok because I will be spending wednesday getting it sorted. This is my PC. No derpy users who visit dodgy websites. No public facing services.

[Lady Fitzgerald]

Just now, GlobalPentahedron Corp said:

no its ok i rotate them daily. If I get ransomware on wednesday, I only lose wednesday's work. Which is ok because I will be spending wednesday getting it sorted. This is my PC. No derpy users who visit dodgy websites. No public facing services.

Malware can infect anyone, no matter how careful they are, and often is time delayed. You may not catch it in time before you rotate the drives. Why take chances?

[GlobalPentahedron Corp]

7 minutes ago, Lady Fitzgerald said:

Malware can infect anyone, no matter how careful they are, and often is time delayed. You may not catch it in time before you rotate the drives. Why take chances?

ok so we are talking about 8 hours of computer use on a normal day. Whether I keep the back up in all day or spend my precious time disconnecting it does not make a difference. In either case I still have yesterday's backup offline and I have only lost a few hours of work. My new priority is getting my computer working so I can do future work, not pissing about over < 8 hours of missed work.

[Lady Fitzgerald]

3 minutes ago, GlobalPentahedron Corp said:

ok so we are talking about 8 hours of computer use on a normal day. Whether I keep the back up in all day or spend my precious time disconnecting it does not make a difference. In either case I still have yesterday's backup offline and I have only lost a few hours of work. My new priority is getting my computer working so I can do future work, not pissing about over < 8 hours of missed work.

Five minutes to connect and disconnect a backup drive two or three times a day is worse than having to recreate eight hours of work?

[GlobalPentahedron Corp]

1 minute ago, Lady Fitzgerald said:

Five minutes to connect and disconnect a backup drive two or three times a day is worse than having to recreate eight hours of work?

Yes 8 hours of work can be tolerated. Disconnecting the drive several times a day will not stop ransomware. If I get ransomware at 3pm then what does it matter if I disconnected my drive earlier in the day? That is impractical & no gain.

[Lady Fitzgerald]

Just now, GlobalPentahedron Corp said:

Yes 8 hours of work can be tolerated. Disconnecting the drive several times a day will not stop ransomware. If I get ransomware at 3pm then what does it matter if I disconnected my drive earlier in the day? That is impractical & no gain.

You keep missing the point. Good luck; you'll need it.

[alex75871]

If you care about your data then get an entry level low-power server platform e.g. Supermicro with embedded Atom server processor. Use ECC RAM and implement ZFS. If your hardware dies (unlikely because your hardware is designed for 24/7 operation with server reliability), you can import your volume on a new device. Configure your NAS to snapshot your data for easy recovery in the event of ransomware attack.

 

Here's a cheap and effective starting point: https://supermicro.com/products/motherboard/atom/A2SDi-2C-HLN4F.cfm

Features: 2 core atom, ECC support, quad intel gigabit NICs, server-grade components, internal USB 3 port for FreeNAS or similar, and VERY low-power consumption.