Software or Hardware Raid? Need suggestions

[downloadedskill]

So I just built a FreeNas server out of my now retired PC. Figured it wouldn't be to difficult since all I wanted to do was run plex and some kind of FTP or private cloud like OwnCloud(Google Drive Clone). It seems I already lost my data after only a month of use from a crash and who knows what else. So I'm rethinking what I'm going to do next. I thought software raids were better then hardware but both seems to have big pros and cons. I have 6-3TB Reds that I could really like to have in a Raid6 setup for redundancy. I've thought about just getting an HBA and just running them as separate drives and only backing up what is super important. After throwing that around I really don't want to deal with having to do all those backups manually or get software to do it. So that puts me back to getting a raid card, maybe LSI? Unless I were to go with in OS software raid which I'm not super fond of. 

 

My OS selection is down to W7 or 10 or Ubuntu. They both are capable of running a plex server and some variation or 3rd part cloud software. That's really all I need out of this server. I would love to be able to remote in as well when not at home. To keep an eye on things.

 

So really I'm open to suggestions, I really just want something that's reliable and somewhat easy to maintain. Hardware/software suggestions are awesome too. 

 

Oh these are the server specs if that will change anyones suggestions.

i7 3770k
GA-B75M-D3H
24gb Ram Non-ECC
6x3tb WD Reds
850W PSU

[samiscool51]

ehardware, it's easyier to work with, some programges/os's need to see the drive's some RAID cards alow this, some software forces the os not to see the drives, also it installs drivers for it to work so unless you are useing RAID software you trust it might be a better option for you, but my personal opinion is hardware

[samiscool51]

sorry my English is shit

[downloadedskill]

28 minutes ago, samiscool51 said:

sorry my English is shit

It's all good dude. I appreciate it.

[leadeater]

FreeNAS should not have lost any data, something was definitely wrong somewhere be it configuration or hardware. Could you give a little more details on how you set it up?

 

As for the question about Software RAID vs Hardware RAID, FreeNAS/ZFS is not strictly software RAID mdadm is. Software RAID is never as good as Hardware RAID. When comparing to something like ZFS both as you mentioned have the pros and cons but if you are going for a very large configuration of many disks ZFS is the better option.

 

If you're going to stay below 8 disks an LSI RAID card is a good option, make sure you get the battery (BBU) or CacheVault Flash Cache else parity RAID configuration write performance will be rather slow. Hardware RAID 6 is also very simple to manage and replace disks if they fail, you don't have to do anything other than physically replace the disk the rebuild will happen automatically.

 

Windows also has a good solution which I currently use the server variant of both known as Storage Spaces. If using Windows 8/10 however do not use parity configuration as it is awfully slow on writes, you can use it but only if you configure a journal SSD to cache writes. Two-Way or Three-Way Mirror configurations performance very well and are very safe but you only get either half the usable space or a third so $/GB isn't great.

[downloadedskill]

10 minutes ago, leadeater said:

FreeNAS should not have lost any data, something was definitely wrong somewhere be it configuration or hardware. Could you give a little more details on how you set it up?

 

As for the question about Software RAID vs Hardware RAID, FreeNAS/ZFS is not strictly software RAID mdadm is. Software RAID is never as good as Hardware RAID. When comparing to something like ZFS both as you mentioned have the pros and cons but if you are going for a very large configuration of many disks ZFS is the better option.

 

If you're going to stay below 8 disks an LSI RAID card is a good option, make sure you get the battery (BBU) or CacheVault Flash Cache else parity RAID configuration write performance will be rather slow. Hardware RAID 6 is also very simple to manage and replace disks if they fail, you don't have to do anything other than physically replace the disk the rebuild will happen automatically.

 

Windows also has a good solution which I currently use the server variant of both known as Storage Spaces. If using Windows 8/10 however do not use parity configuration as it is awfully slow on writes, you can use it but only if you configure a journal SSD to cache writes. Two-Way or Three-Way Mirror configurations performance very well and are very safe but you only get either half the usable space or a third so $/GB isn't great.

Thanks I was looking at the  LSI 9260-8i card. Trying to look into it more and get more suggestions before I go a pull the trigger on something again. Need to see my options first. 

 

I haven't lost my data yet per say. I currently can't mount my array after multiple noob muck ups. I had it setup with a windows share and 3 jails(plugins) only 2 of which were running. I had a system crash while reinstalling a plex jail. Then was alerted my drives were in an unknown state then reset my server to factory defaults. I read a crap post somewhere and interpreted it poorly. Now I can't import my Raidz2 array for the issue of "One or more of your drives are currently unavailable.". Even though they all seem to be online and functional. Sucks because I just put some media on there that I would like to get back but doesn't seem like that's possible so far.

[tt2468]

I have found that in storage server applications, RAID is usualy faster than ZFS or software alternatives. I have tested with only raid10 and raid6 though. All of my tests were done with 600GB 10k SAS drives.

[samiscool51]

@LinusTech used the  LSI 9260-8i raid card in the SSD server which failed so maybe try another card see if that works

[leadeater]

5 minutes ago, samiscool51 said:

@LinusTech used the  LSI 9260-8i raid card in the SSD server which failed so maybe try another card see if that works

The card is very good, everything can fail.

 

@downloadedskill If you are going to switch to hardware RAID then yes that is a good card, remember to also get the BBU if it doesn't come with one.

 

 

[samiscool51]

1 minute ago, leadeater said:

The card is very good, everything can fail.

 

@downloadedskill If you are going to switch to hardware RAID then yes that is a good card, remember to also get the BBU if it doesn't come with one.

 

 

that's prob the best

[Captain_WD]

5 hours ago, downloadedskill said:

~snip~

Hi :)

I just want to pitch in that in case these WD Red drives were already used before it may be a good idea to check their health status before throwing them in a RAID6 array. A small problem with one of them may cause the array to be unstable and cause you a lot of headaches. Running WD Data Lifeguard Diagnostic and checking if all drives pass the quick and the extended tests. This will take some time on all drives but it would put you on the safe side that the drives are fine. Checking the raw values of their S.M.A.R.T. status is also a good idea. 

Post back if you have any questions :)

Captain_WD.

[leadeater]

1 hour ago, Captain_WD said:

-snip-

I see you have a new profile pic .

[downloadedskill]

7 hours ago, Captain_WD said:

Hi :)

I just want to pitch in that in case these WD Red drives were already used before it may be a good idea to check their health status before throwing them in a RAID6 array. A small problem with one of them may cause the array to be unstable and cause you a lot of headaches. Running WD Data Lifeguard Diagnostic and checking if all drives pass the quick and the extended tests. This will take some time on all drives but it would put you on the safe side that the drives are fine. Checking the raw values of their S.M.A.R.T. status is also a good idea. 

Post back if you have any questions :)

Captain_WD.

Thanks man. I'm definitely going to do that. I will. 

[downloadedskill]

11 hours ago, leadeater said:

The card is very good, everything can fail.

 

@downloadedskill If you are going to switch to hardware RAID then yes that is a good card, remember to also get the BBU if it doesn't come with one.

 

 

If I do have a hardware failure like the card is it in theory possible to just pop in a new card and go? Is that how that works? 

[Eniqmatic]

1 hour ago, downloadedskill said:

If I do have a hardware failure like the card is it in theory possible to just pop in a new card and go? Is that how that works? 

Hard to say without it happening/experience as it can often depend on the card. It's possible that if the card died (and somehow managed to do nothing to the array), that you could a new identical card in and import the array. If it detects it.

[downloadedskill]

46 minutes ago, Eniqmatic said:

Hard to say without it happening/experience as it can often depend on the card. It's possible that if the card died (and somehow managed to do nothing to the array), that you could a new identical card in and import the array. If it detects it.

Yea that's the only thing that scares me but pretty much the same thing happened to me in freenas so I guess nothing is truly sage without multiple backups. Lol LOVE your profile pic btw. HAMMOND!!! 

[scottyseng]

4 hours ago, downloadedskill said:

If I do have a hardware failure like the card is it in theory possible to just pop in a new card and go? Is that how that works? 

Usually yes, but it depends on a lot of things...like if the array itself was corrupted from the failure, the manufacturer's support, and which type of RAID level (some might be more sensitive to change). Also, in linus' case, somehow the motherboard and the RAID card both failed...Yeah, nothing is truly safe without backups.

 

As what @Captain_WD said, you want to double check all of the drives to make sure that the ZFS array failure wasn't because one of the WD drives died (it happens)...though ZFS still should've survived that.

 

I would advise trying to get the Cache Vault version of the LSI 9260-8i (Has a CV on the end). Cache Vault is better than a regular battery for a RAID card, it uses capacitors instead of a battery. I own the LSI 9260-8i CV and it's been fine for me for the past four years, despite quite a few power failures and several RAID array creations. I have six 4 TB WD Reds and four 4TB WD Re drives on this RAID card. If you're buying new though, I would push for the 9361-8i to be future proof (SAS 12Gb/s on the 93xx gen vs SAS 6Gb/s on the 92xx gen)

 

You also need to keep the RAID card cool with a fan in the near area of the RAID card, as they do run really hot since they're meant to be in a server with lots of airflow. You can get burned pretty bad grabbing the heatsink of the RAID card if it has no cooling and it was under load.

 

If you want to be even more sure, get a UPS unit for the NAS as well.

 

Overall though, there's both pros and cons as mentioned above to software RAID, specialized software RAID like ZFS, and hardware RAID. Hardware RAID costs a fair bit (I'm sure you've seen the cards aren't cheap..). I'm still surprised FreeNAS failed on you though.

[leadeater]

3 hours ago, scottyseng said:

You also need to keep the RAID card cool with a fan in the near area of the RAID card, as they do run really hot since they're meant to be in a server with lots of airflow. You can get burned pretty bad grabbing the heatsink of the RAID card if it has no cooling and it was under load.

10Gb NICs too, got caught out by that when I put one in my desktop. Computer is completely water cooled with very low airflow, went to pull it out to move the PCI-E slot it was using and OMFG ouch.

[leadeater]

7 hours ago, downloadedskill said:

If I do have a hardware failure like the card is it in theory possible to just pop in a new card and go? Is that how that works? 

I've pretty much always been able to migrate arrays between cards, even different models, but I've never actually had an LSI card fail. I've had 3ware cards fail but those are just shit so..... yea don't use those.

[scottyseng]

1 hour ago, leadeater said:

10Gb NICs too, got caught out by that when I put one in my desktop. Computer is completely water cooled with very low airflow, went to pull it out to move the PCI-E slot it was using and OMFG ouch.

Did not know that. I'll keep that in mind when I get some 10Gb/s cards after I buy a UPS. Yeah, I actually sanded the base of the LSI MegaRAID card heatsink and used some after market thermal paste for it, and wow, it transfers heat really good....like you I tried to move it to a new PCI-E slot and burned my thumb on the heatsink when I grabbed the card.

[downloadedskill]

3 hours ago, leadeater said:

I've pretty much always been able to migrate arrays between cards, even different models, but I've never actually had an LSI card fail. I've had 3ware cards fail but those are just shit so..... yea don't use those.

Of course I can't find the article now but I saw earlier on some post somewhere about some function raid cards have like RLT or something abbreviated. Which in that if sees a drive reallocating sectors it will dump the drive from the array. Is that accurate? I'm obviously not remembering everything exactly but that seems like not something I would want.

[leadeater]

15 minutes ago, downloadedskill said:

Of course I can't find the article now but I saw earlier on some post somewhere about some function raid cards have like RLT or something abbreviated. Which in that if sees a drive reallocating sectors it will dump the drive from the array. Is that accurate? I'm obviously not remembering everything exactly but that seems like not something I would want.

Yea a RAID card will do that as best it can, it relies on the disk SMART data for this to happen. This is how HP does predictive disk failures and you can get disks replaced under warranty before they actually fail.

 

A RAID card will also drop a disk from the array if it stops responding to I/O requests, this is the core reason why TLER is so recommended. Normal disks can get stuck in a sector read error where for example a WD Red won't as the TLER in the firmware prevents it.

[downloadedskill]

13 minutes ago, leadeater said:

Yea a RAID card will do that as best it can, it relies on the disk SMART data for this to happen. This is how HP does predictive disk failures and you can get disks replaced under warranty before they actually fail.

 

A RAID card will also drop a disk from the array if it stops responding to I/O requests, this is the core reason why TLER is so recommended. Normal disks can get stuck in a sector read error where for example a WD Red won't as the TLER in the firmware prevents it.

TLER that's it. So it essentially drops a disk when it senses it may fail or shows signs of failing is the idea? I just took it as it will dump it for any kind of reallocated sector or something that small. So I guess it's a good thing if you want to be careful and very much up on your drives health and not run them into the ground. I guess that would pose more problems in a raid then running it into the ground when it has the possibility to damage the rest of the array.

[scottyseng]

39 minutes ago, downloadedskill said:

TLER that's it. So it essentially drops a disk when it senses it may fail or shows signs of failing is the idea? I just took it as it will dump it for any kind of reallocated sector or something that small. So I guess it's a good thing if you want to be careful and very much up on your drives health and not run them into the ground. I guess that would pose more problems in a raid then running it into the ground when it has the possibility to damage the rest of the array.

Err, no, it drops a disk when it gets stuck on a error while reading data. Your average consumer drive (WD Blue, Black, Green) would try to fix the error itself and suspend any active activity while it corrects the error. That's okay in a non RAID environment, but with RAID arrays, usually controllers will consider the drive as failed if they don't respond within a set time frame. You have WD Reds, so TLER isn't a worry for you.

 

Yeah, you want a RAID card to have such a strict check though, as you definitely don't want the RAID array to go down / a warning to replace a drive is far better than a failed array.

[leadeater]

It's also good to know that a read error does not mean a disk is failed or even anywhere near failing, read errors can happen for many reasons. A disk will mark the sector as faulty and never use it again, SMART keeps track of this and when the number gets too high this indicates the disk is failing and next time you boot your computer your bios will warn you. For a RAID card it will tell you straight away as it monitors this constantly rather than just at boot or when you run a SMART checking utility. Don't rely too much on SMART though as it can often not pickup a disk about to fail.