Home server RAID 6 setup with a hardware RAID controller & online backup services

[MikkoP]

Hi all!

 

My storage space on my server is running low and was thinking of rebuilding the whole storage part of the server. At the moment, I have two 2 TB WD Greens in RAID 1 and some smaller WD drives without RAID. I was thinking of ditching the current drives and buying 4 TB WD Reds. With these I was thinking of going for RAID 6 for good reliability. I don't want to use software RAID this time and instead go for hardware RAID controllers.

 

So, for the biggest problem here. What RAID cards do you recommend? 8 internal SATA lanes sound good, in RAID 6 with 4 TB drives that would mean 24 TB of storage which is more than enough. With a four-lane controller I could get 8 TB in RAID 6 or 12 TB in RAID 5. Though, I think these are going to get filled up quickly. Currently I have a copying process in progress: I'm copying all my movies, TV series and music to hard drives for a Plex media server. At the moment I'm 30 % done with my movies and I have almost 2 TB of data. 

 

Good hardware RAID controllers with support for RAID 5 and 6 are expensive. There are some cheaper ones too, but aren't they a lot worse? You can get a HighPoint RockerRAID 2720SGL for 160 $. Then there are the expensive ones, like LSI MegaRAID 9260-8i and Adaptec 6805. I also found one from Areca called ARC-1224-8i. All these seem to be high performance cards but with a very salty price tag.

 

Any experience with these? Are they reliable? Do they perform well? What kind of reading and writing speeds do these give in RAID 6 with those drives?

 

And what about in case of a failure? If the controller dies, I've read that I can just buy another one and import the settings. Is that right? And in case of a disk failure, I just swap the broken one and let the card rebuild the parity data on the disks?

 

For additional security I will be using a cloud backup service like CrashPlan to back up the entire RAID array. What are your takes on these backup services? What do you use and recommend? Why?

[MikkoP]

No views? Wow! (Edit: after posting this, the view count went up to 83...)

 

But to the point. Those better RAID controllers support an additional battery unit. Does this do anything else but keeps the data in the cache when the power is lost? Someone claimed it would give additional performance benefits on the LSI device, but I highly doubt it. If I'll be running the server off from an UPS, am I right thinking I don't need a ridiculously expensive battery unit?

[N7_Zer0]

I would not use anything other than LSI cards. Adaptec and Highpoint may be cheap but you get what you pay for. The BBU allows you safely enable Write Back Caching which speeds up writes to the array dramatically. Without the BBU, you can still enable Write Back but you will lose any data that's still in the cache if there's a power failure. In my server I have a 9260-4i with 4 x 2TB Reds in RAID-5, very fast and reliable. The 8i is the same card with an additional port for 4 more drives.

WRITE BACK-----------------------------------------------------------------------CrystalDiskMark 3.0.3 Shizuku Edition x64 (C) 2007-2013 hiyohiyoCrystal Dew World : http://crystalmark.info/-----------------------------------------------------------------------* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]Sequential Read : 477.566 MB/sSequential Write : 418.482 MB/sRandom Read 512KB : 69.523 MB/sRandom Write 512KB : 89.564 MB/sRandom Read 4KB (QD=1) : 0.864 MB/s [ 210.8 IOPS]Random Write 4KB (QD=1) : 1.938 MB/s [ 473.1 IOPS]Random Read 4KB (QD=32) : 6.485 MB/s [ 1583.2 IOPS]Random Write 4KB (QD=32) : 2.002 MB/s [ 488.7 IOPS]Test : 1000 MB [D: 0.0% (0.3/5587.4 GB)] (x1) <0Fill>Date : 2014/06/08 14:38:06OS : Windows Server 2008 R2 SP1 [6.1 Build 7601] (x64)WRITE THROUGH-----------------------------------------------------------------------CrystalDiskMark 3.0.3 Shizuku Edition x64 (C) 2007-2013 hiyohiyoCrystal Dew World : http://crystalmark.info/-----------------------------------------------------------------------* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]Sequential Read : 462.879 MB/sSequential Write : 68.374 MB/sRandom Read 512KB : 66.908 MB/sRandom Write 512KB : 27.617 MB/sRandom Read 4KB (QD=1) : 0.854 MB/s [ 208.5 IOPS]Random Write 4KB (QD=1) : 0.397 MB/s [ 96.8 IOPS]Random Read 4KB (QD=32) : 6.185 MB/s [ 1509.9 IOPS]Random Write 4KB (QD=32) : 1.289 MB/s [ 314.7 IOPS]Test : 1000 MB [D: 0.0% (0.3/5587.4 GB)] (x1) <0Fill>Date : 2014/06/06 21:53:54OS : Windows Server 2008 R2 SP1 [6.1 Build 7601] (x64)

[MikkoP]

 

I would not use anything other than LSI cards. Adaptec and Highpoint may be cheap but you get what you pay for. The BBU allows you safely enable Write Back Caching which speeds up writes to the array dramatically. Without the BBU, you can still enable Write Back but you will lose any data that's still in the cache if there's a power failure. In my server I have a 9260-4i with 4 x 2TB Reds in RAID-5, very fast and reliable. The 8i is the same card with an additional port for 4 more drives.

 

Thanks for your reply. 

 

But, if I have an UPS, I don't have to be afraid of power losses. If I have a good UPS with high battery capacity, I can run the server for 15 minutes without AC power from the wall. In this time, the cache data would be stored on the disks and wouldn't be lost. UPS softwares will power down the machine eventually, preventing it from refilling the cache.

 

Or is there something I'm missing?

 

As I am on sort of a budget, spending 150-200 dollars for the small battery seems like a waste when there's an UPS.

[N7_Zer0]

Thanks for your reply. 

 

But, if I have an UPS, I don't have to be afraid of power losses. If I have a good UPS with high battery capacity, I can run the server for 15 minutes without AC power from the wall. In this time, the cache data would be stored on the disks and wouldn't be lost. UPS softwares will power down the machine eventually, preventing it from refilling the cache.

 

Or is there something I'm missing?

 

As I am on sort of a budget, spending 150-200 dollars for the small battery seems like a waste when there's an UPS.

 

The battery is an extra layer of security. What if the PSU dies, or the UPS? What if you're doing a large data transfer that takes more time than what the UPS can provide in a power failure? The shutdown signal can fail too.

 

If it's mission critical data for example you'll want as much security as you can get. Keep in mind that raid cards and BBU's aren't meant for your every day home user however they are essential in a business environment. In most cases, UPS is just fine.

[IdeaStormer]

I like your first idea of getting rid of the 2TB Green drives

 

The difference in RAID card's is more than brand name recognition, you need to look at the fine print for the features they support or provide. Some like RAID 0,1, ... will be the obvious ones but there are many more smaller features you get or don't which factor in the price. Asking for a recommendation is really a rather wide question and answer as we don't really know how you want to use it other than in RAID 6.

 

Then the battery module is for protection from any power issue or sudden shutdown, unless you program your UPS and computer to flush the RAID cache you will still get data loss on a power loss because the RAID card does not know what is happening when you go UPS or how long that will last.

[MikkoP]

The battery is an extra layer of security. What if the PSU dies, or the UPS? What if you're doing a large data transfer that takes more time than what the UPS can provide in a power failure? The shutdown signal can fail too.

 

If it's mission critical data for example you'll want as much security as you can get. Keep in mind that raid cards and BBU's aren't meant for your every day home user however they are essential in a business environment. In most cases, UPS is just fine.

 

What if the battery backup fails? Well, that's unlikely, but so is the UPS failing too. Maybe I check that later.

 

I like your first idea of getting rid of the 2TB Green drives

 

The difference in RAID card's is more than brand name recognition, you need to look at the fine print for the features they support or provide. Some like RAID 0,1, ... will be the obvious ones but there are many more smaller features you get or don't which factor in the price. Asking for a recommendation is really a rather wide question and answer as we don't really know how you want to use it other than in RAID 6.

 

Then the battery module is for protection from any power issue or sudden shutdown, unless you program your UPS and computer to flush the RAID cache you will still get data loss on a power loss because the RAID card does not know what is happening when you go UPS or how long that will last.

 

Well, for home use there's not much I need. As high speeds as possible are good and RAID 6 is a must. Everything else is optional if it can support those and 8 drives.

[N7_Zer0]

What if the battery backup fails? Well, that's unlikely, but so is the UPS failing too. Maybe I check that later.

 

If the BBU fails, then you have the UPS covering it. The likelihood of the BBU, PSU, and UPS failing at the same time is extremely low BTW.

 

Like I said, it's more of a business (non-home user) thing. In my company there are redundant systems for redundant systems. If anything goes down, there's always some type of backup. The servers we have here all have BBU's and redundant PSU's which are hooked up to separate UPS.

[MikkoP]

If the BBU fails, then you have the UPS covering it. The likelihood of the BBU, PSU, and UPS failing at the same time is extremely low BTW.

 

Like I said, it's more of a business (non-home user) thing. In my company there are redundant systems for redundant systems. If anything goes down, there's always some type of backup. The servers we have here all have BBU's and redundant PSU's which are hooked up to separate UPS.

 

Yea, more a enterprise thing. I'll consider it but now I know it's not mandatory.

[MikkoP]

After doing a lot more research, I have some more questions.

 

LSI also has 9266 and 9271 cards. 9260 compared to these has less cache memory (512 MB vs 1024 MB, there's also speed difference) and a single core ROC when the newer models have two cores. As what I have read, the difference between 9266 and 9271 is that the 9271 is PCIe 3.0 and the other one is 2.0. The price difference is very small, and the 9266 and 9271 are over two years newer, so I think I could go with one of those. 9271 is about 30 € more expensive and gives me just PCIe 3. If I'm not able to max out PCIe 2, I can buy the 9266.

 

One other thing I found is SAS expanders. For 280 $ on Newegg I can buy (actually can't, they don't ship here..) Intel RES2SV240. It says it is a 24 port 6 Gb/s per port expander. Does this mean, that even with a 8 port controller I can get 6 Gb/s on all the hard drives that are connected to the expander? Or, am I limited to 8 ports at full speed or 24 ports on 1/3 speed? If I don't need the eight ports on the RAID card to get the speed, I could go with a 4 port controller. This expander and 4 port 9266 or 9271 together equal just a little more than just a 8 port 9266 or 9271. As a side benefit I would be future proof with 16 more ports for future expansion. With 3 TB drives and 8 port controller in RAID 6 I'd get 18 TB of usable storage space. With 24 drives I could get 66 TB of storage - way over what I need.

 

What about connecting the expander? I read that I need to use the SAS ports to link the two cards together. Is the PCI connector just for power and holding it still in place? It also has a molex connector that doesn't need to be hooked up when it's plugged in to a motherboard. This makes me believe that it's not mandatory to connect it to the motherboard.

 

Clarification, suggestions and explanation of these is very appreciated and welcome.

 

 

@_Zer0_ Just giving you a notice, you seem to know of these

[MikkoP]

The expander's bandwidth is limited by the internal SATA lane count of the controller. So, in case of 4i its 24 Gb/s (4 x 6 Gb/s).