Jump to content

Reducing Single Points of Failure (SPoF) in Redundant Storage

First xD

but seriously this is a great read, very informative

And how are there no replies after 2 days 0.o

<p>Wires Suck :angry:
!fY0|_|(4|\|R34[)7#!5PMM37#3(0[)3:1337 70833|\|73R3[)!|\|49!\/34\|/4Y 4|\|[)93741!f3

 

Link to comment
Share on other sites

Link to post
Share on other sites

First xD

but seriously this is a great read, very informative

And how are there no replies after 2 days 0.o

 

Not as interesting or exciting as discussions on CPUs or graphics cards.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

I didn't read the entire guide line by line, more so 3/4 of it, but it is super informative and I enjoyed reading it.

 

very well done

Main rig: Coursair Air 540, Amd 8120(stock clock, had it up to 4.6), Gigabyte 990 fx ud3,  MSI DCU2 r9 280x, seasonic 1000w platinum, Samsung 830 64 GB ssd, Seagate 1tb, 4 GB (2x2) G.Skill, 

 

Secondary: Define r4, Amd A10 5800k(4.6), 1x Sapphire 7950, MSI FM2-A85xa, 8 GB coursair vengeance ram, Seasonic 1250w Gold, 500GB 840 Evo, 223GB 2.5 Hdd, 80 Gb Seasonic, Samsung CD/dvd

Link to comment
Share on other sites

Link to post
Share on other sites

  • 1 month later...
  • 2 weeks later...

Just use unRAID, no need to worry about any controller failure, can survive on 1 harddrive failure, even multiple harddrive failure relatively easy to recover

Link to comment
Share on other sites

Link to post
Share on other sites

  • 3 weeks later...

For RAID 7, we have 3 controllers which are all fully utilized:

 

attachicon.gifRAID 7.JPG

It says RAID 5. 

Just saying.

† Christian Member †

For my pertinent links to guides, reviews, and anything similar, go here, and look under the spoiler labeled such. A brief history of Unix and it's relation to OS X by Builder.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

It says RAID 5. 

Just saying.

 

comics-ouija-board-434125.jpeg

Fixed.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

OK, I've held back a bit after first reading and am re-reading for some other thing (cough cough @Vitalius).

 

Which controller allows crossing RAID arrays?

I roll with sigs off so I have no idea what you're advertising.

 

This is NOT the signature you are looking for.

Link to comment
Share on other sites

Link to post
Share on other sites

OK, I've held back a bit after first reading and am re-reading for some other thing (cough cough @Vitalius).

 

Which controller allows crossing RAID arrays?

No idea if a controller allows it (unless you can connect a controller to a controller). But I assume you could have one side be hardware and the other be software. So use the controllers for the first level, then software for the second level (i.e. RAID 1 on the controllers, and RAID 0 in the software).

However, Software RAID (specifically FreeNAS, and FlexRAID [i assume]) would easily allow it. 

Just create two RAID 1 arrays of two different set of drives, for example, in a zpool and RAID 0 them together as a single VDev (if that's how it works, not sure if they'd be 3 separate vdevs or a single vdev). 

Not sure how you would do it for RAID 5, 6, or 7 (probably more complicated), but with RAID 1 and 0, it should be relatively easy.

SCSI controllers might allow you to do that, but again, I am guessing (because it's enterprise level stuff) that is has such a feature.

Also, thanks Idea. :)

† Christian Member †

For my pertinent links to guides, reviews, and anything similar, go here, and look under the spoiler labeled such. A brief history of Unix and it's relation to OS X by Builder.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

No idea if a controller allows it (unless you can connect a controller to a controller). But I assume you could have one side be hardware and the other be software. So use the controllers for the first level, then software for the second level (i.e. RAID 1 on the controllers, and RAID 0 in the software).

However, Software RAID (specifically FreeNAS, and FlexRAID [i assume]) would easily allow it. 

Just create two RAID 1 arrays of two different set of drives, for example, in a zpool and RAID 0 them together as a single VDev (if that's how it works, not sure if they'd be 3 separate vdevs or a single vdev). 

Not sure how you would do it for RAID 5, 6, or 7 (probably more complicated), but with RAID 1 and 0, it should be relatively easy.

SCSI controllers might allow you to do that, but again, I am guessing (because it's enterprise level stuff) that is has such a feature.

Also, thanks Idea. :)

 

Yes, Software RAID can but all this talk of SPoF and you throw in the "Intel chipset (with 4 ports available), and ASMedia chipset (with 4 ports), and an LSI 9211-4i" as a viable set of controllers to mix <insert mother of ghod meme> :D  , that's sure is reducing the SPoF but adding a cluster @#$@ of controllers that most likely will belly up any RAID array. Good in theory but in reality I'd run away from that system faster than a Over Clocked Liquid Nitrogen cooled CPU.

 

Best to keep any discussion on SPoF back with trusty, tested, and known controllers and drives, if not just Controller X. The LSI I'd keep in the mix but maybe two of them and then maybe look for others depending on budget, but for FreeNAS, FlexRAID basically ZFS an HBA is best.

 

There might be a RAID controller out there that can do the spanning but probably has to have equal brother controllers in the span, and for a nice price.

 

The Theory is good (@wpirobotbuilder), but I think most on LTT need to see it converted to more examples as most are just getting the feel for RAID on its own. One good add on would be an example build so they can see it lay-ed out in terms of actual components. Just a suggestion, not an arm twist.

I roll with sigs off so I have no idea what you're advertising.

 

This is NOT the signature you are looking for.

Link to comment
Share on other sites

Link to post
Share on other sites

-snip-

Ah, well for my personal project, I would use two (identical) RAID controllers and never use the onboard SATA ports if possible (except maybe for the boot drive). 

I'd imagine using that many different controllers wouldn't matter until the actual RAID broke. Then, if the failure was one of the controllers, like a motherboard one, you'd be ... close to SOL. It might work. Not enough experience with RAID to say, but I would assume it would. And admittedly, he did say the motherboard failing would probably mean catastrophic failure and a prior backup would be needed.

wpirobotbuilder I agree with IdeaStormer. I don't understand how you would set up the actual RAIDs themselves. Such as in the last picture with the controller examples. I'm assuming all examples are software RAID (except where hardware RAID is actually possible), because, unless you can somehow span RAID arrays across controllers like that via hardware, there is no other option. And Software RAID could do that fairly easily. 

I could be wrong though. There are still connectors on the motherboard and certain cards that I don't recognize that could somehow synchronize controllers. Which would be sweet but weird and I question how much I would be able to trust something like that with RAID and recovery.

† Christian Member †

For my pertinent links to guides, reviews, and anything similar, go here, and look under the spoiler labeled such. A brief history of Unix and it's relation to OS X by Builder.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

OK, I've held back a bit after first reading and am re-reading for some other thing (cough cough @Vitalius).

 

Which controller allows crossing RAID arrays?

It is not the physical controller which handles the RAID array. The controllers are merely providing the SATA ports for drives which the controlling software (FreeNAS, ZFS on Linux, FlexRAID, etc) uses to build its RAID volumes.

 

The idea is that a RAID volume, controlled by this software, should be spread out over multiple physical 'controllers' (which are just providing SATA ports) such that, if one fails, the loss will not destroy any RAID volumes.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Updated to clarify that all RAID here is software RAID, and explain that controllers only provide SATA ports.

 

Changed some punctuation and wording.

 

Added tags to the post.

 

Also consolidated individual sections into spoilers. Saves a lot of TL;DR.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Yes, Software RAID can but all this talk of SPoF and you throw in the "Intel chipset (with 4 ports available), and ASMedia chipset (with 4 ports), and an LSI 9211-4i" as a viable set of controllers to mix <insert mother of ghod meme> :D  , that's sure is reducing the SPoF but adding a cluster @#$@ of controllers that most likely will belly up any RAID array.

Not quite, since the loss of a single controller will not damage data in any way. Also, there are some very efficient configurations like RAID 10 which only ever require two controllers.

 

With that said, you WILL have to replace a dead controller ASAP to protect against further data loss.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Not quite, since the loss of a single controller will not damage data in any way. Also, there are some very efficient configurations like RAID 10 which only ever require two controllers.

 

With that said, you WILL have to replace a dead controller ASAP to protect against further data loss.

So when you say it won't damage data in any way, does that mean the system can run as if nothing happened (albeit with some performance loss) while some drives are down due to either drive failure or controller loss, so long as it isn't the scenario that causes data loss? 

i.e. I have a RAID 10 array. One drive in each RAID 1 dies. Will it act like nothing happened and keep running? I'd assume so. I ask this question both for data corruption, and uninterrupted access related reasons.

† Christian Member †

For my pertinent links to guides, reviews, and anything similar, go here, and look under the spoiler labeled such. A brief history of Unix and it's relation to OS X by Builder.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

So when you say it won't damage data in any way, does that mean the system can run as if nothing happened (albeit with some performance loss) while some drives are down due to either drive failure or controller loss, so long as it isn't the scenario that causes data loss? 

i.e. I have a RAID 10 array. One drive in each RAID 1 dies. Will it act like nothing happened and keep running? I'd assume so. I ask this question both for data corruption, and uninterrupted access related reasons.

Unless I'm missing something big, that is the case. However you'd better replace the drives quickly.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Unless I'm missing something big, that is the case. However you'd better replace the drives quickly.

Awesome, and understood. I just wanted to know what my options would be if a server like that did have drives that failed and we were in the middle of something fairly important.

† Christian Member †

For my pertinent links to guides, reviews, and anything similar, go here, and look under the spoiler labeled such. A brief history of Unix and it's relation to OS X by Builder.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

Not quite, since the loss of a single controller will not damage data in any way. Also, there are some very efficient configurations like RAID 10 which only ever require two controllers.

 

With that said, you WILL have to replace a dead controller ASAP to protect against further data loss.

 

I think you misunderstood my post, I'm not saying the Theory of multiple controllers will ruin the RAID, but those specific different controller brands used together.

I roll with sigs off so I have no idea what you're advertising.

 

This is NOT the signature you are looking for.

Link to comment
Share on other sites

Link to post
Share on other sites

I think you misunderstood my post, I'm not saying the Theory of multiple controllers will ruin the RAID, but those specific different controller brands used together.

I don't think the controllers matter that much, since they just provide SATA ports and don't communicate with each other. I could be wrong, though.

 

If one bought a really crappy controller then that might break the RAID array quickly, but I don't think specific combinations of controllers spells D-O-O-M for the array.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

I don't think the controllers matter that much, since they just provide SATA ports and don't communicate with each other. I could be wrong, though.

 

If one bought a really crappy controller then that might break the RAID array quickly, but I don't think specific combinations of controllers spells D-O-O-M for the array.

 

So have you used that combination of controllers? Just curious on your results with those if so.

I roll with sigs off so I have no idea what you're advertising.

 

This is NOT the signature you are looking for.

Link to comment
Share on other sites

Link to post
Share on other sites

So have you used that combination of controllers? Just curious on your results with those if so.

I haven't, but I know that this build uses the Intel chipset, an LSI card, and (I believe) a 3rd party onboard controller.

 

It's got some performance numbers as well.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 weeks later...

Added some supplementary pictures for the example configuration towards the end. They outline exactly which drives would die for every controller that might die, and show what would be left to keep the volumes running.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 months later...

Very Informative and helpful.

 

But if I may ask.

Let's say my MB have 6 sata ports and i want to add more but don't want to have spof (single point of failure) and want to expand to 7 drives or more.

So I should get a raid controller (raid card like LSI) but not setting up the raid and instead use only software raid?

 

*sorry for the noob question my english is not good.

MB :MSI Z77a G45 | Proc: I5 3570K (Stock) | HSF : CM 212X turbo | RAM : Corsair Vengeance 8GB (2X4GB) | VGA : MSI GTX 660 Twin Frozr | PSU : Corsair GS600 | Case : CM Storm Enforcer | Storage :  OCZ Vector 128GB, WD Blue 500GB , Samsung 840 Evo 120GB, WD Blue 1TB

Link to comment
Share on other sites

Link to post
Share on other sites

Very Informative and helpful.

 

But if I may ask.

Let's say my MB have 6 sata ports and i want to add more but don't want to have spof (single point of failure) and want to expand to 7 drives or more.

So I should get a raid controller (raid card like LSI) but not setting up the raid and instead use only software raid?

 

*sorry for the noob question my english is not good.

That could work. Removing the controllers as a SPoF is dependent on which drives (i.e. which SATA ports) you are using. For example, if all the drives you used for a RAID 6 array were attached to the LSI controller, that controller would be a SPoF. If you plan your RAID volume correctly, then you will avoid SPoFs.

 

If you want to see what can be done with two controllers (like you have), then go back and look at the configurations involving exactly two controllers. Those RAID configurations will be your possible options. In terms of ease of expansion, I highly recommend RAID 10. While you only get 50% of the total drive space, you only need two controllers and can throw in another RAID 1 to expand very quickly. You could expand up to twelve drives with that, getting 6 worth of usable capacity.

 

If you anticipate needing more than 18TB of storage, then you will likely need more than one LSI controller to keep avoiding SPoFs, regardless of the RAID configuration you choose.

 

P.S. RAID controllers need to be flashed from IR to IT mode to be fully compatible with ZFS/FreeNAS. See alpenwasser's tutorial on how to do that for the LSI 9211-8i here. BTW that is the most recommended controller to use, or the 9211-4i.

 

If you need more help, don't hesitate to ask.

I do not feel obliged to believe that the same God who has endowed us with sense, reason and intellect has intended us to forgo their use, and by some other means to give us knowledge which we can attain by them. - Galileo Galilei
Build Logs: Tophat (in progress), DNAF | Useful Links: How To: Choosing Your Storage Devices and Configuration, Case Study: RAID Tolerance to Failure, Reducing Single Points of Failure in Redundant Storage , Why Choose an SSD?, ZFS From A to Z (Eric1024), Advanced RAID: Survival Rates, Flashing LSI RAID Cards (alpenwasser), SAN and Storage Networking

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×