Have you actually tested your RAID setup?

[MG2R]

I was chatting with @alpenwasser and @Whaler_99 about RAID and

it occured to us that we didn't really test our RAID setup while in

operation, because of a small fear that if it doesn't work as expected,

everything is lost.

 

I decided to take a leap of faith and unplug one of my drives while two

10 GB+ movies were playing concurrently from my NAS. After a small

pause (~10 sec), everything resumed without any issue.

 

Broken array: http://puu.sh/806hg.png

 

It's rebuilding right now: http://puu.sh/807Nn.png

 

Have you actually tested your storage setup? Are you willing to try?

 

 

Disclaimer: any and all lost data is solely the user's fault. I shall not be held accountable for your own stupidity. Now unplug that drive!

[Whaler_99]

I do - running RAID4 on unraid means I rely heavily on that one parity drive. So I run recalculations of parity weekly and on occasion yank out a disk to test - typically once every six months...

 

Again, not something I would recommend you do if you are unsure.

 

Also, some "software" solutions, like Windows Storage Spaces, can go for a real dump when you yank a disk. PITA clearing out the old drive just to add it back in.

 

Also - make sure your system supports you "yanking" a disk out, like hot swap. If it doesn't, you can fry the drive, the port... or worse case, da board. 

 

Which end is up on these things?

[megadarkwood]

I would, IF I HAD ONE!

[MatheusSanzo]

No.

 

But i only run 2 drives in raid 1, if anything happens i'll just mount the array with one drive and then i'll see what i'll do.

[Str_Mape]

I don't have RAID but I do have a 10 TB windows storage space (soon to be 13 TB) and have done no testing.

[MG2R]

I don't have RAID but I do have a 10 TB windows storage space (soon to be 13 TB) and have done no testing.

Go for it

[alpenwasser]

I think once APOLLO is up and running I shall try this with ZFS on the pool

of which I actually have a proper offline backup. Might be a while until I

get to that point though.

[Assassin]

I was chatting with @alpenwasser and @Whaler_99 about RAID and

it occured to us that we didn't really test our RAID setup while in

operation, because of a small fear that if it doesn't work as expected,

everything is lost.

 

I decided to take a leap of faith and unplug one of my drives while two

10 GB+ movies were playing concurrently from my NAS. After a small

pause (~10 sec), everything resumed without any issue.

 

Broken array: http://puu.sh/806hg.png

 

It's rebuilding right now: http://puu.sh/807Nn.png

 

Have you actually tested your storage setup? Are you willing to try?

 

 

Disclaimer: any and all lost data is solely the user's fault. I shall not be held accountable for your own stupidity. Now unplug that drive!

I have a RAID 0 for my OS and I can only imagine what would happen if I pull the plug. Nope I am going to chicken out of this one.

[Hoppa]

ummmm.... naaaa... I'll be right 

Let's just hope it works when I need it haha

[topsu]

What raid testing actually does?

[alpenwasser]

What raid testing actually does?

Basically you break part of your RAID and see if it can successfully

rebuild itself.

[Str_Mape]

Go for it

I don't want to break my striped storage space...

[IdeaStormer]

Testing is the first thing you do when setting up RAID or before actually installing the OS. Can run it off a LiveOS memory stick/CD/DVD, yank a drive and simulate what you will have to do to get it backup and running with some sample data or install the OS if you want. You need to simulate and document what you will need to get it back on line. I just cringe when I hear the story of people who buy a raid controller and drives get it setup and then go away from it as if its going to last forever. Then one day, it fails and their world falls apart since no one documented how to recover then someone attempts the wrong thing and its all gone forever, or they pay a good fortune to get it all back, when they could of just documented it and it might of been just add a new drive and run one command and wait a few hours and everything back as it was.

 

Asside:

Seeing its a software RAID it explains so much

[wpirobotbuilder]

Once I have my ZFS build I will be testing drive and controller failures. If I have an ESXi host, then I will be testing whether or not Fault Tolerance works with a failed NFS export.

[alpenwasser]

- snip -

 

Yup, the best tool is of no use if you don't actually know how to use it or

have verified that it's in good enough condition to do its job.

 

Once I have my ZFS build I will be testing drive and controller failures. If I have an ESXi host, then I will be testing whether or not Fault Tolerance works with a failed NFS export.

Once I'm fully set up and have proper backups of the important bits again,

I will definitely do some breakage in ZFS and see how things go. Want to be

prepared for when poopadilly actually does make contact with the ventilation

device.

[MG2R]

What raid testing actually does?

RAID is a system that takes multiple independant

hard drives and combines them in an array (or pool)

in order to provide either more speed or better data

protection (or both). RAID 5, for example can still

keep functioning like nothing happened when one

drive has died or has been removed from the array.

 

Testing your RAID array basically means that you

yank out as many drives as the RAID level you're

using allows and see if the array still persists.

 

In my example, I unplugged one of the four drives

in my RAID 5 array. Everything worked as expected

and right now, the RAID has rebuilt itself onto the

drive overnight. No data loss has occured and the

array is ready to protect my data again.

[wpirobotbuilder]

Want to be prepared for when poopadilly actually does make contact with the ventilation

device.

 I think that's the nicest way I've ever heard someone use that expression.

[wpirobotbuilder]

What raid testing actually does?

 

See what alpenwasser wrote. In addition, you want to document your recovery procedure and ideally break/rebuild your system multiple times so that, in the event of a failure, you know exactly what to do to successfully recover your data and rebuild your system in the least amount of time possible with the smallest chance of total failure.

 

Data loss sucks, do it right the first time and you won't have to worry.

[ShadowCaptain]

No, im running in raid 10, but only because I dont really know how to test, should I just power down and remove a drive and see what happens?

[IdeaStormer]

No, im running in raid 10, but only because I dont really know how to test, should I just power down and remove a drive and see what happens?

 

RAID 10 testing is easy, yank a drive out watch the error messages pop out, wipe the drive you took out on another computer removing any RAID settings on it (low level format). Then, add the drive back in as if its a new drive (its now blank due to wiping it), rebuild the array.

[ShadowCaptain]

RAID 10 testing is easy, yank a drive out watch the error messages pop out, wipe the drive you took out on another computer removing any RAID settings on it (low level format). Then, add the drive back in as if its a new drive (its now blank due to wiping it), rebuild the array.

Thanks I will try that sometime soon, there is barely any data on my server so even if i destroy it I wont lose anything, worth a test to make sure its all set up right

[alpenwasser]

I think that's the nicest way I've ever heard someone use that expression.

Haha, themodzoo.com has a rather funny profanity filter, I suppose it has

rubbed off on me to some degree.

[Whaler_99]

Again, remember when "testing" your array - MAKE SURE your system supports hot swat. If you have a power and sata cable on the backs of all those drives, IT DOESN'T.

 

Typically this will only be drives in hot swap caddies, and even then the controller and OS must support it. 

 

Yanking can fry a lot of components.

 

Power down, unplug, power up and watch the errors... and hopefully interim recovery.

[tycheleto]

I haven't really tested removing drives on the fly, but in the past few months I've recovered from 2 drive failures. The first failure was a slow failing drive, and I wasn't sure so I shut everything down once it failed, and waited till I got a replacement disk. The second failure I was able to remove the disk and have the rest of the array run fine. Required a restart due to flexraid, but at least I could mostly minimize downtime.

[MG2R]

Yanking can fry a lot of components.

Eh, I just went for it... No guts, no glory!