Jump to content

How does RAID redundancy even work??

swordstoo
By swordstoo
in Storage Devices
 Share
Go to solution Solved by Prodigy_Smit,
1 minute ago, swordstoo said:

I understand that in RAID configurations you can use additional drives as a backup to the current drives so that if you lose one drive, you can simply replace the damaged drive and have it be rebuilt on the fly with no loss of data. Incredible!

 

However... how the hell does that even work? As an example, RAID will allow you to use 9 storage devices which are effectively 8 for parity striping and one for backup. How can a system distribute enough information to repair an entire drive of lost data on a single drive in an 8+1 drive system? Seems like magic to me. Obviously it's not, as I've run these parity+backup solutions effectively and had the magic of replacing a drive after a failure and have it just work.

 

It's probably complex and will go over my head, but I'd really like to know!

Basically your data is stored as an algebra equation. For example 1+2+3+4+5+6+7+8=36. If one of the drives fails such as drive 6, it can recalculate it, so if all of this is equal to this and one is missing what is that missing thing. 1+2+3+4+5+x+7+8=36

Posted

I understand that in RAID configurations you can use additional drives as a backup to the current drives so that if you lose one drive, you can simply replace the damaged drive and have it be rebuilt on the fly with no loss of data. Incredible!

 

However... how the hell does that even work? As an example, RAID will allow you to use 9 storage devices which are effectively 8 for striping and one for parity. How can a system distribute enough information to repair an entire drive of lost data on a single drive in an 8+1 drive system? Seems like magic to me. Obviously it's not, as I've run these parity+striping solutions effectively and had the magic of replacing a drive after a failure and have it just work.

 

It's probably complex and will go over my head, but I'd really like to know!

 

Edit: Fixed my incorrect terminology of parity/backup/striping

Link to comment
https://linustechtips.com/topic/1237985-how-does-raid-redundancy-even-work/
Share on other sites
Link to post
Share on other sites
Posted
  • Solution
1 minute ago, swordstoo said:

I understand that in RAID configurations you can use additional drives as a backup to the current drives so that if you lose one drive, you can simply replace the damaged drive and have it be rebuilt on the fly with no loss of data. Incredible!

 

However... how the hell does that even work? As an example, RAID will allow you to use 9 storage devices which are effectively 8 for parity striping and one for backup. How can a system distribute enough information to repair an entire drive of lost data on a single drive in an 8+1 drive system? Seems like magic to me. Obviously it's not, as I've run these parity+backup solutions effectively and had the magic of replacing a drive after a failure and have it just work.

 

It's probably complex and will go over my head, but I'd really like to know!

Basically your data is stored as an algebra equation. For example 1+2+3+4+5+6+7+8=36. If one of the drives fails such as drive 6, it can recalculate it, so if all of this is equal to this and one is missing what is that missing thing. 1+2+3+4+5+x+7+8=36

I will recommend an NHu12s (or an NHd15 (maybe)) for your PC build. Quote or @ me @Prodigy_Smit for me to see your replies.

PSU Teir List | Howdy! A Windows Hello Alternative 

 

 

Desktop :

i7 8700 | Quadro P4000 8GB |  64gb 2933Mhz cl18 | 500 GB Samsung 960 Pro | 1tb SSD Samsung 850 evo

Laptop :

ASUS G14 | R9 5900hs | RTX 3060 | 16GB 3200Mhz | 1 TB SSD

Link to post
Share on other sites
Posted
  • Author
Just now, Smit Devrukhkar said:

Basically your data is stored as an algebra equation. For example 1+2+3+4+5+6+7+8=36. If one of the drives fails such as drive 6, it can recalculate it, so if all of this is equal to this and one is missing what is that missing thing. 1+2+3+4+5+x+7+8=36

This makes sense, but I can't understand how something as complex and ever changing as user data can be calculated as a missing piece of an algebraic equation? Wouldn't doing the math on that during reads, writes, and rebuilds completely destroy the performance benefits?

Link to post
Share on other sites
Posted
Just now, swordstoo said:

Wouldn't doing the math on that during reads, writes, and rebuilds completely destroy the performance benefits?

That is why you need a baller CPU for servers, the faster it can calculate data the less chance for performance drawbacks you have. That is why people still need 16 or 20 core CPU's on a simple storage server. Also this is another reason rebuilds take so long.

I will recommend an NHu12s (or an NHd15 (maybe)) for your PC build. Quote or @ me @Prodigy_Smit for me to see your replies.

PSU Teir List | Howdy! A Windows Hello Alternative 

 

 

Desktop :

i7 8700 | Quadro P4000 8GB |  64gb 2933Mhz cl18 | 500 GB Samsung 960 Pro | 1tb SSD Samsung 850 evo

Laptop :

ASUS G14 | R9 5900hs | RTX 3060 | 16GB 3200Mhz | 1 TB SSD

Link to post
Share on other sites
Posted
2 minutes ago, swordstoo said:

This makes sense, but I can't understand how something as complex and ever changing as user data can be calculated as a missing piece of an algebraic equation? Wouldn't doing the math on that during reads, writes, and rebuilds completely destroy the performance benefits?

If you've ever paid attention to it, you'd notice CPU-usage being higher when writing to a software-RAID exactly because of the maths involved. Hardware-RAID does the maths on the hardware.

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to post
Share on other sites
Posted
  • Author
Just now, Smit Devrukhkar said:

That is why you need a baller CPU for servers, the faster it can calculate data the less chance for performance drawbacks you have. That is why people still need 16 or 20 core CPU's on a simple storage server. Also this is another reason rebuilds take so long.

I do know about the stupid rebuild times on replacement disks. Last time I replaced a dead drive in a 2+1 RAID config it it basically made the system unusable for half a day.. lmao.

 

Your explanation is great, although it seems a little bit of an over simplification? Is there a more technical explanation you could give?

Link to post
Share on other sites
Posted

https://en.wikipedia.org/wiki/Erasure_code

Main Rig:-

Ryzen 7 3800X | Asus ROG Strix X570-F Gaming | 16GB Team Group Dark Pro 3600Mhz | Corsair MP600 1TB PCIe Gen 4 | Sapphire 5700 XT Pulse | Corsair H115i Platinum | WD Black 1TB | WD Green 4TB | EVGA SuperNOVA G3 650W | Asus TUF GT501 | Samsung C27HG70 1440p 144hz HDR FreeSync 2 | Ubuntu 20.04.2 LTS |

 

Server:-

Intel NUC running Server 2019 + Synology DSM218+ with 2 x 4TB Toshiba NAS Ready HDDs (RAID0)

Link to post
Share on other sites
Posted
  • Author
1 minute ago, WereCatf said:

If you've ever paid attention to it, you'd notice CPU-usage being higher when writing to a software-RAID exactly because of the maths involved. Hardware-RAID does the maths on the hardware.

Having a hardware-RAID would be specifically designed to around doing the maths a RAID system would use, offloading work from the CPU and using specifically tuned hardware increases the performance, right?

Link to post
Share on other sites
Posted
5 minutes ago, swordstoo said:

Having a hardware-RAID would be specifically designed to around doing the maths a RAID system would use, offloading work from the CPU and using specifically tuned hardware increases the performance, right?

A hardware-RAID does all the operations itself, yes, letting the CPU spend its time doing something else. Of note, however, is that hardware-RAID is far less flexible and software-RAID with e.g. ZFS or Btrfs is far preferable nowadays, even with the CPU-hit.

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to post
Share on other sites
Posted
  • Author
1 minute ago, WereCatf said:

A hardware-RAID does all the operations itself, yes, letting the CPU spend its time doing something else. Of note, however, is that hardware-RAID is far less flexible and software-RAID with e.g. ZFS or Btrfs is far preferable nowadays, even with the CPU-hit.

Hardware raid can't be updated like software can, definitely a drawback. For a personal, long term use system, what do you recommend when performance and stability is preferred?

Link to post
Share on other sites
Posted
1 minute ago, swordstoo said:

Hardware raid can't be updated like software can, definitely a drawback. For a personal, long term use system, what do you recommend when performance and stability is preferred?

As I already said, software-RAID is preferable.

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Storage Devices

×