Raid 5 Config

[DailyProcrastinator]

So to provide some insight I'm working on a lot of changes between a few of my PC's. In my main system I currently have my OS and main program files on my Samsung 850 pro, the bulk of my storage is on two Seagate Barracuda 7200rpm 1TB hard drives in a raid 0 config (so effectively 2TB of storage), and then I use file copy software that I run usually every evening or while I'm working that copies everything to a WD Black 7200rpm 2TB drive so I know I have a safe backup. I also have just purchased two Crucial MX300 525GB SSD's (1.05TB of storage) that I am running in raid 0 for files that just sit there while I cannot physically connect my system to my server, this helps cut down time when copying over data or writing lots of data. And from there I move everything to my hard drives.

I normally don't need more storage because my server has six WD Red Pro 7200rpm 6TB drives in raid 10 (effectively 18TB of storage), which was easy to setup as I could do it through my motherboard bios. But now with me moving my computers around a lot, and with starting the build of my 4th computer, which I will hook up to the server too. I'm looking for a raid 5 config for my main system.

This way if I can't bring my server with me I'm not bringing my external hard drive reader and moving my data to a hard drive outside of my system (I have a library of hard drives). Now from my understanding running raid 5 is notoriously slow, however, all the info I have found on raid 5 configs seems to be like 3 years or older. Basically, I want to run raid 5 and have fast read and writes and if I need to move the hard drives to another system I don't want to have any issues. From what I can tell I will need a raid card, but as I said this info seems old.

I have yet to use a raid card and am wondering if anyone has some insight for me? I will most likely be using three Seagate Barracuda 7200rpm 3TB drives as they are the best $/GB here in Canada at the moment. If I need a raid card solution what brands should I look too? And will I have slower reads and writes then just a normal one hard drive storage config?

 

(Side note I can only use three hard drives in this system so any suggestions to run four hard drives won't help me)

 

Thanks for the help in advanced!

[Electronics Wizardy]

2 hours ago, JM21 said:

raid 5 is notoriously slow

No its not. Normally 300+ reads and writes with 4+ drives.

 

You don't normally need a raid card. Id probably use zfs on most unix systems and storage spaces on windows(but parity/raid5 is very slow for writes, you would need a cache drive for fast drives.

 

 

What OS? Id probably use software raid, normally faster, and protects your data better.

 

Some segate 3TB's (the st3000m001 model only) are very bad with very high failure rates. Make sure you don't buy them.

[dalekphalm]

28 minutes ago, Electronics Wizardy said:

No its not. Normally 300+ reads and writes with 4+ drives.

 

You don't normally need a raid card. Id probably use zfs on most unix systems and storage spaces on windows(but parity/raid5 is very slow for writes, you would need a cache drive for fast drives.

 

 

What OS? Id probably use software raid, normally faster, and protects your data better.

 

Some segate 3TB's (the st3000m001 model only) are very bad with very high failure rates. Make sure you don't buy them.

RE: The Seagate 3TB Drives - Are you talking about based on your personal experiences, the BlackBlaze reports, or something else entirely?

 

BlackBlaze reports should be discarded or look at with a grain of salt, since the reports are incredibly flawed, and lack the minute detail and context to really say why a drive failed.

 

Eg: I have 2x of them, been running for 3+ years with no problems at all. Anecdotal, of course. (I assume you meant the st3000dm001)

 

There was apparently a bad batch of one of the 3TB Seagate drives (I don't recall whether it was the DM001), but that was, as far as I'm aware, a single batch, and the odds of you even finding a drive from that specific batch are so infinitesimally small, that you'd likely have a better chance winning the lotto.

[Electronics Wizardy]

2 minutes ago, dalekphalm said:

RE: The Seagate 3TB Drives - Are you talking about based on your personal experiences, the BlackBlaze reports, or something else entirely?

Both.

The backblaze shows it as one of the real outliers with a large quantiy bought and very high failure rates

 

Also i bougt 5 personally, 4 have failed, the last one has errrors.

 

Your unlikey to find drivers from that batch today, but since there cheap drives, id check for it.

[dalekphalm]

3 minutes ago, Electronics Wizardy said:

Both.

The backblaze shows it as one of the real outliers with a large quantiy bought and very high failure rates

 

Also i bougt 5 personally, 4 have failed, the last one has errrors.

 

Your unlikey to find drivers from that batch today, but since there cheap drives, id check for it.

True they are an outlier, but as we all know, correlation does not equal causation.

 

Do they have higher fail rates because they fail more easily, or because they were put in higher workloads/worse scenarios/worse environment, etc?

 

Since we have no idea how the drives were used, if there were any differences between use, age of drives at time of death, etc, there's just far too much information missing from BlackBlaze to really draw a solid conclusion.

 

It's entirely possible you're correct, and Seagate 3TB drives really do fail more often. But if Seagate really had failure rates as bad as they report on BlackBlaze, they simply wouldn't be in business anymore. The cost of facilitating that many returns/warranty service would have put them out of business ages ago.

 

As for your 4 out of 5 drives? Damn that sucks. But most likely, all 5 drives were from the same batch, assuming you bought them at the same time. More than likely you just got unlucky with them.

 

Out of all the HDD's I've owned, for example, the only ones that ever failed were a 500GB Hitachi (before HGST WDC merger), and a 3TB WDC Green drive (Which actually died only recently).

[Electronics Wizardy]

29 minutes ago, dalekphalm said:

there's just far too much information missing from BlackBlaze to really draw a solid conclusion.

There is, but im guessing all of the drives were used in a simmilar workload, and having one model with a 5x faulure rate than the avera is very uncommon.

 

31 minutes ago, dalekphalm said:

Out of all the HDD's I've owned, for example, the only ones that ever failed were a 500GB Hitachi (before HGST WDC merger), and a 3TB WDC Green drive (Which actually died only recently).

How many drives do you have?

 

From my memory I have those 4x segate 3tb fail, a 800gb wd green, a pair of 1tb segates, a 500gb from a macmini, a old 160gb or two, but for my personal drives im normally use whatever old drives i can get for cheap, and then use something like btrfs or lizardfs  and lots of backups so i don't lose data with bad drives.

[dalekphalm]

21 minutes ago, Electronics Wizardy said:

There is, but im guessing all of the drives were used in a simmilar workload, and having one model with a 5x faulure rate than the avera is very uncommon.

 

How many drives do you have?

 

From my memory I have those 4x segate 3tb fail, a 800gb wd green, a pair of 1tb segates, a 500gb from a macmini, a old 160gb or two, but for my personal drives im normally use whatever old drives i can get for cheap, and then use something like btrfs or lizardfs  and lots of backups so i don't lose data with bad drives.

I currently have 6x 3TB drives in use (4x brand new toshiba drives, 2x Seagate). I have a dead 3TB WD Green. I also have a pair of 2TB Seagate drives, 2x SAS 146GB drives and 2x SAS 300GB drives. Plus a half dozen SSD's.

 

I used to have 3 or 4 500GB Hitachi drives, but after the first one died, I gave the rest of them to my brother.

 

And indeed, assuming that all drives are in a similar workload could be a safe assumption. All I'm saying is that it is an assumption, and could be true. Or not.

 

I'd personally love to see a properly done, comprehensive HDD failure study, with full stats, with all the little details, including all the raw stats if anyone ever wanted to actually dig through and verify it.

[Electronics Wizardy]

1 minute ago, dalekphalm said:

I'd personally love to see a properly done, comprehensive HDD failure study, with full stats, with all the little details, including all the raw stats if anyone ever wanted to actually dig through and verify it.

Me too.

 

Your best hope would probably be something like a company that uses 10000's of thousands of desktop workstations. But for them its not worth there time to make the data, and the systems are normally on warranty, and they probably don't care about failure rates too much if dell will just ship them a new one. 

 

Dell/HP/Lenovo Probably also have good data(or could make the data) on doa rates on drives, but they probably don't have as much long term as no every one uses dell/hp/lenovo to repair their system.

 

The manafacutures of the drives have good doa data and returen rates, so you have to comensate for the people who toss bad drives instead of getting them rma.

 

Datacenters also have good data, its just not desktop use. Backblaze is much more public than most about there drive usage and failure rate, but there method is buying cheap drives and diy enclosures, instead of the more enterprise grade san's with data center drives.

[DailyProcrastinator]

2 hours ago, Electronics Wizardy said:

No its not. Normally 300+ reads and writes with 4+ drives.

 

You don't normally need a raid card. Id probably use zfs on most unix systems and storage spaces on windows(but parity/raid5 is very slow for writes, you would need a cache drive for fast drives.

 

 

What OS? Id probably use software raid, normally faster, and protects your data better.

 

Some segate 3TB's (the st3000m001 model only) are very bad with very high failure rates. Make sure you don't buy them.

 

 

 

1 hour ago, dalekphalm said:

RE: The Seagate 3TB Drives - Are you talking about based on your personal experiences, the BlackBlaze reports, or something else entirely?

 

BlackBlaze reports should be discarded or look at with a grain of salt, since the reports are incredibly flawed, and lack the minute detail and context to really say why a drive failed.

 

Eg: I have 2x of them, been running for 3+ years with no problems at all. Anecdotal, of course. (I assume you meant the st3000dm001)

 

There was apparently a bad batch of one of the 3TB Seagate drives (I don't recall whether it was the DM001), but that was, as far as I'm aware, a single batch, and the odds of you even finding a drive from that specific batch are so infinitesimally small, that you'd likely have a better chance winning the lotto.

 

 

Yeah, I've had hard drive failures but it was due to a shitty server rack and its wiring shorting out and killing hard drives, and not the hard drives actually dying due to their own failures. I only have had one 2.5" laptop one nearly die but I replaced it with an SSD before that happened and that's my oldest and most used laptop I have ever owned. So I feel pretty confident with these 3TB Seagate drives, the model # is ST3000DM008 though. For my server I did spend some heavy cash even though I got a deal to buy all my WD Red Pro 6TB hard drives, I just wanted no messing around.

 

And speaking of raid 5 I am also thinking of setting up raid 5 with four Seagate Ironwolf 7200rpm 10TB hard drives, so i want to know its effective and reliable. Cause that would add another 30TB to my server wich will greatly help me this coming summer.

 

@Electronics Wizardy As far as OS I use Windows 10 home 64bit with a Unix-based shell primarily (so I can get the benefits of Unix while still running Windows 10). 

 

But I can't risk having slower read and writes so that's why I ask, also I can ONLY have a 3 hard drive setup unfortunately. But software raid would be the best bet?

[Electronics Wizardy]

32 minutes ago, JM21 said:

s far as OS I use Windows 10 home 64bit with a Unix-based shell primarily (so I can get the benefits of Unix while still running Windows 10)

But you don't get the greatness which is the linux drive system and things like zfs and linux networking.

 

 

What speeds do you need? Whats your network setup? Id try out storage spaces and see how it goes, it should work fine for your use.

[DailyProcrastinator]

8 minutes ago, Electronics Wizardy said:

But you don't get the greatness which is the linux drive system and things like zfs and linux networking.

 

 

What speeds do you need? Whats your network setup? Id try out storage spaces and see how it goes, it should work fine for your use.

 

I actually have a dual boot setup and have an Intel 540s SSD with Linux Mint 18.1 OS on it and my systems 4th hard drive is for storage when I'm running Linux.

 

As far as speeds as long as the raid 5 config isn't slower than using a single 7200rpm hard drive via a 6GB's sata port then I'm happy. I have my two MX300 SSD's for fast data transfers anyways, it's made a world of difference for me. My network for my server is 10GB/s.

[Electronics Wizardy]

23 minutes ago, JM21 said:

As far as speeds as long as the raid 5 config isn't slower than using a single 7200rpm hard drive via a 6GB's sata port then I'm happy. I have my two MX300 SSD's for fast data transfers anyways, it's made a world of difference for me. My network for my server is 10GB/s.

Well in windows try storage spaces and see if you life the performance. If you not happy with it, use zfs or md on linux, otherwise, id get a hardware raid card like a used dell h700 if you want one inexpensive, of a lsi 9260 8i if you want a new card.

[Erkel]

Get an LSI card, there are a lot of rebadge IBM / Lenovo ones that you can get cheap,  When it comes to RAID5, 3 disk home user arrays are about the only application, anything larger or business orientated you want to go with RAID6.

[DailyProcrastinator]

On 2/16/2017 at 3:06 AM, Erkel said:

Get an LSI card, there are a lot of rebadge IBM / Lenovo ones that you can get cheap,  When it comes to RAID5, 3 disk home user arrays are about the only application, anything larger or business orientated you want to go with RAID6.

 

Yeah with my server I will go for raid 6 as I will be using 4+ hard drives but in this particular system I can only use 3 hard drives so raid 5 is the better option. Thanks for the advise!

[DailyProcrastinator]

On 2/16/2017 at 0:26 AM, Electronics Wizardy said:

Well in windows try storage spaces and see if you life the performance. If you not happy with it, use zfs or md on linux, otherwise, id get a hardware raid card like a used dell h700 if you want one inexpensive, of a lsi 9260 8i if you want a new card.

 

Sounds good, thanks for the advice, I'll try that!