Backblaze releases some more flawed HDD failure statistics for Q3

[Oshino Shinobu]

As many of you know, a company called Backblaze releases their HDD failure rates from their data centres, often publishing them as HDD reliability statistics. Their previous graphs and statistics showed ridiculously high failure rates, particularly for Seagate drives (failure rates of as much as 40+%). This lead to a misconception of Seagate drives as being unreliable becoming wide spread across various forums including LTT. I also encountered people outside of forums who had got the trickle down of misinformation, claiming that Seagate was unreliable, despite not being able to explain why they thought that. 

 

The issue with Backblaze's results is the way in which they pick the drives they use and the environments they use the drives in. Basically, they tend to pick the cheapest drives and buy a load of them, shove them in the data centre and just replace them when they fail. While it's somewhat of a questionable business model, it can work out cheaper than buying data centre class drives, so they can pass on savings to their customers. The primary issue of their drive selections is when it comes to their failure rates. The vast majority of the drives in their data centres are desktop class drives, ones that are not designed for data centre conditions whatsoever. They have even had drives that were ripped from external enclosures because they were cheap. The fact that the drives are being used in an environment that is significantly different from their intended purpose makes the results meaningless right away, but it is not the only issue with the statistics. The general lack of information on the drives is a big issue when trying to compare failure rates. They ommit things such as the age of the drives, the proximity of the drives, the temperatures they were running at, the case/rack revision they were mounted in and so on. Without any of this, the results are even more meaningless. 

 

Ultimately, these results are like comparing which super cars died when driving them off-road. It's irrelevant to their intended use and if that's pretty much the only information provided, it's invalid anyway. 

 

 

 

As you can see, the tables they published simply group the drives by model and that's it. No grouping by age, writes/reads quantity, conditions and so on. According to this table, WD tops out the failure rates with 11.31% failure on their WD60EFRX drive. This is a 6TB WD Red drive designed for small home NAS systems, not large datacentre environments. Coming in second with 10.20% failure is the Seagate St4000DX000, which is a 4TB Barracuda drive, which is an internal desktop drive. In terms of raw numbers of failed drives, the Seagate ST4000DM000 was top with 278 drives failing. This is also a 4TB Barracuda drive, but they did use 34,744 of them and they had a total of 3,187,409 drive days run time, more than three times any of the other drives. 

 

Needless to say, these results are meaningless for the drives' intended purposes. If a company like WD would allow a drive to have a greater than 11% failure rate, they wouldn't be in business for long. 

 

At least Backblaze hasn't tried to make the data look as much of a reliability chart this time round. The fact that they still use the term "reliability stats" in their article is wrong, though. Here's their previous graphs, which caused a lot of misinformation, in case you haven't seen it: 

 

 

 

Sources: Backblaze  eTeknix

 

[Electronics Wizardy]

2 minutes ago, Oshino Shinobu said:

10.20% failure is the Seagate St4000DX000

The sample size is a bit off here to me. 5 drive failes and 192 total is a bit low, but is enough to tell you the batch isn't really bad.

[TheRandomness]

When will they learn you can't rely on a consumer drive in a datacenter...

[Prysin]

this report is ABSOLUTELY TRASH and has no relevant nor accurate information. The sample sizes are so disproportional that they arent possible to compare with eachother. Nor is there point in posting that second image. WHICH IS FROM YEARS AGO AND STILL NOT CORRECT TODAY!!

[Oshino Shinobu]

1 minute ago, Electronics Wizardy said:

The sample size is a bit off here to me. 5 drive failes and 192 total is a bit low, but is enough to tell you the batch isn't really bad.

The issue with the results is that they're basically meaningless. It's a small sample size, but a large failure rate in terms of percentage. They have drive models that have less than 45 in use. It just adds to the irrelevance of their results, given how inconsistent their drive use is. They have tens of thousands of some and only tens or less of others. 

[Electronics Wizardy]

Just now, Oshino Shinobu said:

The issue with the results is that they're basically meaningless. It's a small sample size, but a large failure rate in terms of percentage. They have drive models that have less than 45 in use. It just adds to the irrelevance of their results, given how inconsistent their drive use is. They have tens of thousands of some and only tens or less of others. 

Does any one have good drive reliabty data? I know the companies have the rma numbers, but they dont realse them.

[Oshino Shinobu]

1 minute ago, Prysin said:

this report is ABSOLUTELY TRASH and has no relevant nor accurate information. The sample sizes are so disproportional that they arent possible to compare with eachother. Nor is there point in posting that second image. WHICH IS FROM YEARS AGO AND STILL NOT CORRECT TODAY!!

Did you read what I posted? That's basically what my entire post says. 

 

"Here's their previous graphs, which caused a lot of misinformation, in case you haven't seen it:"

[Oshino Shinobu]

Just now, Electronics Wizardy said:

Does any one have good drive reliabty data? I know the companies have the rma numbers, but they dont realse them.

Not as far as I've seen. Getting actual representative and accurate data would cost a fortune.

 

There are drives that have bad batches, but in general most drives are decent as long as they're from a somewhat known company and are used as intended. If companies like Seagate and WD actually had 10%+ failure rates, they wouldn't be in business. 

[Bouzoo]

Yeah, seems to me like math and especially statistics are not really their stronger side.

[Prysin]

Just now, Bouzoo said:

Yeah, seems to me like math and especially statistics are not really their stronger side.

nor is their testing methodology

 

"hey let's test some HDDs"
"mkay, how"
"Oh, i dont know, 24/7 surveilance taping with heavy IO loads for 2 years?"
"oh great. So what disks we testing??"
"some cheap ones"

[alexyy]

ahh great another useless backblaze report, please ignore this it's meaningless and not even controlled. 

[Oshino Shinobu]

Just now, alexyy said:

ahh great another useless backblaze report, please ignore this it's meaningless and not even controlled. 

Did you read the post? Or even just the title? 

[Bouzoo]

4 minutes ago, Prysin said:

nor is their testing methodology

 

"hey let's test some HDDs"
"mkay, how"
"Oh, i dont know, 24/7 surveilance taping with heavy IO loads for 2 years?"
"oh great. So what disks we testing??"
"some cheap ones"

Seems to me like their logic was "If it can survive our test it can survive anything". Basically like taking an i3 and putting it under Prime95 for a week and saying "Yeah, our test is good".

[Coaxialgamer]

3 hours ago, huilun02 said:

I had a personal experience of the Seagate Barracuda pain train. 3 drives in regular desktop use, different batches. 

 

Sorry for not wanting to cheap out and take any more risks with my data. 

 

I learned the hard way that it doesn't cost much more to avoid all that agony. Now I just use WD Black/Reds and Deskstars. 

There's also psychological validation .  

How many other drives have failed ? ( noy saying you are lying ,  don't get me wrong ) 

 

People often fail to provide relevant data when talking about drive reliability and often cite personal (or "personal" ) experience about seagate while ignoring all the other times drives from different manufacturers have failed.  

 

Add the fact that people with bad experiences are much more likely to speak out then those who didn't ( and considering how the human brain works ) ,  means you have an echo chamber of people reporting false information while sometimes not even using the drives. 

 

This can be the case with any product .  The fact is that the failure rate of seagate drives isn't significantly high than any other brand ,  otherwise they wouldn't be in business . 

[Doobeedoo]

This again.

[Oshino Shinobu]

1 minute ago, huilun02 said:

It doesn't hurt to spend a little more for better quality and at least warranty. 

I'd agree with spending more on a better warranty, but there is no reliable data for WD drives being more reliable than Seagate or any other manufacturer for that matter. Personal experience can definitely sway people away from certain brands, but it doesn't then mean that that brand is bad based on such a small sample size. 

 

I've personally seen more WD Black drives fail than any other type of drive (I have 4 faulty WD Black drives in the glovebox of my car right now), but I still know that they're not unreliable drives. I personally have two 2TB Seagate Barracuda drives in my system, one of which has been running for nearly 4 years and the other for 2 with no issues. If we're going by very small sample sizes, WD Blacks must be unreliable based on what I've seen (they're not, but you get what I mean). 

[QueenDemetria]

I've had all brads of HDD(Seagate, WD, HGST, Hitachi, Toshiba, Maxtor) fail, and some of their products have lasted a stupid long time. I've had bad batches from Seagate and WD too, so shit happens to everyone.

[Curufinwe_wins]

Not really meaningless. just not representative of ordinary consumer use.

 

 

Still far better than no data at all, and it tells us a bit more about how delineated the individual hardware sets actually are.

[Curufinwe_wins]

43 minutes ago, Bouzoo said:

Seems to me like their logic was "If it can survive our test it can survive anything". Basically like taking an i3 and putting it under Prime95 for a week and saying "Yeah, our test is good".

I mean it's still better than nothing. and an i3 under p95 shouldn't ever have issues. if it did that would be a very big deal.

[Dabombinable]

Don't knock HDD from Segate's external HDD caddies. My 2TB (7200RPM) survived weeks of running at over 60^oC, and has proven quiet and reliable since I moved it from the POS caddy to my desktop (can't say the same about their HDD from the early 2000's-they were fragile AF).

[Oshino Shinobu]

Just now, Curufinwe_wins said:

Not really meaningless. just not representative of ordinary consumer use.

 

 

Still far better than no data at all, and it tells us a bit more about how delineated the individual hardware sets actually are.

Well, I guess it has meaning if you want to see how the drives cope when they're used in completely the wrong conditions. It's not really about consumer use, it's about intended and designed use. 

 

Using a desktop drive in a data centre immediately voids the warranty, so I'd really consider it meaningless, paired with all of the issues with their testing methodology (or lack of one)

 

1 minute ago, Curufinwe_wins said:

I mean it's still better than nothing. and an i3 under p95 shouldn't ever have issues. if it did that would be a very big deal.

I think it's about the same as nothing. It's just completely irrelevant and useless. Even if you're going to be putting the drives in data centres, these results aren't helpful in any way due to massively skewed sample sizes, no testing methodology or proper use case data. From this data, you cannot determine the cause of failure, the date of failure, the age of the drive, the run time of the drive or really anything else. 

 

 

[Jovidah]

1 hour ago, Electronics Wizardy said:

Does any one have good drive reliabty data? I know the companies have the rma numbers, but they dont realse them.

Well there's the periodical hardware.fr reports. It's essentially the RMA rates from an unknown 'high volume supplier', and is updated every half year to show the results of the last 6 months. So to get a good idea you have to read a few of them. The advantage though is that it immediately illustrates the variability (with WD & Seagte often changing places within category between different reports). 

There are some caveats there as well though. They are blanket RMA rates, taking no regard of different usage of the products. So you'll see some expensive OC boards getting way more RMAs than some budget boards, which likely has more to do with the different usage than the quality of the product (people burning them down with overcloacking). NAS drives with failure rates close to normal desktop drives might be making far more hours. Etc. 

In essense, the numbers aren't necessarily comparable as the products might not have been treated equally. So it's best only to compare 'within category'.

 

There's also no mention of sample size, although they usually do filter the ones with few sells (as they'd too easily create outliers). Some products aren't sold enough so don't show up in the statistics; this makes it really difficult to judge & compare these.

 

Some things to take away from it: 

-WD & Seagate are usually comparable, and trading places in comparable categories

-Toshiba is consistently behind both in reliability

-Hitachi tends to win out there as well, though usually by a far smaller margin (and not as consistently)

-Differences between products / product lines can be larger than products between brands (so a blanket "go for this brand" statement is pointless)

-Usually, Red / NAS drives are more reliable

-Reliability can vary quite a bit between different capacities and models.

 

If you want to get anything from it I recommend you go through at least the last 5 editions.

And pretty much for any product, not just harddrives, my most important recommendation would be to compare specific products, not brands.

 

http://www.hardware.fr/articles/947-1/taux-retour-composants-14.html

[Curufinwe_wins]

15 minutes ago, Oshino Shinobu said:

Well, I guess it has meaning if you want to see how the drives cope when they're used in completely the wrong conditions. It's not really about consumer use, it's about intended and designed use. 

 

Using a desktop drive in a data centre immediately voids the warranty, so I'd really consider it meaningless, paired with all of the issues with their testing methodology (or lack of one)

 

I think it's about the same as nothing. It's just completely irrelevant and useless. Even if you're going to be putting the drives in data centres, these results aren't helpful in any way due to massively skewed sample sizes, no testing methodology or proper use case data. From this data, you cannot determine the cause of failure, the date of failure, the age of the drive, the run time of the drive or really anything else. 

 

 

You say this as if people didn't take the shittiest cheapest drives they can find and throw them in NAS/Datacenters/Surveillance systems anyways...

 

Have you never been to the freenas forum?

[Jovidah]

13 minutes ago, Oshino Shinobu said:

Well, I guess it has meaning if you want to see how the drives cope when they're used in completely the wrong conditions. It's not really about consumer use, it's about intended and designed use. 

 

Using a desktop drive in a data centre immediately voids the warranty, so I'd really consider it meaningless, paired with all of the issues with their testing methodology (or lack of one)

 

I think it's about the same as nothing. It's just completely irrelevant and useless. Even if you're going to be putting the drives in data centres, these results aren't helpful in any way due to massively skewed sample sizes, no testing methodology or proper use case data. From this data, you cannot determine the cause of failure, the date of failure, the age of the drive, the run time of the drive or really anything else. 

 

 

You haven't even mentioned one of the biggest problems. The drives were mounted in all kinds of different storage solutions. So they weren't even given the same treatment. The major improvement the Seagate drives have suddenly seen is quite telling - you'd expect the larger disks to be less reliable, not more.

Not even going to bother with the rest of the flaws. It's garbage. If a student turned this in, they'd fail.

[Dabombinable]

4 minutes ago, Jovidah said:

Well there's the periodical hardware.fr reports. It's essentially the RMA rates from an unknown 'high volume supplier', and is updated every half year to show the results of the last 6 months. So to get a good idea you have to read a few of them. The advantage though is that it immediately illustrates the variability (with WD & Seagte often changing places within category between different reports). 

There are some caveats there as well though. They are blanket RMA rates, taking no regard of different usage of the products. So you'll see some expensive OC boards getting way more RMAs than some budget boards, which likely has more to do with the different usage than the quality of the product (people burning them down with overcloacking). NAS drives with failure rates close to normal desktop drives might be making far more hours. Etc. 

There's also no mention of sample size, although they usually do filter the ones with few sells (as they'd too easily create outliers). Some products aren't sold enough so don't show up in the statistics; this makes it really difficult to judge & compare these.

 

Some things to take away from it: 

-WD & Seagate are usually comparable, and trading places in comparable categories

-Toshiba is consistently behind both in reliability

-Hitachi tends to win out there as well, though usually by a far smaller margin (and not as consistently)

-Differences between products / product lines can be larger than products between brands (so a blanket "go for this brand" statement is pointless)

-Usually, Red / NAS drives are more reliable

-Reliability can vary quite a bit between different capacities and models.

 

If you want to get anything from it I recommend you go through at least the last 5 editions.

And pretty much for any product, not just harddrives, my most important recommendation would be to compare specific products, not brands.

 

http://www.hardware.fr/articles/947-1/taux-retour-composants-14.html

It is really sad that Samsung no longer makes HDD, or at least in the way that they used to. The reliability of their IDE (that weren't from the early 90's) is brilliant compared to all other manufacturers, and Samsung kept their 3 year warranty when all other manufacturers at the same time