Backblaze: SSDs might be as unreliable as disk drives

[Lightwreather]

Summary

Cloud backup and data storage provider BackBlaze is finding its SSDs fail at nearly the same rate as its disk drives at the equivalent stage in their life cycle.

 

Quotes

Quote

A Backblaze blog by Andy Klein, its oddly titled Principal Cloud Story Teller, outlines how Backblaze used to boot its systems off disk drives and then started using SSDs instead. It monitors its disk drive reliability and does the same for its SSDs, so it can compare their failure rates. To make the comparison fair, it compared the SSD failure rates to the HDDs used for boot drive duty at a similar age in their life cycle and found similar failure rates.

Klein wrote: “Where does that leave us in choosing between buying a SSD or a HDD? Given what we know to date, using the failure rate as a factor in your decision is questionable. Once we controlled for age and drive days, the two drive types were similar and the difference was certainly not enough by itself to justify the extra cost of purchasing a SSD versus a HDD. At this point, you are better off deciding based on other factors: cost, speed required, electricity, form factor requirements, and so on.”

 

My thoughts

Well, this is pretty interesting. It might be possible that SSDs fail at a similar rate to HDDs? If you want to have a look at the testing methodology used by blackblaze, it's in a blog post linked in the sources below. Personally, this does not annul the prospect of buying an SSD anytime soon, that speed and boot times are amazing. But for a data center or backup company like backblaze, this might impact their decision. But we'll have to wait and see, I'm hoping for some testing from other sources in order to confirm or refute this claim.

Sources

Blocks and Files

Backblaze

[Shimejii]

Its really hard to say since each drive is different, Samsung drives or High end enterprise drives should do better then most lower end drives. The biggest factor for SSD for this kind of thing honestly is the No spinning disc and Arms, Even if the drive failure rates are about the same, although it looks to be slightly better, ill take SSD for most use cases. For data that under no circumstances can be lost, sure ill take the HDD since worst comes to worst, you just put the platers on a new drives.

[tridy]

I think that one of the arguments for the SSDs would still be the size of them and the power consumption in notebooks, for instance. Well acoustics should probably count. It could be that SSDs have "some" place for improvement as well.

[Taf the Ghost]

2 minutes ago, tridy said:

I think that one of the arguments for the SSDs would still be the size of them and the power consumption in notebooks, for instance. Well acoustics should probably count. It could be that SSDs have "some" place for improvement as well.

Or HDD technology is just so refined at this point that the difference in failure rate is actually just "noise" in the data. It's actually pretty impressive how well those are holding up.

[porina]

1% effective failure rate for drives averaging just over a year old under constant use doesn't sound too bad to me for either drives. My personal sample size is much smaller, but I've had two SSDs go "sudden death" on me within the 1st year of owning them from new.

 

Also I started a thread recently in that I think I'm seeing longer term bit rot on cheaper SSDs. Most recently it was a Kingston A400. One before that I think was a Sandisk of some description, but in both cases I bought it on price, not performance.

 

20 minutes ago, Shimejii said:

For data that under no circumstances can be lost, sure ill take the HDD since worst comes to worst, you just put the platers on a new drives.

If you need that level of data availability, then redundancy is the obvious solution (for example a mirror), or at least a good continuous backup strategy. You'd have to be unlucky to have both fail at the same time unless caused by external factors, which would be a different more serious problem.

[Sauron]

Not that surprising, you might see a difference in situations where the drive can be subjected to shock or poor operating conditons. Obviously a datacenter is a best case scenario for a HDD whereas an SSD will operate more or less equally regardless of where you install it.

[paddy-stone]

To date I have never had an SSD fail on me. I still have my 9-10 year old Samsung 830 128GB drive in rotation for use moving data when networking isn't an option. And the 64GB drive of the same age is now being used by my nephew for watching movies on his android box. They're not the fastest drives, but still operating at the same speeds as when I bought them. For the price I paid for those 64 and 128GB drives you can now buy a 1TB NVME drive

 

[mr moose]

Blackblaze is as defective as you can get.  I have yet to see them redeem themselves form their last attempt at pretending to be science.

 

A proper study of hdd and ssd longevity has to explain how it controlled for batch (random acquisition of specific drives) and end use variance (things like temperature, primary use conditions, etc).  There is no point in comparing 1600 low end ssd's to 1600 enterprise grade hdd's or even to 3400 shucked seagate specials. 

 

 

[LAwLz]

22 hours ago, mr moose said:

Blackblaze is as defective as you can get.  I have yet to see them redeem themselves form their last attempt at pretending to be science.

 

A proper study of hdd and ssd longevity has to explain how it controlled for batch (random acquisition of specific drives) and end use variance (things like temperature, primary use conditions, etc).  There is no point in comparing 1600 low end ssd's to 1600 enterprise grade hdd's or even to 3400 shucked seagate specials. 

Yep, exactly what I thought too.

 

For those who didn't read about it, Backblaze used to post "reliability reports" or something along those lines for drives. The problem was that their sample sizes were all over the place, and they were running drives way outside of their intended use.

For example they could buy a 100 dollar external hard drive, rip it out of the enclosure, put it into a rack with vibrations from a hundred other drives, and then subject it to a ton of read/writes and then they went "look at how poorly this Seagate drive did in our test".

 

Sometimes they had a sample size of like 20 drives and 1 of them died, making the failure rate 4%. Meanwhile, they would put that up against a drive where they had 1000 and 40 of them died. 

 

That's why their reports were always soooo out of line with other stats such as RMA rates. For someone like Newegg their drive RMA rates were like 1%. Meanwhile, stats from Backblaze showed failure rates like 10 times higher.

It's like if I started using smartphones as hammers and then reported on "reliability of modern smartphones" based on that. 

 

As long as Backblaze has such varied sample sizes and run their drives outside of their intended uses, I can't take their statistics seriously.

 

 

Edit:

Read the full report now.

For those wondering, yes, their sample size for SSDs are way smaller than the HDDs.

They have had 7 SSD failures and 551 HDD failures, and somehow they think that's enough data to extrapolate what the numbers will look like several years into the future. They only had a handful of SSD models, one of which they only had 49 drives of and one of them died so they put the AFR of that at model at 2.82%. 

 

And before anyone asks, yes, they still use consumer grade parts in their server environment. That SSD they only has 49 of? That's a Seagate BarraCuda SSD which is no longer sold, but I found an old listing for it that seems to indicate it was a $80 drive that was "perfect for laptops and desktop PCs".

 

For crying out loud they even list "DELLBOSS VD" as one of the "drives" they use. The problem is that "DELLBOSS VD" is not even an SSD. It's a PCIe to M.2 expansion card. So we have no idea which SSDs they even put in that.

 

 

Edit 2:

As @wanderingfool2 pointed out here, the full report" I read was not actually the "full report" they used as the basis for the article. 

For example the 551 HDD failures vs 7 SSD failures I quoted were measured over the course of 7 years. Meanwhile, the HDD vs SSD article only uses a subset of that data when it draws their conclusions.

I still think it's a really shitty article full of holes and irrelevant data, but it's not as out of whack as I first thought.

[Vishera]

It all depends on the make and model of the drive.

 

In general QLC DRAM-less SSDs last less than other types of SSDs

SAMSUNG drives are in general higher quality than most consumer drives out there,but some high performance models have a track record of dying within the warranty period.

 

As for hard drives in general all TOSHIBA and Western Digital drives are of good enough quality for long longevity,

As for SEAGATE it depends on the model - cheaper drives like their laptop and Barracuda drives are the lowest quality drives on the market.

 

When considering the TBW limits of SSDs,and the weak points of anything mechanical together with the quality of the drive - you may predict which drive is more likely to fail.

[InstantNewt]

15 minutes ago, LAwLz said:

Yep, exactly what I thought too.

 

For those who didn't read about it, Backblaze used to post "reliability reports" or something along those lines for drives. The problem was that their sample sizes were all over the place, and they were running drives way outside of their intended use.

For example they could buy a 100 dollar external hard drive, rip it out of the enclosure, put it into a rack with vibrations from a hundred other drives, and then subject it to a ton of read/writes and then they went "look at how poorly this Seagate drive did in our test".

 

Sometimes they had a sample size of like 20 drives and 1 of them died, making the failure rate 4%. Meanwhile, they would put that up against a drive where they had 1000 and 40 of them died. 

 

That's why their reports were always soooo out of line with other stats such as RMA rates. For someone like Newegg their drive RMA rates were like 1%. Meanwhile, stats from Backblaze showed failure rates like 10 times higher.

It's like if I started using smartphones as hammers and then reported on "reliability of modern smartphones" based on that. 

 

As long as Backblaze has such varied sample sizes and run their drives outside of their intended uses, I can't take their statistics seriously.

 

 

Edit:

Read the full report now.

For those wondering, yes, their sample size for SSDs are way smaller than the HDDs.

They have had 7 SSD failures and 551 HDD failures, and somehow they think that's enough data to extrapolate what the numbers will look like several years into the future.

They had 49 SSDs of a particular model and one of them has died, so they put the AFR of that at 2.82%. 

 

And before anyone asks, yes, they still use consumer grade parts in their server environment. That SSD they only has 49 of? That's a Seagate BarraCuda SSD which is no longer sold, but I found an old listing for it that seems to indicate it was a $80 drive that was "perfect for laptops and desktop PCs".

 

For crying out loud they even list "DELLBOSS VD" as one of the "drives" they use. The problem is that "DELLBOSS VD" is not even an SSD. It's a PCIe to M.2 expansion card. So we have no idea which SSDs they even put in that.

Not to mention, Google’s report found that SSDs, even within a data center, have a significantly lower replacement rate. 
 

https://www.usenix.org/conference/fast16/technical-sessions/presentation/schroeder

[Vishera]

4 minutes ago, InstantNewt said:

Not to mention, Google’s report found that SSDs, even within a data center, have a significantly lower replacement rate. 

The SSDs in data centers are significantly higher quality than any consumer SSD out there.

You can't apply data about data center SSDs to consumer SSDs.

[InstantNewt]

7 minutes ago, Vishera said:

The SSDs in data centers are significantly higher quality than any consumer SSD out there.

You can't apply data about data center SSDs to consumer SSDs.

Right, which means we agree BackBlaze’s data is useless. 

[mr moose]

9 minutes ago, Vishera said:

 

You can't apply data about data center SSDs to consumer SSDs.

It appears blackblaze think they can,  bunch of fucking morons.

 

 

[LAwLz]

1 hour ago, Vishera said:

The SSDs in data centers are significantly higher quality than any consumer SSD out there.

You can't apply data about data center SSDs to consumer SSDs.

Yes, but the problem is that BackBlaze is putting consumer drives in their data centers, and use them for data center workloads.

Of course they will break. They were not designed to handle that type of environment or workload.

 

Google's numbers are not representative of what consumers can expect because they use data center hardware.

But BackBlaze's numbers are not representative of what consumers can expect either because they use data center workloads and a data center environment.

 

 

Google's numbers shows what you can expect if you build your DC properly.

BackBlaze's numbers shows what you can expect if you build your DC improperly.

 

None of the numbers shows what consumers can expect.

[Doobeedoo]

Data all over the place. Anyway, I'd trust flash storage more than mechanical one, I mean I even had much better experience with SSDs than HDDs too. 

[Helpful Tech Witch]

So, in  acontroled, safe for drives envirnment, hdds fail less? That makes sense

THe thing with ssds ins that they can survive things like being dropped, or shaken

[Kisai]

1 hour ago, James Evens said:

That shit of Toshiba P300 still works a few years down the road. Still wouldn't call it a good drive as it causes very noticeable vibrations and sound emission.

One of the SSD's from a scavenged laptop that I threw in my desktop just because the desktop has M2 slots, runs hot, but otherwise nothing wrong with it.

 

For reference, the boot drive with the OS on it:

 

So by all accounts my C drive has 6 times the writes to it, and twice as many power on hours. However how do they compare in performance?

Versus the scavenged drive:

 

 

Much lower performance (I just use this drive for steam games, as I question the reliability.)

 

Keep in mind that compared to a SATA drive, even the scavenged NVMe is still better.

[Lord Szechenyi]

TL:DR

 

can someone tell me if the SSDs include M.2 SSDs?

[InstantNewt]

1 hour ago, LAwLz said:

None of the numbers shows what consumers can expect.

Agreed, I put Google’s findings up to show another source which came to the opposite findings. Failure rate on consumer drives is all over the place, so taking BackBlaze’s data and making generalizing statements like they are is useless. 
 

Anybody remember OCZ drives? How about Seagate 3TB drives? Failure rates will come down to the individual drives and even the batches they’re produced in. If you want accurate data you need to find the failure rates of specific drives and make your choice from that. None of this broadly speaking HDD vs SSD fluff. 

[HenrySalayne]

11 minutes ago, Lord Szechenyi said:

TL:DR

 

can someone tell me if the SSDs include M.2 SSDs?

No. They used a bunch of < $50 SSDs with 200 Gigs in their servers. There is no mention of failure modes or specific models.

So it's just an analysis of their own stock of SSDs and no general statement about SSDs.

 

[jagdtigger]

3 hours ago, LAwLz said:

look at how poorly this Seagate drive did in our test

To be fair seagate really is bad, you know it when 3 drives fail in their 4th year in a row.....

[LAwLz]

32 minutes ago, jagdtigger said:

To be fair seagate really is bad, you know it when 3 drives fail in their 4th year in a row.....

No, Seagate drives aren't bad. You have been unlucky if that's your experience. 

Seagate has roughly the same failure rate as all other consumer HDD makers like WD, Toshiba and Samsung. 

Seagate got a bad rep because they had one or two models that were bad like a decade ago. 

[jagdtigger]

12 minutes ago, LAwLz said:

No, Seagate drives aren't bad. You have been unlucky if that's your experience. 

Look when i have a wd green from 2011 that even survived running torrents and being in a raid vs 3 hdd from 2017 which died with something like 30k hours on it used in its intended use-case. Thats way more than just bad luck.

(I even have a 200 GB WD somewhere that still works with <10 bad sectors.... [and even those are old AF, the drive was one or two years old i think])

/EDIT

Oh and did i mention for not much more i could get wd dc-hc drives instead of crappy ironwolfs? Yeah seagate can go bust for all i care.....

[LAwLz]

59 minutes ago, jagdtigger said:

Look when i have a wd green from 2011 that even survived running torrents and being in a raid vs 3 hdd from 2017 which died with something like 30k hours on it used in its intended use-case. Thats way more than just bad luck.

(I even have a 200 GB WD somewhere that still works with <10 bad sectors.... [and even those are old AF, the drive was one or two years old i think])

/EDIT

Oh and did i mention for not much more i could get wd dc-hc drives instead of crappy ironwolfs? Yeah seagate can go bust for all i care.....

Seagate has lower RMA rates than Western digital. 

It was 0.93% vs 1.26% in 2017 (no more up to date data). 

 

Failure rate of the 4TB WD Red - 2.95%

Failure rate of 4TB IronWolf - 2.81%

 

 

Source: https://www.hardware.fr/articles/962-6/disques-durs.html

 

It's RMA rates from a very large French retailer. 

 

I don't doubt your experience, but the fact of the matter is that your experience is just a very tiny sample and as a result of bad luck, it is very skewed compared to the real world generalized numbers. 

 

 

Edit: 

For those interested, here are the RMA statistics for HDDs and SSDs according to the French retailer, which I think is way more representative of what consumers doing consumer things can expect. 

 

HDDs:

• HGST 0,82%
• Seagate 0,93%
• Toshiba 1,06%
• Western 1,26%

 

SSDs:

• Samsung 0,17%
• Intel 0,19%
• Crucial 0,31%
• Sandisk 0,31%
• Corsair 0,36%
• Kingston 0,44%