Storage for Music mainly...

[SmirGel]

Hi

 

I was thinking about buying WD Green 4TB mainly for music..I've pretty large collection of music FLAC, Mp3 everything between don't really care about the file format...anyway I heard that 3-4TB drivers has failure rate much higher than 1TB. So that made me feel like is it safe to buy them even..I mean will it last even 2-3 years? or will it be from 1 year and boom dead? 

 

Tho if you've suggestions for other HDD that's 3-4TB you can tell that too..but I was thinking WD Green would fit nicely for music and pictures maybe some movies sometimes but mainly music..

[Rorgash]

depends a lot really, i got 1 old external HDD that is still alive after 10 year while i have 1 that i used for 3-4 years and it died

[SmirGel]

depends a lot really, i got 1 old external HDD that is still alive after 10 year while i have 1 that i used for 3-4 years and it died

 

Of course it will be in daily use since I use my pc anyway?

[Rorgash]

if you worry about losing it do a Raid 1 system with 2 Hdd's  but in general unless it breaks from you moving it or such you get warning signs before it breaks, and using something like Hard Disk sentinel will make sure you know even before that if its getting bad.

[SmirGel]

I already got two HDDs I don't even know what RAID they are atm lol -.-

 

some really OLD HDD, not so old and WD black 1tb

[alpenwasser]

I heard that 3-4TB drivers has failure rate much higher than 1TB. So that made me feel like is it safe to buy them even..I mean will it last even 2-3 years? or will it be from 1 year and boom dead?

I have not been able to find any reliable info about larger HDDs having higher

failure rates. I think you might be confusing two things here (not sure, so

correct me if my assumption is incorrect).

A problem you do run into with larger HDDs is that the chance of a read and/or

write error occurring on the drive is starting to become a relevant factor.

I'm a bit pressed for time at the moment, so I don't have time to look up

specific numbers and will just make a hypothetical example: Let's say your

drive is spec'd to make a read error every 10^12 bytes (on average). If your

drive has 4 TB of storage (so, 4*10^12 bytes), the chances of encountering

a read error when you go over the entire drive are rather significant. This

can be a serious problem when your disks are running in a RAID array, for

instance.

Say you have two disks in a RAID1 array, one of them fails. In order to

rebuild the array, you'll need to copy the data from the working drive

to a new one. But, if the chances of a read error occurring on the intact

drive are rather high, you are likely to encounter a serious problem with

data integrity (since now you have different data on the two drives in

some places).

This is a very rough generalization, leaving out quite a few things, and

as said the numbers are purely hypothetical, but it should give you a general

idea of the phenomenon.

Aside from that, there is the fact that larger drives tend to have more

platters, which might lower live expectancy somewhat (more moving parts,

more sources of failure), but as said I have not been able to find anything

conclusive and reliable on that and would be surprised if the impact was

actually significant.

@Vitalius might be able to write a more in-depth essay on the topic if he

has the time and inclination.

[Vitalius]

-snip-

And I do feel so inclined.

 

10^14 bits is the average URE rate most HDD's had in 2010-2011. That's 12TB. Reading an entire 4TB drive once has a 33.3333...% chance (4/12) of an unrecoverable read error (with that average) which means the HDD literally can't get the data off the platter correctly and knows it.

 

The actual problem with Unrecoverable Read Error (URE) rates is what they do to parity RAID. An average RAID controller considers a drive failed if it doesn't respond to a read request within 8 seconds. TLER or Time Limited Error Recovery is on Enterprise and NAS drives which limit a read attempt to <8 seconds so this doesn't happen. Either it comes back to check that data again later after it's done with everything else, or that data is just considered corrupt.

 

The problem with a RAID array considering a drive failed is that the RAID is either compromised, or failed, which means you either have a greater chance to lose all your data, or you lose all your data.

 

The even bigger problem is that when a single drive fails (normal failure, not URE induced pseudo-failure) in parity RAID, you have to rebuild the array with a new drive. Rebuilding the array means reading all the parity data. Reading all the parity data means basically asking for a URE (as tons of reading is happening) which means another drive will fail, and then you lose all your data in RAID 5, or get much closer to it in RAID 6/7 (and it's very likely depending on how many drives you have and how big/old they are).

 

URE's are likely the cause in a correlation between bigger drive size and drives failing in Parity RAID.

 

RAID 1 isn't as affected by this as it's rare for a RAID 1 to get as big as a RAID 5 in terms of drive size (i.e. in RAID 5, you can have multiple large TB drives, while in RAID 1, you tend to only have 1-3 large drives at most).

 

And even then, a URE wouldn't actually replicate the data as the HDD would know it failed to read it. The file would just be corrupt on both drives (due to it being unreadable or nonexistent, depending on which drive you look at). Aside from bit rot, which is different from URE's, which can actually do what Alpen mentioned about the data being different between the two.

 

The simple way to avoid losing an entire parity RAID array is to get Enterprise or NAS drives which have TLER which basically just makes the file corrupt rather than causing you to lose the entire array when the drives have UREs.

 

However, those are a bit more expensive than consumer drives. One other thing to consider about NAS/Enterprise drives is that they will perform better over time as they will give up on read errors within a reasonable amount of time, whereas a consumer drive will take much longer. IIRC, the time limit on URE recovery for consumer drives is anywhere from 30 seconds to 1 minute, which is your drive literally at a reading stand still as it is just reading the same sector over and over.

 

It's effectively your CD Skipping.

[alpenwasser]

And I do feel so inclined.

Most gracious, your eminence. Your peers thank you for your efforts.

EDIT:

I see no manual line breaks in that post...

[wpirobotbuilder]

-snip-

I can't find this anywhere on the internet, whether ZFS protects against UREs. Maybe you can confirm this?

[alpenwasser]

I can't find this anywhere on the internet, whether ZFS protects against UREs. Maybe you can confirm this?

Interesting question. Maybe @Eric1024 could shed some light on it?

[wpirobotbuilder]

Interesting question. Maybe @Eric1024 could shed some light on it?

Looks like it works to combat UREs, presumably because it prevents them from being created in the first place. But I'd definitely feel more comfortable with a definitive "yes" or "no".

 

http://www.theregister.co.uk/2013/03/13/2013_storage_primer/

 

I like their quote:

 

never, ever, under any circumstances, lie to ZFS.

[alpenwasser]

Looks like it works to combat UREs, presumably because it prevents them from being created in the first place. But I'd definitely feel more comfortable with a definitive "yes" or "no".

http://www.theregister.co.uk/2013/03/13/2013_storage_primer/

Yeah, that would be nice.

I like their quote:

never, ever, under any circumstances, lie to ZFS.

Yes, ZFS is like your doctor in that regard.

[Vitalius]

Most gracious, your eminence. Your peers thank you for your efforts.

EDIT:

I see no manual line breaks in that post...

 

You're welcome. And yeah. Didn't think of it. I don't see how you guys do that.

 

I can't find this anywhere on the internet, whether ZFS protects against UREs. Maybe you can confirm this?

 

The thing is is that no matter what, ZFS can only access drives through a SATA or RAID controller. And the controller is the thing that throws a fit if the drive doesn't respond promptly (i.e. drops the drive which to ZFS, or any other file system, looks like a failure). 

ZFS uses full checksums to measure data integrity, but from what I've read, there's no specific way it has to handle URE's aside from what you said. Preventing them in the first place. 

Here's an interesting webpage I found about failure rates calculated for certain RAID configurations which has a link to a site with calculators for this stuff that I think is about right. Specifically www.raid-failure.com which is kinda awesome. And here's a reliability study of ZFS from the University of Wisconsin. It doesn't actually address your question, but I thought it was interesting regardless. Article about ZFS and Data Protection on Oracle's blog section.

I know none of that answers your question, but from what I've read up there and on the FreeNAS forums, the URE thing is an issue between the controller and the HDD which ZFS can only do so much to prevent (i.e. intelligently write and backup data). But there's no real way of fixing it without changing the SATA controller, HDD, or how a RAID failure is handled, and ZFS only has absolute control over that last part, but options are limited there (i.e. rebuild now, or rebuild later). 

[alpenwasser]

You're welcome. And yeah. Didn't think of it. I don't see how you guys do that.

We do it by hand!  

 

- snip -

Hm, interesting.

In all the hoopla I think we forgot to answer one of OP's questions: @SmirGel

Yes, the 4 TB Green should work OK for that purpose, most HDDs would do the

job just fine these days.