Please recommend a hard drive, for long term storage.

[keavlar]

Hey guys. 

My plan to store all the data, that I gathered for the last 10 years, and I would like to store it for another 20 years. 

I use a portable hard drive station to view that storage so that the hard drive not in use for a long time.

I read that the western digital black is the best choice. 

Is that correct, or you would suggest something else?

 

PS: I don't mind if it will be 1tb capacity each, I actually think it is better, as there are some files I am not using for a long period of time. 

[Jeremy Schmidt]

This is what I use. Just 1tb but it getd the job done. https://ca.pcpartpicker.com/product/MwW9TW/western-digital-internal-hard-drive-wd10ezex

[Princess Luna]

Well there's the Seagate BarraCuda Pro 2TB also, it's warranty is of 5 years and is rated for 1500TB of writing/rewriting on it if any thing on pair with the WD Black.

 

Now clod storage also is a solution if you absolutely can't afford losing stuff.

[Lord Nicoll]

42 minutes ago, keavlar said:

Hey guys. 

My plan to store all the data, that I gathered for the last 10 years, and I would like to store it for another 20 years. 

I use a portable hard drive station to view that storage so that the hard drive not in use for a long time.

I read that the western digital black is the best choice. 

Is that correct, or you would suggest something else?

 

PS: I don't mind if it will be 1tb capacity each, I actually think it is better, as there are some files I am not using for a long period of time. 

Would help to know what size the archive is, and what format you want. If you wanan get really fancy you can build a server to host a database for the archive, and pick good drives from the start, but I'm not sure if any modern HDD can survive 20 years, I've found a lot of dead PATA and SCSI that where meant to be working but had been stored a long time, in some cases in anti static bags and left for literally there for 20+ years, a lot where fine though. As for actual long term HDD's, the 1TB drive I'm using right now is almost 10 years old and has 56,194 power on hours, It's a WD Enterprise class SATA drive, seems to have been a HP model, GB1000EAMYC, most enterprise class drives are pretty reliable, but cost more. WD Reds are not really enterprise class but they certainly get used and are quite good too. 

[TrigrH]

if u wanna be super safe maybe get a helium drive as they are completely sealed?

[Lord Nicoll]

7 minutes ago, TrigrH said:

if u wanna be super safe maybe get a helium drive as they are completely sealed?

Helium drives have an almost negibigle lower failure rate, though that's purely within their life times, they haven't been around long enough to comment on extra life after they're espected failure date is past, that remains to be seen,.

[keavlar]

1 hour ago, Lord Nicoll said:

Would help to know what size the archive is, and what format you want. If you wanan get really fancy you can build a server to host a database for the archive, and pick good drives from the start, but I'm not sure if any modern HDD can survive 20 years, I've found a lot of dead PATA and SCSI that where meant to be working but had been stored a long time, in some cases in anti static bags and left for literally there for 20+ years, a lot where fine though. As for actual long term HDD's, the 1TB drive I'm using right now is almost 10 years old and has 56,194 power on hours, It's a WD Enterprise class SATA drive, seems to have been a HP model, GB1000EAMYC, most enterprise class drives are pretty reliable, but cost more. WD Reds are not really enterprise class but they certainly get used and are quite good too. 

Well, I store basically movies there. 

They are between 2-36 GB per file.

Building server kinda not an option for me now, but I did though about getting online storage or something like that, but yet again, I feel that it will cost much more. 

[keavlar]

2 hours ago, Princess Cadence said:

Well there's the Seagate BarraCuda Pro 2TB also, it's warranty is of 5 years and is rated for 1500TB of writing/rewriting on it if any thing on pair with the WD Black.

 

Now clod storage also is a solution if you absolutely can't afford losing stuff.

Well I prefer not to lose stuff, but well , we living in not perfect world. 

[TrigrH]

1 hour ago, Lord Nicoll said:

Helium drives have an almost negligible lower failure rate, though that's purely within their life times, they haven't been around long enough to comment on extra life after they're expected failure date is past, that remains to be seen,.

well you technically can get liquid in there

 

 

 

[Lady Fitzgerald]

Whatever you shoose, make sure you have at least one backup so you won't lose any data should a drive fo bad in storage. Basically, for data to be reasonably safe, it needs to exist in at least three separate places. If your data will exist only on the long term storage drive, you should also have an onsite backup drive and an offsite backup drive. The onsite backup drive should be stored elsewhere in the house away from the storage drive. 

 

Long term storage drives (and their backups) should be powered up and accessed at least every year to ensure they still work and the data is still there. It also exercises the drives to make sure they don't freeze up due to disuse. Also, it will remind you to make sure you still can access the data on the drive. Too often, people (and businesses) stick data on certain media and forget it it only to discover they can't access it umpteen years down the road because technology has moved on and the available hardware to access the data no longer exists. Floppy disks and IDE drives are examples of media that has become obsolete (and people have lost data because they no longer had the hardware to read the data). Current HDDs may be (or may not be: who knows?) obsolete 20 years from now so it may be neceessary to move your data to new media someday

[Xplo1t]

I can almost guarantee that putting your faith in a single drive for 20 years is not a safe bet.  You will need to actively manage it across multiple drives and at 1TB or less, cloud storage probably has a much higher retention possibility.  

[Lord Nicoll]

On 11/4/2018 at 7:30 AM, keavlar said:

Well, I store basically movies there. 

They are between 2-36 GB per file.

Building server kinda not an option for me now, but I did though about getting online storage or something like that, but yet again, I feel that it will cost much more. 

A server doesn't have to be fancy, an Ahlon or Pentium with a raid card or onboard raid on a nice board would be enough, decent PSU and a big case with a lot of hard drives is enough, the magic happens in the software and whatever you choose to go with. Obviously that also has issues and costs, but actual servers are not really an option usually. 

[Lady Fitzgerald]

19 hours ago, Xplo1t said:

I can almost guarantee that putting your faith in a single drive for 20 years is not a safe bet.  You will need to actively manage it across multiple drives and at 1TB or less, cloud storage probably has a much higher retention possibility.  

If you are referring to putting all the data on each of several drives, then I agree. If you are talking about breaking up the data in to partial bgroups distributed over several drives, then I disagree. Go back to what I wrote in my previous post.

6 hours ago, Lord Nicoll said:

A server doesn't have to be fancy, an Ahlon or Pentium with a raid card or onboard raid on a nice board would be enough, decent PSU and a big case with a lot of hard drives is enough, the magic happens in the software and whatever you choose to go with. Obviously that also has issues and costs, but actual servers are not really an option usually. 

For static, long term storage, a server is way overkill and multiple servers would be needed. Again, RAID is NOT a backup. And again, go back to my previous post.

[Lord Nicoll]

12 hours ago, Lady Fitzgerald said:

For static, long term storage, a server is way overkill and multiple servers would be needed. Again, RAID is NOT a backup. And again, go back to my previous post.

Sure RAID is not backup but it is redundancy (obviously, it's what R stands for) and redundancy is a form of back. Either way he'd need to replace the HDD's are they approach their expected failure time, and depends how fast he wanted to access said data, if purely storage, then data decay is more of a concern and avoiding said data decay could be expensive. 

[Lady Fitzgerald]

3 hours ago, Lord Nicoll said:

Sure RAID is not backup but it is redundancy (obviously, it's what R stands for) and redundancy is a form of back. Either way he'd need to replace the HDD's are they approach their expected failure time, and depends how fast he wanted to access said data, if purely storage, then data decay is more of a concern and avoiding said data decay could be expensive. 

You just contradicted yourself. Redundancy is NOT a backup. Period! All redundancy protects from is drive failure (up to the point of drive failure tolerance). It will NOT prevent data loss or corruption due to viruses and other malware, theft, fire, flood, voltage and current spikes from a failing PSU or a power line surge large enough to blow through any surge protection you may have, user error (such as accidental deletion or formatting), etc.

 

You can't predict when a drive or any other kind of storage media will fail. Some drive brands and models will have varying expected life spans but that is no gurantee they will last that long or not last longer. Even brand new drives can fail or even arrive DOA (I've had it happen). I had a drive last seven years under mostly 24/7 operation.

 

For the umpteenth time, the ONLY way for data to be reasonably safe is for it to exist in three separate places!

 

Bit rot (data decay) is nowhere even near the issue it once was back in the day. What's far more likely to happen with long term data storage is to have the media become obsolete.

 

[Lord Nicoll]

7 hours ago, Lady Fitzgerald said:

You just contradicted yourself. Redundancy is NOT a backup. Period! All redundancy protects from is drive failure (up to the point of drive failure tolerance). It will NOT prevent data loss or corruption due to viruses and other malware, theft, fire, flood, voltage and current spikes from a failing PSU or a power line surge large enough to blow through any surge protection you may have, user error (such as accidental deletion or formatting), etc.

 

You can't predict when a drive or any other kind of storage media will fail. Some drive brands and models will have varying expected life spans but that is no gurantee they will last that long or not last longer. Even brand new drives can fail or even arrive DOA (I've had it happen). I had a drive last seven years under mostly 24/7 operation.

 

For the umpteenth time, the ONLY way for data to be reasonably safe is for it to exist in three separate places!

 

Bit rot (data decay) is nowhere even near the issue it once was back in the day. What's far more likely to happen with long term data storage is to have the media become obsolete.

 

Not sure why an archive would be open to the outside web like that, and assuming a modest level of security I don't think a virus or malware is much of a threat, Acts of god really aren't something you can predict and thus can't really counter, like you said data centres get around it by having data sometimes in even more places than 3. For the most part cloud storage would be his best bet but, that could be expensive, as he wants to store movies, I'm not sure how many TB total he needs. I don't think I'd justify a massive investment in hardware, but I wasn't 100% sure what he wanted to store but was just informing him that a server solution doesn't have to be expensive afterwards as he seemed to think so. Either solution will cost money. 

 

Plus you can predict when a HDD will fail, there are whole graphs for expected failure rates and time scales. Doesn't mean the prediction is correct, but they have predicted failures. I'm usually found they underestimate drive failures once you get past early failures (nothing's more fun than a drive dying after a month). Thankfully data decay is not much of a problem now for HDD's, but it's fun having to write new DVD's and CD's every 5 to 10 years, even modern optical disks still seem to suffer from decay, but that could just be local climate and such. 

 

[Lady Fitzgerald]

6 minutes ago, Lord Nicoll said:

Not sure why an archive would be open to the outside web like that, and assuming a modest level of security I don't think a virus or malware is much of a threat...

If the computer is connected to the internet, the computer and anything attached to it are subject to infection. No matter how good most people's security may be, viruses and other malware can still sneak through.

 

10 minutes ago, Lord Nicoll said:

...For the most part cloud storage would be his best bet but, that could be expensive, as he wants to store movies, I'm not sure how many TB total he needs. I don't think I'd justify a massive investment in hardware, but I wasn't 100% sure what he wanted to store but was just informing him that a server solution doesn't have to be expensive afterwards as he seemed to think so. Either solution will cost money...

 Cloud storage is rarely reliable or secure, especially for the freebies, unless you use the really expensive commercial grade sites. Even then, it should never be your only repository for your data. Good paid cloud backups services, on the other hand, are safe but are expensive and require high speed internet, which is also costly. Initial uploads and full recovery are very time consuming even at gigabit speeds.

7 minutes ago, Lord Nicoll said:

...Plus you can predict when a HDD will fail, there are whole graphs for expected failure rates and time scales. Doesn't mean the prediction is correct, but they have predicted failures. I'm usually found they underestimate drive failures once you get past early failures (nothing's more fun than a drive dying after a month)...

 

Those failure rate graphs and time scales are NOT predictions. All drives, no matter the quality or be they HDDs or SSDs, can suddenly fail irrecoverably without any warning whatsoever. Granted, better quality drives will probably last longer before failure but it is still no guarantee failure won't happen.

 

31 minutes ago, Lord Nicoll said:

...Acts of god really aren't something you can predict and thus can't really counter...

 

Actually, you can counter them (unless it is on a global or near global scale). It's the reason why having data exist in three or more places is essential. In my case, for every data drive in my desktop computer, I have a set of four backup drives: two of each set are kept onsite in a drawer away from the computer and the other two of each set is kept offsite in my safe deposit box at my credit union five miles (as a drunken crow would fly) from my home. It would take an asteroid strike large enough to destroy both my home and my credit union's vault (and it's reinforced concrete anchored to granite bedrock) or the EMP from a nuclear strike to cause me to lose my data (the reasons for the duplication of local and offisite backup drives include any drive, including backup drives, can fail without warning and I don't have to worry about losing data should a backup drive get corrupted while updating the backup). Assuming I was lucky enough to be out of State when either happened, I would still have my critical data with me on one of my notebook computers.

 

41 minutes ago, Lord Nicoll said:

...like you said data centres get around it by having data sometimes in even more places than 3...

Actually, I didn't say that but it is often true. When data centers have data on servers located in different geographical locations, it's called georedundancy. Consumer level cloud backup services do not use georedundancy; you have to upgrade to far more expensive business plans to get georedundancy. It's far less expensive to also have your own local onsite and offsite backups in addition to a consumer grade cloud backup (recovery is also faster from a drive at hand).

[Lord Nicoll]

12 hours ago, Lady Fitzgerald said:

Actually, I didn't say that but it is often true.

Yes you did

 

21 hours ago, Lady Fitzgerald said:

For the umpteenth time, the ONLY way for data to be reasonably safe is for it to exist in three separate places!

Did I imagine that? 

 

12 hours ago, Lady Fitzgerald said:

Those failure rate graphs and time scales are NOT predictions. All drives, no matter the quality or be they HDDs or SSDs, can suddenly fail irrecoverably without any warning whatsoever. Granted, better quality drives will probably last longer before failure but it is still no guarantee failure won't happen.

 

Yes, they are predictions, they're literally called predicted failure rates. Whether the predictions are correct or not doesn't matter, they're still predictions, we can continue to argue semantics until the Sun swallows the earth and we'd still be at the same place as we are now. 

 

12 hours ago, Lady Fitzgerald said:

Actually, you can counter them (unless it is on a global or near global scale). It's the reason why having data exist in three or more places is essential. In my case, for every data drive in my desktop computer, I have a set of four backup drives: two of each set are kept onsite in a drawer away from the computer and the other two of each set is kept offsite in my safe deposit box at my credit union five miles (as a drunken crow would fly) from my home. It would take an asteroid strike large enough to destroy both my home and my credit union's vault (and it's reinforced concrete anchored to granite bedrock) or the EMP from a nuclear strike to cause me to lose my data (the reasons for the duplication of local and offisite backup drives include any drive, including backup drives, can fail without warning and I don't have to worry about losing data should a backup drive get corrupted while updating the backup). Assuming I was lucky enough to be out of State when either happened, I would still have my critical data with me on one of my notebook computers.

 

That's just a reiteration of this, and already established

21 hours ago, Lady Fitzgerald said:

For the umpteenth time, the ONLY way for data to be reasonably safe is for it to exist in three separate places!

 

12 hours ago, Lady Fitzgerald said:

If the computer is connected to the internet, the computer and anything attached to it are subject to infection. No matter how good most people's security may be, viruses and other malware can still sneak through.

 

Possible but using firewalls and non Window based OS's significantly reduces the changes to almost marginal. Once the appropriate safe guards are in place, like only allowing LAN connections and making sure all the PCs on the vLAN are also not susceptible, it massively reduces the risk. Of course when it comes to home applications, it's fun for the first week but then gets really annoying. Using all Linux seems fun at first but it gets annoying when you have to switch back and forth between windows. 

 

12 hours ago, Lady Fitzgerald said:

 Cloud storage is rarely reliable or secure, especially for the freebies, unless you use the really expensive commercial grade sites. Even then, it should never be your only repository for your data. Good paid cloud backups services, on the other hand, are safe but are expensive and require high speed internet, which is also costly. Initial uploads and full recovery are very time consuming even at gigabit speeds.

I agree with that whole heartily actually, I actively avoid cloud storage and use my own one to avoid as much as possible stuff like Google. Not sure if they actually do spy on people or not, I don't care really, I am just cognisant of keeping my files on someone else's system and what they get out of it. 


Finally, if someone so bad happens, like your house gets flood or burns down, tbh fuck the data that's store, you're house just burned, or got flooded, those are actually pretty shit things happen and the last thing on my mind would be "Oh fuck I lost my minecraft world" and "oh fuck I lost all my CAD files that I've been working on" because I just lost my fucking home they're useless anyways. Same goes for small businesses with only a handful of premises, it takes so long to get back on track often the company has either folded or can't fully use the back ups they have, because their main place is unusable. 

 

This is also somewhat derailing the original thread where OP just asked for HDD's recommendations, which was pretty much answered as something along the lines of WD red or Barracuda Pros or other enterprise drives. 

[Lady Fitzgerald]

3 hours ago, Lord Nicoll said:

Yes you did...

You are correct. Sorry.

 

3 hours ago, Lord Nicoll said:

...Yes, they are predictions, they're literally called predicted failure rates. Whether the predictions are correct or not doesn't matter, they're still predictions, we can continue to argue semantics until the Sun swallows the earth and we'd still be at the same place as we are now...

You are still missing the point, though. You can't rely on those "predictions" to keep your data safe. They do not indicate when a drive will actually die, only when most may die. it's the ones that fall outside the norm that will get your data unless you have proper backups.

 

3 hours ago, Lord Nicoll said:

...Possible but using firewalls and non Window based OS's significantly reduces the changes to almost marginal. Once the appropriate safe guards are in place, like only allowing LAN connections and making sure all the PCs on the vLAN are also not susceptible, it massively reduces the risk. Of course when it comes to home applications, it's fun for the first week but then gets really annoying. Using all Linux seems fun at first but it gets annoying when you have to switch back and forth between windows...

We weren't specifically discussing non-Windows machines. And even Macs get viruses.

 

3 hours ago, Lord Nicoll said:

...That's just a reiteration of this, and already established...

Apparently, it wasn't established since you had posted, "...Acts of god really aren't something you can predict and thus can't really counter..." The fact is, data loss due to "acts of god" most definitely can be countered by having multiple copies of your data knocking about.

 

4 hours ago, Lord Nicoll said:

...Finally, if someone so bad happens, like your house gets flood or burns down, tbh fuck the data that's store, you're house just burned, or got flooded, those are actually pretty shit things happen and the last thing on my mind would be "Oh fuck I lost my minecraft world" and "oh fuck I lost all my CAD files that I've been working on" because I just lost my fucking home they're useless anyways. Same goes for small businesses with only a handful of premises, it takes so long to get back on track often the company has either folded or can't fully use the back ups they have, because their main place is unusable...

Maybe the only data you store isn't worth much to you but that's not true for many, if not most, people. Many, if not most people keep their personal data, such as tax records, insurance policy information, medical records and information, purchase receipts, family photos, home inventory lists, books, music, movies, etc. on their computers. Many of those would be very costly, if not impossible, to replace. The data would be invaluable for filing insurance claims. Paper copies of home records alone would take up for more room than a bank safe deposit box could hold. Paper copies burn or can otherwise be destroyed. All this becomes even more critical for people who work from home.

 

Apparently, you haven't run a business before, particularly your own. Even if you have no intention of reviving a destroyed business, you will still need to file and pay taxes, pay of creditors, close contracts, collect any outstanding debts, etc. all of which would be pretty much impossible if you lost all the data (except for the taxes; Governments will collect whatever they feel is owed unless you can prove what you actually owe or do not owe).

 

If a destroyed business can't fully access their backed up data, the backups weren't properly done. Proper backups can be recovered from any computer. 

[kingknightrider]

If the drive is going to be connected only to back up the data then disconnected then almost any drive is acceptable.

 

That said some personal experience with manual Harddrives I have used and my opinion on how how SSD should be used in backup:

 

Standard Harddrives:

Western Digital Black seem to claim performance and longevity -so far they have been running good so far 3 years without issues -well except for one of 5. Definitely seen less failures than Western Digital blues (1 vs 5) but extremely more sensitive to motion -especially if dropped -which can lead into the direction of drive failure.

 

Seagate Barracuda have a tenacity of probably being a better storage device -if you have fears of the drive dropping -they can take it better than a WD Black. However from what I have seen from the 3 I have owned -of course not the current generation -they don't like long up times and constant use -referring to 24/7 use of years of operation. They seem to have better use to store data and be disconnected than constant use -still that is information that only applies to 1TB and lower. Have not used them in a while. 2 of the 3 total failure drives on which on partial data has been recovered have been 1 TB Barracuda drives.

 

Samsung Seagate Spinpoint In my opinion are the middle ground between Seagate and WDB they can take a drop -not as hard as a plain seagate nor as easy as a WDB. They can have strange form factors though -had one that was listed as 1TB and turned out to be a 1.5 TB drive. They also usually run at 5400 RPM or with only a 32MB cache not both usually. 

 

Solid State Drives -IN MY OPINION:

Are the best option for storage and back up but unless the whole system will be SSD not the most efficient use in a computer system.

Other will disagree and the time and speed difference for most is negligible.

Though if they are to be used as backup and that means connected to put data on and removed and stay disconnected till data from them is needed I would always advise having a duplicate SSD of the same back up SSD of the main HDD to be located where the system is going to be but not connected too. Always advisable to keep them in safe dry space too.

Though with as small as they are these days it would be good to keep them organized lest you end up with a mess:

[Lady Fitzgerald]

52 minutes ago, kingknightrider said:

If the drive is going to be connected only to back up the data then disconnected then almost any drive is acceptable... 

 

Solid State Drives -IN MY OPINION:

Are the best option for storage and back up but unless the whole system will be SSD not the most efficient use in a computer system.

Other will disagree and the time and speed difference for most is negligible.

Though if they are to be used as backup and that means connected to put data on and removed and stay disconnected till data from them is needed I would always advise having a duplicate SSD of the same back up SSD of the main HDD to be located where the system is going to be but not connected too. Always advisable to keep them in safe dry space too.

Though with as small as they are these days it would be good to keep them organized lest you end up with a mess:

I pretty much agree with what you have said in what I quoted above (I didn't comment on HDDs since I haven't used them in the past three years or so and haven't bought any in over five years so I'm a bit out of touch woth recent developments). I have two comments to add:

 

1. SSDs are still too expensive for most people to use for data storage and backups. Other than that, they are tough to beat (literally and figuratively). I completely abandoned HDDs several years ago and have been happy with being all SSD.

 

2. If their higher cost is not a problem and one decides to go all SSD for backups, be aware that the voltage state used to store data will slowly bleed down over time, causing data to be lost, thus making them impractical for static, long term storage (this can take as little as a few months to well over a year depending on the SSD brand and model, its age, and how full it is). That potential loss can be avoid by simply power them up and reading them at least a few times a year. I power up my backup SSDs every month and run FreeFileSync, the backup program I use for data drives, even though there has been no changes on the data drive in the computer. That ensures all data gets read. It will also detect if any files have become lost or corrupted (so far, that has yet to happen).