data rot Cold Storage Data Lifetime? aka Unpowered Lifetime Until Data Rot?

[CoolJosh3k]

I’m curious about how long different types of media can retain their data until data rot occurs, if power is not supplied for a time.

 

My understanding is that data on flash storage, without occasional power being supplied, will only last 5 to 10 years. How true is this?

What about Hard Disks, CDs/DVDs and Tape?

 

Thinking I might email various manufacturers and ask if they can tell me the expected lifespan for their products under cold storage conditions.

[SorryBella]

4 minutes ago, CoolJosh3k said:

How true is this?

Pretty much true except the fact that it needs constant power. Whats needed is flashes of power, so basically as long as you power it on every 3-5 years, you can still retain data theoretically. But of course, thats very impractical for a proper cold storage thats why people use a less volatile storage medium like tapes.

[mariushm]

In a controlled environment (proper humidity and temperature) tapes have a significant life, provided you periodically turn them. Even then, you would have to read them once in a while to prevent layers of tape from sticking together and prevent the material holding data from sticking to the back side of the top layer.

 

I'd say hard drives have a chance of lasting at least 10-15 years, but again in controlled environment and provided they're periodically turned on to have the motor work for a bit and have the data read so that any weak magnetic area can be "refreshed".  With drives, you risk heads crashing, controller chips blowing up if too much humidity was absorbed in the chip package before the drive is started and heats up, the motor can get stuck if not spun for very long periods of time...

 

The bits stored inside Flash memory cells as energy charges in super tiny capacitors can last for more than 5 years, the leakage is very very low, and at ambient temperatures it's not a big problem.  Ideally, you still want to turn them on once in a while for the controller to analyze the flash chips and maybe shuffle some data around (where it detects difficulty reading bits reliably)

 

You can find papers about Flash memory reliability, endurance etc ... here's a random one I found that's freely available: https://www.slac.stanford.edu/exp/npa/misc/00220908.pdf

 

 

[CoolJosh3k]

Interesting about tape storage.

 

I guess if you dedicate yourself to powering a flash storage device for a few minutes every year, then it should be the best choice, right? It can be much faster and not an issue if you ever need to transport them.

 

I’ve always used Hard Drives due to the cost, with confidence on cold storage.

 

For me, I also learned somewhere that a HDD could be recovered after serious accidents because the magnetic presence still existing. SSD I understand does not have this useful property.

 

I might end up moving to using external solid state due to speed, leaving HDDs to more extreme cold storage.

[Kilrah]

In the end the only solution is having multiple copies since any of the storage types can always have an unexpected failure regardless of how long it can last. 

I've never seen HDD bit rot and I play a lot with old machines from the 90s, if an HDD doesn't work it's from other causes (stuck bearings, degrading plastic/rubber etc).

 

My solution is to not "archive" anything I care about, I have a large array in my main system that holds both old and current, and that I have 2 backups of.

[Mark Kaine]

On 11/22/2021 at 9:37 AM, CoolJosh3k said:

Thinking I might email various manufacturers and ask if they can tell me the expected lifespan for their products under cold storage conditions.

Hehe, that would be interesting,  they'll probably won't tell you though. 

 

[CoolJosh3k]

1 hour ago, Kilrah said:

In the end the only solution is having multiple copies since any of the storage types can always have an unexpected failure regardless of how long it can last. 

I've never seen HDD bit rot and I play a lot with old machines from the 90s, if an HDD doesn't work it's from other causes (stuck bearings, degrading plastic/rubber etc).

 

My solution is to not "archive" anything I care about, I have a large array in my main system that holds both old and current, and that I have 2 backups of.

A great way to do things is to keep a 2nd copy of backups on a cloud service, but this means that if needed to access that 2nd copy it can cost a good amount of money (and time).

 

I think for people keeping only 1 local copy, while having its own obvious issues, if it is a HDD that fails, the data will still be there. My fear is that for those people, if they choose a solid state media type then a failure could mean loss of data too.

 

How likely is it that a failure with an SSD would also make data unrecoverable?

My thought is that in a best case, with solid state, you just need the same chipset from a donor drive.

A HDD in best case, you’d have to replace the read assembly (a difficult task on its own), but could load publicly known configuration into a custom controller.

Am I right in this?

[Kilrah]

16 minutes ago, CoolJosh3k said:

My thought is that in a best case, with solid state, you just need the same chipset from a donor drive.

There are loads of possible failure modes, but usually it's possible to read data from flash chips and recreate the FTL (where's what, how it's encoded etc) although it's a pretty complex job requiring expensive tools and someone skilled to operate them, like for an HDD. Can be made harder or impossible if the contents are encrypted with a key unique to to the particular controller, etc.

 

But if the data has disappeared from flash due to charge loss there wouldn't be anything to do. Just like if an HDD platter is scratched that's gone.

 

This guy has some neat videos about recovery from different media types where you can see the process:

https://www.youtube.com/c/hddrecoveryservices/videos

 

It's often in no way trivial.

 

In the end the right thing to do is act as if recovery didn't exist and have multiple copies. Even buying $1000+ of extra HDDs to store duplicates will be cheaper than having to hire recovery services, not counting the fact that them being able to recover the data is never a given. 

[CoolJosh3k]

26 minutes ago, Kilrah said:

There are loads of possible failure modes, but usually it's possible to read data from flash chips and recreate the FTL (where's what, how it's encoded etc) although it's a pretty complex job requiring expensive tools and someone skilled to operate them, like for an HDD. Can be made harder or impossible if the contents are encrypted with a key unique to to the particular controller, etc.

 

But if the data has disappeared from flash due to charge loss there wouldn't be anything to do. Just like if an HDD platter is scratched that's gone.

 

This guy has some neat videos about recovery from different media types where you can see the process:

https://www.youtube.com/c/hddrecoveryservices/videos

 

It's often in no way trivial.

 

In the end the right thing to do is act as if recovery didn't exist and have multiple copies. Even buying $1000+ of extra HDDs to store duplicates will be cheaper than having to hire recovery services, not counting the fact that them being able to recover the data is never a given. 

Thanks for the link. I’ll check it out.