Long term storage solutions?

[pcguy0000]

Hey!

So, I am looking for a long term storage solution, mainly for photos but also for videos, etc. The storage needs to be:
- Accessible at least within my home network (nas)
- 1TB or larger
- Long term

- Easy to setup

I would also prefer it to:
- Have a way to automatically back it up to an offsite location, or an external hard drive (I just want the data to be safe)

 

Here are the things I have already tried:
- Getting a 2TB WD Hard drive, and hooking it up to my router as a nas (swisscom)

- Using the same hard drive whenever I need my photos

- Using swisscom mycloud (included with my subscription, will move in the future though.)

 

What should I use?

(wd cloud sounds good)

Thanks.

[Collinator24]

If you want to, you can use a refurbished server PC, and run a file server off that. Just set a static network address if you did that. 

[Radium_Angel]

4 minutes ago, pcguy0000 said:

Hey!

So, I am looking for a long term storage solution, mainly for photos but also for videos, etc. The storage needs to be:
- Accessible at least within my home network (nas)
- 1TB or larger
- Long term

- Easy to setup

I would also prefer it to:
- Have a way to automatically back it up to an offsite location, or an external hard drive (I just want the data to be safe)

 

Here are the things I have already tried:
- Getting a 2TB WD Hard drive, and hooking it up to my router as a nas (swisscom)

- Using the same hard drive whenever I need my photos

- Using swisscom mycloud (included with my subscription, will move in the future though.)

 

What should I use?

(wd cloud sounds good)

Thanks.

How long term are we talking about?

20 years?

40?

100?

 

This will "adjust" the price tag accordingly, so...time costs, how long do you want to spend?

[pcguy0000]

10 years @Radium_Angel

 

 

[LtStaffel]

7 minutes ago, pcguy0000 said:

10 years @Radium_Angel

 

 

@Radium_Angel

[mariushm]

Look into Amazon Glacier ... 0.004 $ per GB so around 4$ a month for storing 1 TB of data. 10 GB a month free to retrieve , 0.0025$ a GB for bulk retrieve, 0.01$ per GB for standard retrieve (getting just some files)

 

Link: https://aws.amazon.com/glacier/pricing/

 

Basically they put stuff into data tapes if you don't work with the data for a while, so if you need something back you make a standard request and some robot picks the tape from storage and connects it back and makes data available to you within something like 1-2 hours.

 

I would never rely on a single hard drive to backup stuff. At very least, have copies in two separate location and check weekly or monthly the condition of both systems. I'd run both systems 24/7 also and replace the drives after around 3 years.

 

[Radium_Angel]

1 minute ago, mariushm said:

Look into Amazon Glacier ... 0.004 $ per GB so around 4$ a month for storing 1 TB of data. 10 GB a month free to retrieve , 0.0025$ a GB for bulk retrieve, 0.01$ per GB for standard retrieve (getting just some files)

 

Link: https://aws.amazon.com/glacier/pricing/

 

I would never rely on a single hard drive to backup stuff. At very least, have copies in two separate location and check weekly or monthly the condition of both systems. I'd run both 24/7 also.

 

A lot can happen in 10 years, and I wouldn't trust any "cloud" storage as a sole point of backup.

 

11 minutes ago, pcguy0000 said:

10 years @Radium_Angel

 

 

Ok, 10 years. WORM drive, Blu Ray, DVD-DL, LTO4 tape.

That's easy.

[Razor Blade]

Before I had a NAS I had a simple raid 1 storage array on my PC I stored most of my archive data on and an external hard drive I used to back it up. I've had data from more than 15 years back still on that drive. Moving that data off of my main PC adds the benefit of being able to access it from any computer on my network and a small bit of protection from something like ransomware as I do not map the drive and have to login when I view my archive (I say small protection because if I got hit while logged into the share, I'd be screwed). But nothing beats just backing up the data every so often...things go wrong.

 

I have family members that have had their PC for more than 10 years and never backup their data...but they still have it. Though there is nothing that can be done if one day their hard drive decides it's had enough. As long as you backup your data on a regular basis and keep up with maintenance, just about anything is going to last you 10 years.

[mariushm]

19 minutes ago, Radium_Angel said:

A lot can happen in 10 years, and I wouldn't trust any "cloud" storage as a sole point of backup.

 

Ok, 10 years. WORM drive, Blu Ray, DVD-DL, LTO4 tape.

That's easy.

Even the CD-R discs I had died after around 4-5 years - humidity and stuff attacks the edge and corrodes the aluminum layer and eventually the discs break down.

DVD-R and +R were imho even more sensitive, some being unreadable or with lots of errors after 2-3 years. 

Bluray discs I don't know... considering the data density i wouldn't trust them for more than 6 months.

 

There are archival grade DVD discs which use different chemicals and reflective materials (gold and other stuff) and those probably last longer, but again I wouldn't trust them.

 

I have mechanical drives that are older than 6 years in my pc right now, and I keep my pc running 24/7 ... so considering a 4 TB drive is what... 100-150$ ... if you think about it's really cheap..  For example HGST Deskstar NAS 4 TB is 130$ on Newegg ... 130$  / ( 3 years x 12 months ) = 3.6$ a month... add maybe 2$ a month in electricity costs to keep a pc idle 24/7 in your house and you get yourself the equivalent of Amazon Glacier in your house.

 

These drives will last more than 3 years, and especially if you have two systems in two different locations (maybe one at your parents or sister's place) it would be perfectly fine to keep them in service for longer time.

Just don't buy the cheapest, get NAS grade or server or basically any hdd with at least 3-5 year warranty.

[Razor Blade]

4 minutes ago, mariushm said:

Spoiler

 

Even the CD-R discs I had died after around 4-5 years - humidity and stuff attacks the edge and corrodes the aluminum layer and eventually the discs break down.

DVD-R and +R were imho even more sensitive, some being unreadable or with lots of errors after 2-3 years. 

Bluray discs I don't know... considering the data density i wouldn't trust them for more than 6 months.

 

There are archival grade DVD discs which use different chemicals and reflective materials (gold and other stuff) and those probably last longer, but again I wouldn't trust them.

 

I have mechanical drives that are older than 6 years in my pc right now, and I keep my pc running 24/7 ... so considering a 4 TB drive is what... 100-150$ ... if you think about it's really cheap..  For example HGST Deskstar NAS 4 TB is 130$ on Newegg ... 130$  / ( 3 years x 12 months ) = 3.6$ a month... add maybe 2$ a month in electricity costs to keep a pc idle 24/7 in your house and you

 

get yourself the equivalent of Amazon Glacier in your house.

Spoiler

 

These drives will last more than 3 years, and especially if you have two systems in two different locations (maybe one at your parents or sister's place) it would be perfectly fine to keep them in service for longer time.

Just don't buy the cheapest, get NAS grade or server or basically any hdd with at least 3-5 year warranty.

 

 

Just be sure to read the fine print about data retrieval...

 

https://medium.com/@karppinen/how-i-ended-up-paying-150-for-a-single-60gb-download-from-amazon-glacier-6cb77b288c3e

[mariushm]

42 minutes ago, Razor Blade said:

Just be sure to read the fine print about data retrieval...

 

https://medium.com/@karppinen/how-i-ended-up-paying-150-for-a-single-60gb-download-from-amazon-glacier-6cb77b288c3e

[Radium_Angel]

2 hours ago, Pangea2017 said:

This is not suitable for long term storage or for simple backups.

Nonsense.

[Lady Fitzgerald]

4 hours ago, Pangea2017 said:

This is not suitable for long term storage or for simple backups.

 

2 hours ago, Radium_Angel said:

Nonsense.

Actually, Pangea2017 is correct. Standard CDs, DVDs, and BDs have fairly short life spans, just handful of years at the very best. Archival quality discs last quite a bit longer but have limited capacity and are expensive.

 

The problem here is expecting any solution to be good for a decade or more. Technology is constantly changing and storage media becomes obsolete over time. A lot of archived data gets lost because people waited until the hardware need to read the data from the media wqs no longer available, having been replaced by newer technology. Mediia that is available today may be obsolete ten years for now.

 

The solution is for data archives to be updated to newer technology long before the current technology can become obsolete, rendering the data unreadable. Also, archived data should be accessed frequently to ensure it is still there.

 

Currently, tape and HDDs are the best for long term data storage. SSDs are usuable only if they are accessed once or twice a year to replace the charge that slowly bleeds down while unpowered for extended periods.

[Razor Blade]

4 hours ago, mariushm said:

So make sure you plan for the "per gigabyte" retrieval fee if you ever need to get your data back.

[Radium_Angel]

8 hours ago, Lady Fitzgerald said:

11 hours ago, Radium_Angel said:

 

Actually, Pangea2017 is correct. Standard CDs, DVDs, and BDs have fairly short life spans, just handful of years at the very best.

Well then, I guess you better tell that to my 15 year old DVDs that I have my pics stored on, because I went back last night to pull data from them, to see how badly deteriorated they were supposed to have been, and had no issues with any of them.

So I hit my music CDs (yes, CDs, that;'s how old they are) and played a few. No issues there.

Now for long long term (25 years) sure, they probably will be bad, but I maintain 10 years is nothing for an optical disk.

Shit, at work the other day I pulled data of some floppy disks, without a single issue mind you...83 floppies, that were written in the *90s* with the oldest one being 27 years old.

No issues with reading and pulling data off them, and they had not been stored gently either.

 

So, I stand by my assertion that 10 years is a snap for optical media.

 

And while we're on the subject, I had a 160 *Megabyte* Conner (remember them? Very few of us would without a Google search) hard drive, also 27 years old (build date was 1992) that was still viable for pulling data from it.

 

[Lady Fitzgerald]

2 hours ago, Radium_Angel said:

Well then, I guess you better tell that to my 15 year old DVDs that I have my pics stored on, because I went back last night to pull data from them, to see how badly deteriorated they were supposed to have been, and had no issues with any of them.

So I hit my music CDs (yes, CDs, that;'s how old they are) and played a few. No issues there.

Now for long long term (25 years) sure, they probably will be bad, but I maintain 10 years is nothing for an optical disk.

Shit, at work the other day I pulled data of some floppy disks, without a single issue mind you...83 floppies, that were written in the *90s* with the oldest one being 27 years old.

No issues with reading and pulling data off them, and they had not been stored gently either.

 

So, I stand by my assertion that 10 years is a snap for optical media.

 

And while we're on the subject, I had a 160 *Megabyte* Conner (remember them? Very few of us would without a Google search) hard drive, also 27 years old (build date was 1992) that was still viable for pulling data from it.

 

I'm glad your media has survived for so long but you're an exception to the rule--most people do not fare that well--and you probably have been lucky enough to have used high quality discs and have ideal storage conditions. Refer to these articles:

 

https://www.pcworld.com/article/124312/article.html

 

https://www.theguardian.com/technology/askjack/2008/may/08/howlongshouldadvdlast

 

Flopy disks aren't optical media; they are magnetic, same as HDDs and tape. How many  people even have floppy drives anymore? Those are becoming increasingly hard to find. I haven't even seen one in well over a decade although 3.5" floppy drives still can be found online. Good luck finding 5.25" and 8" floppy drives. Should the drives your employer ever have fail, that data will become inaccessable if, by then, the drives are no longer available.

 

I've had burned CDs from independent artists that failed in two to five years. I've even had stamped commercial CDs and DVDs fail in as little as two years up to ten years. I stand by my assertion that, unless expesive archival quality ODD disks are used, optical media is not suitable for long term storage.

 

 

[Zodiark1593]

1 hour ago, Lady Fitzgerald said:

I'm glad your media has survived for so long but you're an exception to the rule--most people do not fare that well--and you probably have been lucky enough to have used high quality discs and have ideal storage conditions. Refer to these articles:

 

https://www.pcworld.com/article/124312/article.html

 

https://www.theguardian.com/technology/askjack/2008/may/08/howlongshouldadvdlast

 

Flopy disks aren't optical media; they are magnetic, same as HDDs and tape. How many  people even have floppy drives anymore? Those are becoming increasingly hard to find. I haven't even seen one in well over a decade although 3.5" floppy drives still can be found online. Good luck finding 5.25" and 8" floppy drives. Should the drives your employer ever have fail, that data will become inaccessable if, by then, the drives are no longer available.

 

I've had burned CDs from independent artists that failed in two to five years. I've even had stamped commercial CDs and DVDs fail in as little as two years up to ten years. I stand by my assertion that, unless expesive archival quality ODD disks are used, optical media is not suitable for long term storage.

 

 

My experience with optical media has been very mixed. Some of my media that had been burned functions well after 10 years. I've also had media die within months of burning.

[mariushm]

3 hours ago, Radium_Angel said:

Well then, I guess you better tell that to my 15 year old DVDs that I have my pics stored on, because I went back last night to pull data from them, to see how badly deteriorated they were supposed to have been, and had no issues with any of them.

So I hit my music CDs (yes, CDs, that;'s how old they are) and played a few. No issues there.

Now for long long term (25 years) sure, they probably will be bad, but I maintain 10 years is nothing for an optical disk.

Shit, at work the other day I pulled data of some floppy disks, without a single issue mind you...83 floppies, that were written in the *90s* with the oldest one being 27 years old.

No issues with reading and pulling data off them, and they had not been stored gently either.

 

So, I stand by my assertion that 10 years is a snap for optical media.

 

And while we're on the subject, I had a 160 *Megabyte* Conner (remember them? Very few of us would without a Google search) hard drive, also 27 years old (build date was 1992) that was still viable for pulling data from it.

 

See https://en.wikipedia.org/wiki/Disc_rot and https://www.makeuseof.com/tag/cds-truth-cddvd-longevity-mold-rot/

CDs lasted for a longer time because they were made better than DVDs and because the data is not as densely packed and when burning the discs the laser used to hit the surface much harder, making a stronger digital signal. 

 

I had CDs that I burned using an external 1x HP writer (on parallel port if you can believe me) and those were some of the discs that lasted the longest but even with those eventually the data on the outside area of the disc started to become hard to read due to edges deteriorating.

 

DVDs are even more sensitive and the quality of the burn process will vary with the type of disc (the chemicals in the data layer) and how well the firmware in the dvd writer is optimized for that chemistry - there are actually profiles in the dvd writer firmware which change how the writer burns data depending on what the disc (the disc has on it information like what speeds it can handle, the chemistry used and other crap)

 

With DVDs even the quality of the burning is different from drive to drive and depending on the chemicals inside the disc. Some discs use some chemicals and some discs use others, and DVD writers have different - the quality of the data will vary from writer to writer and between discs ... see http://www.digitalfaq.com/guides/media/dvd-tests.htm -  see #5 in that page .. such tools tell you have easy the writer can read the data stored on disc and how hard the error correction works to read the data that was just freshly put on disc ... imagine as the disc ages and the plastic goes bad and so on, how much harder the unit will work to read data if the data was already badly written on disc from the start.

 

Anyway.. even with magnetic tapes it's a pain in the ass... in most archival places, the humidity and temperature is controlled and even then every few months they go through the shelves and FLIP the tapes to reduce the effect of electromigration ... bits on digital tapes can be flipped over time due to this, so every few months, they go and flip the tapes on the other side. Then there's the risk of tapes getting sticky over time... it's not perfect.

 

Yeah, floppy discs were relatively safe but that's only a round disc of material between two layers of soft material that cleans the dust right away. Tapes are miles of band and the surface of the tape is pretty much constantly on contact with some other surface. But floppy discs were still damaged by floppy drive heads and other things, weren't really reliable.

 

Mechanical hard drives are really much better than both dvd/cd/bluray, and probably only tapes exceed the reliability ... but for quantity vs price you can't beat them.

 

Much safer than these... probably laser etching dots on metal sheets or printing on archival paper with lots of error correction (but you're talking about maybe 100 KB per page or something like that)