Archiving like it's the 90's - Optical disks could soon replace tapes for long-term storage.

[BachChain]

Summary

 

Ohio-based company "Folio Photonics" has announced the development of a new high-density long-term archival technology based on a design reminiscent of the venerable CD. The "ACTIVE" disk format, as it's called, sports a full terrabyte of storage per disk. The stated advantages include:

• a century or more of shelf-life
• resistance to adverse temperature and humidity
• immunity to radiation and EMPs
• random access capability
• 80% better power efficiency and 5x better cost per TB compared to hard drives

The company also expects each disk to cost less than five dollars at launch, and for that cost to eventually go down to around one dollar per. Expected launch is slated for 2024

 

Quotes

Quote

The new discs are claimed to hold 1TB each, and are expected to be packaged in 10-disc caddies. A particular highlight of these optical discs is that they can be produced at a "radical low-cost." Thanks to the use of next-gen materials and efficient production tech, ACTIVE discs cost less than $5 per TB, with a roadmap to $1 per TB.

Quote

While archival discs today employ three layers at the most, Active discs are created with next-gen materials, patented polymer extrusion, and film-based disc construction processes, explained Monroe. The discs work in concert with customized optical pickup units which together facilitate discs with up to 16 film layers per side.

 

My thoughts

We'll see how well the claims pan out. I'm not super familiar with datacenter tech, so I can't really say if it will really take off.

 

Sources

https://www.tomshardware.com/news/new-optical-discs-cost-less-than-dollar5-per-tb

https://foliophotonics.com/news/folio-photonics-announces-breakthrough-multi-layer-optical-disc-storage-technology-to-enable?s=31

[Bombastinator]

I’m real curious about the specifics of the archival technology, but this looks real real promising.  My memory of the previous archiva cd tech was drilling into the substrate to a specific depth and that was the 1 or the 0. It was single layer only.  These things are going to be a lot slower than hard drives, but it’s archiving so that doesn’t matter very much.

[leadeater]

43 minutes ago, BachChain said:

The new discs are claimed to hold 1TB each, and are expected to be packaged in 10-disc caddies. A particular highlight of these optical discs is that they can be produced at a "radical low-cost." Thanks to the use of next-gen materials and efficient production tech, ACTIVE discs cost less than $5 per TB, with a roadmap to $1 per TB.

So less capacity and higher cost per TB than LTO tapes, LTO is already around the $1 mark.

[TrigrH]

22 minutes ago, leadeater said:

So less capacity and higher cost per TB than LTO tapes, LTO is already around the $1 mark.

So the real question is sequential read and write speed. If they are slower than tape this is a DOA product.

[leadeater]

11 minutes ago, Caroline said:

But not everyone has a tape drive at home. These discs might be an easier more friendly way to store a lot of files at once. If they fit 5.25" bays that is, hopefully this isn't another LaserDisc size disc.

TH's article mentions 10-disc caddies, I'd love a dual 5.25" drive, just like old floppies.

Not everyone will have one of these disc drives either. Tape drives also fit in 5.25" bays too. Really the only benefit is not being humidity sensitive, nothing else.

[Cavalry Canuck]

2 hours ago, leadeater said:

Not everyone will have one of these disc drives either. Tape drives also fit in 5.25" bays too. Really the only benefit is not being humidity sensitive, nothing else.

I’d argue that the EMP protection is becoming more relevant today than at any other time in the past 30 years. Yes tapes can be stored in protected locations, but better that the medium itself be unaffected by them in the first place.

[StDragon]

3 hours ago, leadeater said:

Not everyone will have one of these disc drives either. Tape drives also fit in 5.25" bays too. Really the only benefit is not being humidity sensitive, nothing else.

Tape doesn't age well. Not sure of the current film formulation, but older tapes would stick and require baking in a controlled environment so they can later be read for data recovery. It's a risky process that can incur data-loss.

 

Optical disks such as M-DISK stand a much greater chance of being read hundreds or even thousands of years if stored in container of inert gas such as pure nitrogen. By then, there wouldn't be a need for complicated electromechanical readers. These disks in the far future would probably be imaged off a flatbed scanner of some type and read.

[cmndr]

4 hours ago, TrigrH said:

So the real question is sequential read and write speed. If they are slower than tape this is a DOA product.

Ehh...

Sequential is overrated.

Tape has AWFUL random read/write.

Like AWFUL. Truly bad.

This is NOT my wheelhouse but assuming rewind time is representative of random seek time... https://www.ibm.com/docs/en/ts4500-tape-library?topic=performance-lto-specifications

 

1 minute rewind time implies an average seek time of 30 seconds. Middle 50% would be 15-45s. 95% CI would be 1.5s to 58.5s.
Compare against an HDD with an average seek time of ~10ms. It's 3000x faster.

I wouldn't be surprised if 30s is optimistic.

[wanderingfool2]

6 hours ago, BachChain said:

a century or more of shelf-life

I remember the old advertising that CDs/DVDs will last forever...only to be plagued by disc rot.

 

I would like to know why they think it will be stable for that long, and the technology they are using.

 

4 hours ago, leadeater said:

Not everyone will have one of these disc drives either. Tape drives also fit in 5.25" bays too. Really the only benefit is not being humidity sensitive, nothing else.

Well the other more immediate benefit is the closer to random IO that these have over tape.  So it could be used for some sort of vault storage where immediate access isn't necessary but still a reasonable length of access makes having it practical.

 

The other thing, not listed, will the cost of the read/write device itself.  If they can manage to pull it off for under $500, it is something that could be a lot more practical for smaller businesses...since tape drives tend to be prohibitively expensive for mid-range businesses (then again haven't looked into tape drives in years so maybe things have changed)...actually if they can get it to sub $200 for a burner/reader, that could actually make it in the realm of home backup again (I have so many TB of information that I want stored but wont change).

 

The big thing though, what's the write speeds, what's the seek speeds.  There are really too many variables to properly access what this technology will be like.

 

Seems like this is a write once, read many type of tech.  So could be good for law firms as well that need to legally retain data for a certain amount of time (prevents accidental rewrites)

 

5 hours ago, leadeater said:

So less capacity and higher cost per TB than LTO tapes, LTO is already around the $1 mark.

I'm curious, at what volume of purchase price?  The cheapest I could readily find is about $8/TB; which puts this still cheaper (if they can pull of $5)...with the average price being about $12/TB (uncompressed).

 

Reading their white paper, seems like they are targeting having 10TB discs eventually, and basing it off of tech that blu-ray brought forward...so the cost per write/read device might be sub $500...which if the claims of longevity are true, and if they can pull of $5/TB discs when they enter the market that might have quite a bit of use cases.  *That assumes they actually launch the product...they were suppose to launch one in 2022 based on the white paper at an estimated $3/TB or something like that*

[Quackers101]

Wasnt the glass cubes also very good for this. As an "3D" storage device, but was write once, read only.

[suicidalfranco]

Yes please

[Kisai]

31 minutes ago, wanderingfool2 said:

 

 

The big thing though, what's the write speeds, what's the seek speeds.  There are really too many variables to properly access what this technology will be like.

 

Seems like this is a write once, read many type of tech.  So could be good for law firms as well that need to legally retain data for a certain amount of time (prevents accidental rewrites)

 

Well assuming we're staying within the limits imposed of CD/DVD/BD (which is Under 23,000 RPM for a CD and 28000 for a DVD)  This why CD-ROM's never went over 52X and DVD/BD-ROMS were never seen over 16x.

 

So to burn an entire CD (20m at 4X) or DVD (15m at 4X) has to be time competitive to copying to a hard drive. So a 7200RPM hard drive is around 120MB/sec (7,200MB per minute) 108GB=15minutes. If a disc holds 1TB and presumably can be burned at, at least the speed of a 7200RPM drive, it would take probably 145m per 1TB disc, best case.

 

If we start looking at being comparable to a SSD, well, assuming you can even get a 1TB 4GB/sec SSD, 250 SECONDS to copy the drive. Somehow I doubt burning a 1TB disc in 4 minutes is going to be possible. We'll probably be bottlenecked to the SATA port speed (thus can't be less than 32 minutes) the drive gets connected at OR the drives will be USB4 only.

 

 

 

[Needfuldoer]

1 hour ago, Kisai said:

f we start looking at being comparable to a SSD, well, assuming you can even get a 1TB 4GB/sec SSD, 250 SECONDS to copy the drive. Somehow I doubt burning a 1TB disc in 4 minutes is going to be possible. We'll probably be bottlenecked to the SATA port speed (thus can't be less than 32 minutes) the drive gets connected at OR the drives will be USB4 only.

I wouldn't directly compare an offline backup storage media to an active SSD, they fill completely different use cases. Discs and tape will retain their data indefinitely when they're sitting on a shelf powered off, NAND flash won't.

[Bombastinator]

3 hours ago, StDragon said:

Tape doesn't age well. Not sure of the current film formulation, but older tapes would stick and require baking in a controlled environment so they can later be read for data recovery. It's a risky process that can incur data-loss.

 

Optical disks such as M-DISK stand a much greater chance of being read hundreds or even thousands of years if stored in container of inert gas such as pure nitrogen. By then, there wouldn't be a need for complicated electromechanical readers. These disks in the far future would probably be imaged off a flatbed scanner of some type and read.

I sort of assume some sort of tech like the stuff they use to read previous layers on vellum.  Plastic slumps so a thousand years from now they’d probably be seeing a flat disk with little lines in it where the plastic oxidized differently because it was exposed.  Still readable with tech assistance.  Possibly a huge PITA, but it would be archeologists looking at stuff so that sort of doesn’t matter.  Me I’m not sure if there will be people in a hundred years these days.

[Sauron]

8 hours ago, Caroline said:

But not everyone has a tape drive at home.

I doubt these won't require specialized drives. Most people don't have an optical drive of any kind at home anymore, anyway.

8 hours ago, leadeater said:

So less capacity and higher cost per TB than LTO tapes, LTO is already around the $1 mark.

I'm guessing the selling point would be drastically better random access compared to tape at a similar price point. Might also be easier to store than tape and less susceptible to damage from simply being dropped or transported on bumpy roads.

[Bombastinator]

4 minutes ago, Sauron said:

I doubt these won't require specialized drives. Most people don't have an optical drive of any kind at home anymore, anyway.

I'm guessing the selling point would be drastically better random access compared to tape at a similar price point. Might also be easier to store than tape and less susceptible to damage from simply being dropped or transported on bumpy roads.

A humidity thing was also mentioned. Inert gas storage isn’t cheap either though.  I suspect it’s going to com down to cost per megabyte with both media cost and media storage being taken into consideration.

[Kisai]

1 hour ago, Needfuldoer said:

I wouldn't directly compare an offline backup storage media to an active SSD, they fill completely different use cases. Discs and tape will retain their data indefinitely when they're sitting on a shelf powered off, NAND flash won't.

I think you missed what I was pointing out. I was pointing out that mechanical speeds are a limit that can not be overcome for mechanical media. If you're backing up SSD's, the best case is 32min/TB If you're a data hoarder, you're not gonna sit there and burn 50TB of stuff to an optical disc that you might not be able to read in 10 years because they don't make the drives anymore. That's the same problem that LTO drives have as well. Sure you can write to a tape today, but in 10 years, will there be a drive you can read the tape with? 

 

The safest storage medium for short term recovery is always going to be "another hard drive". For long term recovery, you're going to have to save the entire computer with the ODD, and if the ODD breaks while in storage? Well so much for any long term storage.

 

I feel that ODD's as backups has always been a bad idea, and even discs I burned back in 1997 tend to not be readable because guess what? CD-R's are not great backup media. The best case is using them as "data dumps", where you might save all your less-important work, eg drafts/WIP's, client data you aren't working with, your "download" folder, transcoded media, etc  to the disc so you have something to recover from, rather than absolutely nothing.

 

Like if I really gave half a care, I would sit there and AV1 encode my entire Disney BD collection and stick it on a USB drive for when I want to watch it, but I'd also burn a copy of it to an OD so I wouldn't have to do the rip again. The original BD's can then go in storage where kids aren't going to put their fingerprints on them. BUT, and I say this as someone who is the family archivist... There is no damn way am I going to scan all the photos and negatives a second time. I would rather burn a dozen discs and send them to every relative so they have a copy rather than put them on facebook. If they lose the disc, it's easier to make another, where as a USB drive is easier to lose or accidently overwrite.

 

[Bombastinator]

The thing about SSDs though is it’s electrostatic.  There is just a limit to how long that charge persists if it is not renewed and that limit isn’t terribly long if one is looking at time periods like what is needed in archiving. The things for these purposes are effectively RAM.  If you’re going to let something sit for 50 years or 500 the difference between a fraction of a second and 20 years doesn’t matter all that much.  Mechanical hard drives are ferromagnetic. In 500 years the platter will still be there and there will still be material on it but will that material maintain enough magnetic integrity to be readable?  At that point the methodology of the platter layout starts to matter.  I worry about that with shingled drives and they are becoming a lot more common.  The layers may interact with each other magnetically. There is also the method of attachment of the ferromagnetic material to the platter. If that degrades turning the thing into basically a platter with dust on it it can’t be spun without all the dust flying off and getting mixed up.  This is also a potential problem with tape.  There is the substrate, the adhesive material, and then the actual data containing material on top.  If the substrate disintegrates the data becomes scrambled.  If the adhesive disintegrates the data can become scrambled when reading is attempted.  And then there is print through.

Edited September 2, 2022 by Bombastinator

[Kisai]

6 minutes ago, Bombastinator said:

The thing about SSDs though it it’s electrostatic.  There is just a limit to how long that charge persists if it is not renewed and that limit isn’t terribly long if one is looking at time periods like what is needed in archiving. The things for these purposes are effectively RAM.  

There simply is no "good" archiving media in existence. Not even paper. We really should be looking at some kind of media that:

1. Is impossible to destroy (magnetic, EMP, Fire, mechanically, etc)

2. Simple to make or repair a reader for

3. Fast enough to use as a primary media, but random-accessible enough to use as archival media.

 

Like OD's all fail the first two points, tapes fail all three, and hard drives fail the first two. I'm not asking for media to survive a nuclear blast or lava, but I'm asking for it to survive a house fire, flood, being manhandled, being swallowed by children or animals, and so forth. Basically anything that someone encounters once or more in their life due to the climate or geographic hazards they live on/nearby.

 

Basically we need a "storage brick" about the size of deck of cards, no moving parts, the entire storage medium and it's reader is self-contained so that it can be connected via USB-C, or you can open it and transplant the storage medium into a new reader. No fancy encryption, compression, or other schemes that require specialized software in order for it to be used. Basically I just described an external USB-C hard drive with a SSD in it. So what we should be looking for is an archival "flash-like" media that is designed to be written to only once, and in the event that you store files on it that you want to delete, you invoke the same trick used with the cd-r multisessions, where it auto-versions the "drive" into each write session. So if you try to overwrite a file that already exists and hasn't changed, there's no additional storage taken up. One of the reasons why data is so hard to recover from hard drives is because the sector scheme makes it impossible to recover files that have been fragmented. So if the file is fragmented, you might be unable to recover more than the the first fragment.

 

 

[Bombastinator]

15 minutes ago, Kisai said:

There simply is no "good" archiving media in existence. Not even paper. We really should be looking at some kind of media that:

1. Is impossible to destroy (magnetic, EMP, Fire, mechanically, etc)

2. Simple to make or repair a reader for

3. Fast enough to use as a primary media, but random-accessible enough to use as archival media.

 

Like OD's all fail the first two points, tapes fail all three, and hard drives fail the first two. I'm not asking for media to survive a nuclear blast or lava, but I'm asking for it to survive a house fire, flood, being manhandled, being swallowed by children or animals, and so forth. Basically anything that someone encounters once or more in their life due to the climate or geographic hazards they live on/nearby.

 

Basically we need a "storage brick" about the size of deck of cards, no moving parts, the entire storage medium and it's reader is self-contained so that it can be connected via USB-C, or you can open it and transplant the storage medium into a new reader. No fancy encryption, compression, or other schemes that require specialized software in order for it to be used. Basically I just described an external USB-C hard drive with a SSD in it. So what we should be looking for is an archival "flash-like" media that is designed to be written to only once, and in the event that you store files on it that you want to delete, you invoke the same trick used with the cd-r multisessions, where it auto-versions the "drive" into each write session. So if you try to overwrite a file that already exists and hasn't changed, there's no additional storage taken up. One of the reasons why data is so hard to recover from hard drives is because the sector scheme makes it impossible to recover files that have been fragmented. So if the file is fragmented, you might be unable to recover more than the the first fragment.

 

 

It is likely that #1 is unsolvable. Even stone can be degraded. The most archival electronic media I think we have come up with so far is wire recording but even that is fairly susceptible.  One can increase probability though.  #2 I discount because after a period of time such things will be done by archeologists rather than common users. #3 I think misses the point of offline storage.  Sure it’s nice for things to be fast, but maintaining the bare existence of the data trumps the ease with which it is accessed.  It would affect what gets stored archivally though.  I am reminded at this point of Roman graffiti carved into rock in Italian sewers. Saved by location.

Edited September 2, 2022 by Bombastinator

[Needfuldoer]

1 hour ago, Kisai said:

I think you missed what I was pointing out. I was pointing out that mechanical speeds are a limit that can not be overcome for mechanical media.

My point is that backup drives don't have to be blazing fast, they have to be shelf-stable. Start a backup job, let it finish and verify, then leave the media offline where you'll hopefully never have to touch it again.

 

1 hour ago, Kisai said:

That's the same problem that LTO drives have as well. Sure you can write to a tape today, but in 10 years, will there be a drive you can read the tape with? 

There's a predictable cadence to LTO tape. Up to LTO-7, the drives can read up to two generations back and write their own and the previous generation. (An LTO-7 drive can read LTO-5 tapes, and read and write LTO-6 and LTO-7 tapes.) Starting with LTO-8, drives only handle their own generation and the one previous.

 

1 hour ago, Kisai said:

CD-R's are not great backup media.

Yeah, cheapo dye-based recordable discs aren't stable in the long term. M-Disc claims their proprietary formulation solves that problem, and it might also be less of a problem with phase-change metal alloy discs like CD-RW and DVD-RAM. (Of course this assumes the layers of the disc don't separate or degrade over time, like early CD-ROMs and many Laserdiscs have.)

 

Something tells me the tolerances on their 16-layer discs are going to be so tight that they'll be physically susceptible to dust and scratches. As it is, BDXL discs spread 100 or 128 gigs across 3 or 4 layers. Either this magic new disc is a different diameter, or they figured out how to double Blu-Ray's per-layer density. (They're probably also not compatible with other optical drives; at least a Blu-Ray drive can read anything from a BDXL to a 650 MB CD-R.)

[wanderingfool2]

3 hours ago, Kisai said:

Well assuming we're staying within the limits imposed of CD/DVD/BD (which is Under 23,000 RPM for a CD and 28000 for a DVD)  This why CD-ROM's never went over 52X and DVD/BD-ROMS were never seen over 16x.

 

So to burn an entire CD (20m at 4X) or DVD (15m at 4X) has to be time competitive to copying to a hard drive. So a 7200RPM hard drive is around 120MB/sec (7,200MB per minute) 108GB=15minutes. If a disc holds 1TB and presumably can be burned at, at least the speed of a 7200RPM drive, it would take probably 145m per 1TB disc, best case.

 

If we start looking at being comparable to a SSD, well, assuming you can even get a 1TB 4GB/sec SSD, 250 SECONDS to copy the drive. Somehow I doubt burning a 1TB disc in 4 minutes is going to be possible. We'll probably be bottlenecked to the SATA port speed (thus can't be less than 32 minutes) the drive gets connected at OR the drives will be USB4 only.

51 minutes ago, Kisai said:

I think you missed what I was pointing out. I was pointing out that mechanical speeds are a limit that can not be overcome for mechanical media. If you're backing up SSD's, the best case is 32min/TB If you're a data hoarder, you're not gonna sit there and burn 50TB of stuff to an optical disc that you might not be able to read in 10 years because they don't make the drives anymore. That's the same problem that LTO drives have as well. Sure you can write to a tape today, but in 10 years, will there be a drive you can read the tape with? 

 

The bit you quoted on mine though is a perfect example, I think, of where things like this could beneficial.  Backups of lets say a casefile doesn't need to occur in a live environment, but need to be retained for at least 6 years (sometimes decades).  LTO you would have to periodically re-write it to maintain the integrity.  With HDD/SSD you have to do the same thing.  For those you would likely need to keep those on lets say a file server, which opens you up to a lot more risks...and in some cases the amount of documents could be staggering.

 

Facebook has used blu-ray's before for cold storage data as well (ones that they don't think people will access, but still wants to be retained)  The reason, it was cheaper per TB than the alternatives and LTO doesn't work because they still needed the opportunity for random seeks.

 

Backwards compatibility is also a thing, which is why to answer your question yea you could read a LTO drive in 10 years time (although realistically you should be reading it before that and rewriting to a new one before then).  Just like you can still find CD rom drives, if successful, this would allow access to data for a long time (and if data was that critical, you would likely have a second one).

 

Error rates on harddrives and SSD's are also higher than proper archival storage.  I would not trust the data-integrity of a harddrive that's been sitting for a few years without being powered on (and def. wouldn't trust a SSD in that situation)

[Doobeedoo]

Ahh so they're preparing this for future CoD game cool.

But still cool to see especially if it's to be as cheap. Would be interesting if we'd go back to physical media. So price wise it's potentially for consumer side too. But yeah we'll see. 

[leadeater]

7 hours ago, StDragon said:

Optical disks such as M-DISK stand a much greater chance of being read hundreds or even thousands of years if stored in container of inert gas such as pure nitrogen

Well unless you're a museum then this is basically an ultra rare requirement. All our LTO-5 tapes and the data on them is irrelevant and gone through a data destruction chipper already anyway.

[jagdtigger]

3 hours ago, Sauron said:

I'm guessing the selling point would be drastically better random access compared to tape at a similar price point.

Since we are speaking about archival purposes random access isnt a thing. 99% of the time it will be a targeted thing looking for a specific file, grabbing the whole thing and putting it on a storage that is more suitable for working on said file.