The power of netflix, on a single disk: A 700TB optical drive?!

[WolframaticAlpha]

Summary

Researchers have laid out plans for a 700TB optical disk drive and the production of the disk is actually underway. These discs use a nanocomposite material to make this possible. The design has been termed suitable for mass production by the researchers. For scale, this single disc contains data worth 28000 blu ray discs. The main use of these discs will be in datacenters.

 

Quotes

Quote

Their goal is to make a disk with a long-lasting store to end the dependence on magnetic discs with a limited life span

The researchers stated, “ while the advances are needed to optimize the technology, the results open new avenues to advance the global challenge of data storage. The technology is suited to the mass production of optical disks so that the potential is enormous,”

 

My thoughts

At the risk of sounding like billie g(640k is enuff), WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

 

Sources

Researchers build 700TB optical disc that can probably store all of Netflix - TECHTELEGRAPH

Researchers working on material that could yield 700TB optical discs | TechRadar

Researchers Built a 700TB Disk That can Store An Entire Netflix Data - THEPRO (theprotechie.com)

[Techstorm970]

1 minute ago, WolframaticAlpha said:

My thoughts

 

At the risk of sounding like billie g(640k is enuff), WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

I see this as a huge win for the enterprise markets.  Datacenters will love these!

[Murasaki]

2 minutes ago, WolframaticAlpha said:

WHY THE HECK WOULD YOU NEED 700TB

2 minutes ago, WolframaticAlpha said:

The main use of these discs will be in datacenters.

 

 

[WaggishOhio383]

5 minutes ago, WolframaticAlpha said:

WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

A) It's not for the average consumer. You aren't buying these things to store your homework on them

B) It's not just about how much the drive can store, but also how long-term it can store that data and in how compact of a form factor it can do that. For large datacenters those are important things to consider.

[WolframaticAlpha]

6 minutes ago, WaggishOhio383 said:

A) It's not for the average consumer. You aren't buying these things to store your homework on them

B) It's not just about how much the drive can store, but also how long-term it can store that data and in how compact of a form factor it can do that. For large datacenters those are important things to consider.

Ah, now I get it!

[Bananasplit_00]

OK but is it smaller than 28000 blue ray disks? And how many years does it take to write an entire disk?

[mariushm]

20 minutes ago, WolframaticAlpha said:

My thoughts

At the risk of sounding like billie g(640k is enuff), WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

 

 

 

Long tail of things.... daily backups, big volumes of data that need to be accessible but with low seek or transfer speed demands (ex high resolution shots of some region and you can load tens of GB of images as you scroll or zoom in somewhere)

 

long tail I mean content that's accessed very rarely ... Think for example how many Youtube videos have just a few views a year?  Dump those videos on such a disc, and if/ when someone accesses the video, start serving the first minute or so of the video (which you still keep in live storage) and while that's streamed, a robot or something can pull the disc from a rack and plug it in a optical drive and load it ... all within 10-2s or so ...

 

Similar to how tape drives would work but without the seeking.

 

[Tellos]

Another thing to remember, is tape drives compressed data can go up to 30TBs in some cases. BUT magnetic tape CAN degrade over time and is SLOW to read. Plus not every new system can normally handle tape drives so if you can do optical drives it's much easier to use on more modern systems. Also if more compact as stated before storage becomes much less an issue. 

 

Also possibly cost. 2.5TB 5 pack of IBM tapes is almost 125 bucks and thats on the lower side of some of them. If this optical is less that could significantly help.

[BuckGup]

 I highly doubt you could store all of Netlix on 700Tbs unless they took metrics from the lowest quality settings compressed to hell. One BluRay is 60Gbs and the new UHD Blurays are over 100Gbs per movie. So 7,000 4k movies? Maybe but as of 2021 Netflix has the rights to 13,612 titles. That means some are TV shows that have feature movie length episodes and have 8 episodes per season with multiple seasons. So whoever did the napkin math just wanted that clickbait title goodness. 

[leadeater]

1 hour ago, Techstorm970 said:

I see this as a huge win for the enterprise markets.  Datacenters will love these!

mmm not sure about that, optical discs tend to not have nearly as good storage longevity as LTO tapes do.

[Pickles von Brine]

Currently enterprise uses a lot of tape. I think the largest LTO tape does like 30TB uncompressed. Having a disc that can do 700TB is not only cost effective, but also space saving. If these discs are also super durable and reliable then having long term storage with such large capacities would be a win. I have heard of several data centers reaching into the exobytes and having backups is a bitch to maintain at those capacities. 

[X-System]

2 hours ago, WolframaticAlpha said:

My thoughts

[...] WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

But it's still not enough for Linus Media Group servers

 

 

[Kisai]

7 hours ago, WolframaticAlpha said:

 

 

My thoughts

At the risk of sounding like billie g(640k is enuff), WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

 

You've never met a data-hoarder.

 

To put it bluntly, enterprise systems use tapes, and the tapes, while being reliable, are also not random-access. So if you ever need to restore data, you have to either:

a) restore every tape in sequence for delta backups

b) find a complete backup and restore the entire thing

 

What would be helpful is if each server had a ODD that could use such a disc, even if that disc is never removed, just keep adding sessions to it until there's no space left, so you reduce the complexity of the server with the ODD, and when the disc is done, that goes into your permanent archive, and you pop a new disc  in and start with a full backup again and do incremental after that.

 

That said, M-disc's claim to last centuries, but I sincerely doubt that except under controlled conditions, which would also be true for regular burned discs. Like most of the CD-rom's i burned in 1998 will still work as long as nothing has touched them.

[PeachGr]

Note that to have any reason to exist, it needs an Nvme write speed or multiple write/ read I/O. The reason for that, is that on 300mbps, you will need a decade to fill, so much if 200users will search all together to find a file, cause that disk will have many  simultaneous users

[Maticks]

your estimated raid rebuild time is 250 Days...

[RejZoR]

10 hours ago, Techstorm970 said:

I see this as a huge win for the enterprise markets.  Datacenters will love these!

Not really. Optical discs are super useless for enterprises. Single write, fragile medium. They are not even really useful for backups. And I can only imagine it would take forever to write one at such capacity.

[whm1974]

26 minutes ago, RejZoR said:

Not really. Optical discs are super useless for enterprises. Single write, fragile medium. They are not even really useful for backups. And I can only imagine it would take forever to write one at such capacity.

I tried using a few DVD-RW I gotten cheap. You know, Packet Writing and all that. So slow that I ended up just using them as DVD-R instead.

[Sauron]

Should be enough for my meme collection

[Loote]

10 hours ago, BuckGup said:

 I highly doubt you could store all of Netlix on 700Tbs unless they took metrics from the lowest quality settings compressed to hell. One BluRay is 60Gbs and the new UHD Blurays are over 100Gbs per movie. So 7,000 4k movies? Maybe but as of 2021 Netflix has the rights to 13,612 titles. That means some are TV shows that have feature movie length episodes and have 8 episodes per season with multiple seasons. So whoever did the napkin math just wanted that clickbait title goodness. 

Netflix UHD is up to 25 Mbit/s, aka 16GB per movie on average. It is in many ways inferior to normal Blu Rays, no to mention UHD ones. You should also remember a lot of content there doesn't have a 2160p version.
But Netflix does keep different versions of the movie and for their own productions they must keep RAWs, which probably take tens of TBs for a single TV series. It's also an example usage for a big cheap resilient storage medium, after you cut and master the movie, move the raw files onto such disks, you still can produce a different cut when needed, but you don't need to eat the precious space on your servers.
Sure, 700TB sounds outrageous, but it's probably going to have more sizes than BD which ranges from 25GB to 128GB right now.

Quote

the production of the disk is actually underway

So someone must've found a use case for it, I am curious.

[leadeater]

5 hours ago, Kisai said:

a) restore every tape in sequence for delta backups

b) find a complete backup and restore the entire thing

No that isn't correct.

 

If you want to restore a file or a set of files all you need is the tapes with the data on them, not the whole tape set and daily incremental tapes. Order is not important.

 

If you want to restore everything, a full restore to latest point in time, then you need the latest full and then only the daily backup tapes for the latest version of the files if they have changed (daily backups are done on disk now days anyway). Also restoring in sequence is not a requirement either. You can mount as many tapes as you like, from the full or the daily, and restore from them.

 

What you are probably confusing is transaction log backups for database servers. What you said is correct for that, you have to do a restore from the full and then replay the transactions from the transaction logs up until the time point you set to restore to. This however has no relation to tape medium at all, this process is the same if you backup to disk.

[Bombastinator]

19 hours ago, WolframaticAlpha said:

Summary

Researchers have laid out plans for a 700TB optical disk drive and the production of the disk is actually underway. These discs use a nanocomposite material to make this possible. The design has been termed suitable for mass production by the researchers. For scale, this single disc contains data worth 28000 blu ray discs. The main use of these discs will be in datacenters.

 

Quotes

 

My thoughts

At the risk of sounding like billie g(640k is enuff), WHY THE HECK WOULD YOU NEED 700TB. Even the most studious students don't have that much homework.

 

Sources

Researchers build 700TB optical disc that can probably store all of Netflix - TECHTELEGRAPH

Researchers working on material that could yield 700TB optical discs | TechRadar

Researchers Built a 700TB Disk That can Store An Entire Netflix Data - THEPRO (theprotechie.com)

I’m assuming it would compete with tape backup.  Might be faster.  Optical burns were always very slow, even after they got faster.

[Doobeedoo]

Oh a ton of data on that.

[Kisai]

7 hours ago, leadeater said:

No that isn't correct.

 

If you want to restore a file or a set of files all you need is the tapes with the data on them, not the whole tape set and daily incremental tapes. Order is not important.

 

And how would you know you have the right version if you don't restore all of them hmm?

 

The office couldn't be bothered to even retrieve the tapes for a lawsuit they were involved in because it was too expensive to have the data restored, instead they seeked out the physical disk backup that was made before the switch over to cloud storage.

[Mark Kaine]

18 hours ago, leadeater said:

mmm not sure about that, optical discs tend to not have nearly as good storage longevity as LTO tapes do.

I would hope these are magneto optical, which has very good longevity, otherwise, yeah, its probably not great.

 

 

[StDragon]

22 hours ago, leadeater said:

mmm not sure about that, optical discs tend to not have nearly as good storage longevity as LTO tapes do.

Depends on the optical disk.

CD Audio are pressed which puts the reflective material sandwiched between layers of polycarbonate, so that will last a very long time in ideal conditions.

 

CDR and CDRW have the media on the other side of the label, which is printed on top of single polycarbonate layer. The problem is that with humidity and heat, it will oxidize and "desilver" much like the edges of an old mirror. That's data literally rotting off. Hence the phrase "bit rot". Average shelf life 5 to 10 years. In theory, they could last hundreds or thousands if kept in a sealed container with 100% nitrogen (inert) gas at 1 bar. But that's not a practical solution at archival.

 

Then there's M-Disk format which can last up to a thousand years and beyond.

Personally, if I was to archive family photos and data for generations (say the next thousand years), I would choose M-Disk. Even if the tech is so obsolete as to find a reader, they might have flatbed scanners that have a resolution high enough to scan the entire disk into a virtual ISO file, and pull data off it. Or, some future replicator to build a new reader. It's "low-high tech".

 

Tape wont last that long. Even tape in the 70s has to be slowly baked in an oven to separate the material, and you're lucky to get just one raw read from it if it's a destructive read.