Magnetic Tape Storage is making a comeback and could replace hard drives in Enterprise and business storage

[Pickles von Brine]

12 minutes ago, FakeCIA said:

I have multiple offsites, friends houses and storage units. Very rarely do I use the cloud.

321 is being followed even if not in the cloud. 

[Lady Fitzgerald]

2 hours ago, XenosTech said:

You mean several offsite backups. Just incase same thing happens to one of you offsites, all your data isn't lost.

While I totally agree that having multiple backups is a good thing, it doesn't have to be limited to just offsite backups. Also, you have to weigh benefit against cost.

 

The reason for having an onsite backup is it is far easier for keep it up to date at all times than to keep an offsite backup up to date at all times. If you put critical, can't afford to lose data onto your computer, you can quickly back it up to your onsite backup. However, if something happens to the onsite backup as well as the original data, such as the fire mentioned earlier, then having an offsite backup will at leat save most of your bacon data (which is the reason to keep the offsite backup as up to date as practical).

 

The problem with most offsite backups is they are harder to keep up to date than onsite backups. Data you haven't had a chance to put onto your offsite backup because it is physically located out of immediate reach will be lost if you lose the data on your computer. That's why you have an onsite backup as well as an offsite backup. Still, recovering only most of your data beats losing all your data.

 

I'm not saying having multiple offsite backups is a bad idea. As long as it is cost effective, both in time and money, you can't have too many backups anywhere. Still, the chances of losing both an onsite backup and an offsite backup at the same time is so low, it's not enough to worry about. You can't easily protect your data from, say, an asteroid strike that takes out both your onsite backup and your offsite backup. It would be pointless, anyway, since the asteroid strike would probably take you out as well.

 

A few words about cloud backups. First, most cloud storage sites are not all that secure. Unless you encrypt your data before uploading it, it is subject to being hacked and/or harvested. The ones that are secure are expensive. A good cloud backup service, on the other hand, has far better security (some of those servers are better protected than Fort Knox). Besides being physically secure, they will encrypt your data for you before they upload it to their servers.

 

The advantages of a cloud backup service include that all the work of updating the backup is done for you and your backup is as up to date as possible. The backup may be on a server hundreds of miles away from you, reducing the chances of data loss from a local disaster.

 

The downsides to cloud backups include the cost which can range from $60 a year, the last time I checked, to hundreds of dollars or more, depending on the level of security, the amount of data being backed up, and the speed of uploads and downloads available. If you have very much data and depending on the level of service you pay for, recovering from a total loss can take a long time, as much as several weeks. You also have to have a broadband connection that is fast enough to handle uploading all the new and changed data you generate every month and has a high enough data cap to handle all that data without hitting you with data overage charges (which are usually outrageous).

 

The only affordable and reliable backup service I can recommend for most people is Backblaze and it is limited only to Windows and Mac OS (which is the only reason I don't use it; I'm on Linux now). While affordable, data recovery will be slow if if you ever need to do a full recovery and you have very much data. For that reason, I recommend also having a physical offsite backup. That way, if the worst were to happen and you lost your data and the onsite backup, you could relatively quickly recover most of your data from the offsite backup, then download the data that hadn't made it to the offsite backup yet from the cloud backup in far less time than it would have taken to download all your data. I used to do that with Carbonite before their customer service was moved offshore again and their customer service quality plummeted.

 

If you are as paranoid as I am (not necessarily a bad thing), you could have two onsite backup drives and two offsite backup drives for every data drive in your computer. This can get expensive if you have very much data.

[straight_stewie]

This has me thinking:

 

Sony reports that the highest magnetic tape data density they've been able to achieve is 148Gb/in².

Seagate reports that their HAMR drives have a data density of 2Tb/in².

 

So let's say we want a state of the art 250GB drive. For hard drives that takes 1in² of platter area. For tape that takes ~13in² of tape area. That's a 1:13 storage density ratio.

So, in terms of space efficiency, tape still loses. By a long shot. However, I think you could get IOPS performance close to that of a harddrive. And here's why:

With a 1 inch wide tape, we need a roughly 13 inch long tape. If we use 2 inch diameter wheels on each end, we have two flat planes that are roughly 6 inches long on which to put read/write heads. This gives us a lot of space to use multiple heads. By combining a fast (for tape) reel speed with modern head speeds, and multiple heads, you might be able to approach the random read/write performance of even the most advanced hard drives.

Of course, I understand that that's not the point of tape drives, but it would make for an interesting engineering exercise.

[leadeater]

1 hour ago, straight_stewie said:

So, in terms of space efficiency, tape still loses. By a long shot. However, I think you could get IOPS performance close to that of a harddrive. And here's why:

You need to adjust your calculations for volume of space, tapes kill hard drives when you do that. Tapes don't need to be flat ridge surfaces.

 

Also:

Quote

In a new world record, scientists at IBM have captured 330 terabytes of uncompressed data — or the equivalent of 330 million books — into a cartridge that can fit into the palm of your hand. The record of 201 gigabits per square inch on prototype sputtered magnetic tape is more than 20 times the areal density currently used in commercial tape drives.

 

 

1 hour ago, straight_stewie said:

Of course, I understand that that's not the point of tape drives, but it would make for an interesting engineering exercise.

Well nobody direct to tapes for LTFS anyway, gateways with caches have been used for years. Other systems tier data down through multiples stages form SSD to HDD to Tape.

 

This, Virtual Integrated File System (VIFS), is just a better iteration of LTFS.

[Kisai]

14 hours ago, Forgotten_Fox said:

"Fujitsu has announced a new technology called Virtual Integrated File System that it says could help magnetic tape storage compete with hard disk drives as a low-cost, large capacity storage alternative... This iteration will deliver capacities up to 26.1TB (uncompressed) and raw throughput of up to 708MB/sec."

 

It's essentially RAID for magnetic tape storage, which could be a cheaper and faster alternative to large capacity hard drives.

 

Original link: https://www.techradar.com/news/tape-could-replace-hard-drives-in-some-cases-thanks-to-this-breakthrough

That's barely faster than a SATA SSD. Any single PCIe NVME would be faster, and my guess here is random access would be measured in minutes.

 

[leadeater]

33 minutes ago, Kisai said:

That's barely faster than a SATA SSD. Any single PCIe NVME would be faster, and my guess here is random access would be measured in minutes.

 

NVMe is very much not low cost storage though.

 

Also that 708 MB/s is for a single LTO-9 tape drive and the purpose of VIFS is to stripe the data access over multiple tape drives, to address that latency, so even with a small deployment of 4 drives that's 2832 MB/s. These tape systems aren't really designed to be used with small deployments though so 4 drives is likely the smallest viable configuration if it even is viable.

 

Add on compression and you're getting throughput of over 1 GB/s easily, 1770 MB/s theoretical with optimal compression, so for the purposes of these systems you'll be purely network limited at factors of hundreds of times cheaper for large capacities compared to NVMe, or any SSD at all.

[Kisai]

1 hour ago, leadeater said:

NVMe is very much not low cost storage though.

 

Also that 708 MB/s is for a single LTO-9 tape drive and the purpose of VIFS is to stripe the data access over multiple tape drives, to address that latency, so even with a small deployment of 4 drives that's 2832 MB/s. These tape systems aren't really designed to be used with small deployments though so 4 drives is likely the smallest viable configuration if it even is viable.

 

Add on compression and you're getting throughput of over 1 GB/s easily, 1770 MB/s theoretical with optimal compression, so for the purposes of these systems you'll be purely network limited at factors of hundreds of times cheaper for large capacities compared to NVMe, or any SSD at all.

How much are the tapes though.

[leadeater]

11 minutes ago, Kisai said:

How much are the tapes though.

Well I think we pay about $60 NZD per tape for LTO-7, fairly sure for a bulk order I can get LTO-8 tape for around $100-$120 (pre LTO-9 prices). I would only be able to guess based on price history from subsequent tape generations in the past so like $200 NZD per LTO-9 tape, $125 USD. Have to bulk buy to get those kinds of prices though, much higher if only buying like a pack of 10.

 

$125 USD for 24TB raw to 60TB compressed is a great deal cheaper than any NVMe that would actually get deployed in a server configuration, e.g. Samsung PM1725a. Edit: and you actually have to have multiple servers to put them in unlike a tape library.

[mr moose]

I would love a tape system to archive all my data, I have quite a lot of photos and home movies I'd like to get off hdd's in the near future.

[leadeater]

4 minutes ago, mr moose said:

I would love a tape system to archive all my data, I have quite a lot of photos and home movies I'd like to get off hdd's in the near future.

I'd still stick to HDDs for home use, cheap old tape drives are a time bomb and I don't think you want to be spending $5k AUD to $15k AUD for a new tape drive, every 5 years.

[mr moose]

Just now, leadeater said:

I'd still stick to HDDs for home use, cheap old tape drives are a time bomb and I don't think you want to be spending $5k AUD to $15k AUD for a new tape drive, every 5 years.

I didn't want to hear that, what you were supposed say was that I can get a decent tape drive for A$100 and tapes for $40 that would hold data for 50 years.

 

 

 

 

 

 

 

 

[leadeater]

2 minutes ago, mr moose said:

I didn't want to hear that, what you were supposed say was that I can get a decent tape drive for A$100 and tapes for $40 that would hold data for 50 years.

Tape has never made sense in small scale, not even way back in the day when it was the only option, the reason most used it in the first place. Now you should only use it if it actually makes sense and pretty much almost always is no it doesn't. Tapes are hyper edge case now.

[Kisai]

20 minutes ago, leadeater said:

Tape has never made sense in small scale, not even way back in the day when it was the only option, the reason most used it in the first place. Now you should only use it if it actually makes sense and pretty much almost always is no it doesn't. Tapes are hyper edge case now.

Unfortunately, there is no "permanent" storage medium. Magnetic storage loses it's magnetic storage ability after a while due to stray magnetic sources, and unless rewritten, is lost. Flash memory loses it's charge after some time (though how much time is still up for debate), not sure if flash memory "rots" the same way magnetic storage does though it's probably safe to assume neither will survive strong EM fields. The chip might just burn out rather than get erased.

 

Optical storage unfortunately is not going to survive either since plastic can not be made to last long enough and the dye's fade with time. Early optical media (laserdiscs) physically rotted due to bad manufacturing. CD-R's are destroyed by scratches on the label-side. DVD-R and BD-R's tend to be a bit better at that, but they are still ultimately just sandwitched together and the possibility of rotting due to poor storage/handling still exists. So Optical I wouldn't trust, at least not on plastic media.

 

Still waiting for that day Holographic storage exists.

 

[leadeater]

17 minutes ago, Kisai said:

Unfortunately, there is no "permanent" storage medium. Magnetic storage loses it's magnetic storage ability after a while due to stray magnetic sources, and unless rewritten, is lost.

Tapes back then were generally cycled, still are today. But now instead of shipping tapes off site you're much better off pushing the data to an S3 cloud bucket. In the past there just weren't any options like that and even if there were you didn't have the internet connection to make any practical use of it.

 

In saying that much of the really cheap archive cloud storage tiers are actually tape, those 3 to 6 hour recall times are a dead giveaway of that. I have my doubts there is much wide scale usage of blu ray for this given that tape is very mature of that usage and easily customized hardware and physical storage wise. 

 

For us at most data will sit on the same tape for 2 years before being overwritten. Library itself is in a complete underground concrete room with it's own humidity controlled air con, UPS and generator backup feed.

 

Only time my hands touch a tape is when it's new out of the box and being placed in to the library or is ejected from the library because it's deemed faulty, smashed and secure document destroyed.

[Franck]

17 hours ago, WereCatf said:

devices handling those tapes would still cost an arm and a leg

Yup they are crazy expensive last 2 one i bought was in 2013 and it was 8,000$ and 14,000$ for 2 little darn box.

[Guest]

1 hour ago, Kisai said:

Unfortunately, there is no "permanent" storage medium.

 

 

 

Cave paintings seem quite a permanent storage medium..

[Guest]

1 hour ago, leadeater said:

Tapes back then were generally cycled, still are today. But now instead of shipping tapes off site you're much better off pushing the data to an S3 cloud bucket. In the past there just weren't any options like that and even if there were you didn't have the internet connection to make any practical use of it.

 

In saying that much of the really cheap archive cloud storage tiers are actually tape, those 3 to 6 hour recall times are a dead giveaway of that. I have my doubts there is much wide scale usage of blu ray for this given that tape is very mature of that usage and easily customized hardware and physical storage wise. 

 

For us at most data will sit on the same tape for 2 years before being overwritten. Library itself is in a complete underground concrete room with it's own humidity controlled air con, UPS and generator backup feed.

 

Only time my hands touch a tape is when it's new out of the box and being placed in to the library or is ejected from the library because it's deemed faulty, smashed and secure document destroyed.

The biggest problem a lot of companies have with cloud storage is security. Trust issues, potential political restrictions, responsibility for hacking, loss of control etc are all things that have to be considered. That is why products lie Eternus CS and VIFS exists, and why companies like Quantum are still in business.

 

Yes, tapes are cycled and rarely touched. They sit in a library away from human hands. Human hands are the biggest risk in most backup scenarios.

[leadeater]

16 minutes ago, Phill104 said:

The biggest problem a lot of companies have with cloud storage is security. Trust issues, potential political restrictions, responsibility for hacking, loss of control etc are all things that have to be considered. That is why products lie Eternus CS and VIFS exists, and why companies like Quantum are still in business.

Realistically a lot of those data security and sovereignty concerns aren't actually that big, don't just go out and use Bob's Super Awesome Storage Ltd hosted in Russia 

 

In the absence of one of the big players having a in country datacenter there's always a local provider you can use and it's easy to vett if they are trusted or not. You're either big enough to handle it yourself and it's on you if your operational procedures are not up to scratch or you aren't and it's better to pay for a service from someone that can do it properly. It does you no good to think you are doing it securely just because you are in control when in fact you're actually not and when it comes to backups restorability is the only thing that matters, otherwise don't even bother doing it.

 

Can be as simple as dropping one tape in a recovery set and that it's entire backup set is gone. Or worse if you do get security breached and access to backup server connected to tape library is gained all tapes can be secure erased that are currently in the library, in larger libraries all tapes are able to be loaded so you better notice they are getting erased pronto. 

[Guest]

30 minutes ago, leadeater said:

Realistically a lot of those data security and sovereignty concerns aren't actually that big, don't just go out and use Bob's Super Awesome Storage Ltd hosted in Russia 

 

In the absence of one of the big players having a in country datacenter there's always a local provider you can use and it's easy to vett if they are trusted or not. You're either big enough to handle it yourself and it's on you if your operational procedures are not up to scratch or you aren't and it's better to pay for a service from someone that can do it properly. It does you no good to think you are doing it securely just because you are in control when in fact you're actually not and when it comes to backups restorability is the only thing that matters, otherwise don't even bother doing it.

 

Can be as simple as dropping one tape in a recovery set and that it's entire backup set is gone. Or worse if you do get security breached and access to backup server connected to tape library is gained all tapes can be secure erased that are currently in the library, in larger libraries all tapes are able to be loaded so you better notice they are getting erased pronto. 

Most of our customers either have their own data centres, or use one of the many we have. They want air gaps between their hardware and those belonging to anyone else. This is especially true for government run agencies, some of which are still big users of tape.

[leadeater]

13 minutes ago, Phill104 said:

They want air gaps between their hardware and those belonging to anyone else.

Doesn't matter when you (they) are the one that got breached and the lovely "we are in control" is the reason your (their) backups got erased . I say this because it's happened more than once. There's really good examples of epic fails from bad cloud security and also on-prem failures, but when it's a cloud failure it's typically not the provider but rather customer/operator error.

 

Paranoia itself doesn't keep you safe, sort of sadly. 

 

Don't get me wrong I'm very much not a cloud is the best thing ever type, not even it's the defacto best choice. I actually have a blanket rule of "if it's IaaS then it's not the correct choice", IaaS pricing sucks so find a PaaS or SaaS solution or rent rack space. But I'm weary of 'industry trends' and sales engineers because their is always bias in the mix and just because something is a trend doesn't make it best fit for you.

[Guest]

11 minutes ago, leadeater said:

Doesn't matter when you (they) are the one that got breached and the lovely "we are in control" is the reason your (their) backups got erased . I say this because it's happened more than once. There's really good examples of epic fails from bad cloud security and also on-prem failures, but when it's a cloud failure it's typically not the provider but rather customer/operator error.

 

Paranoia itself doesn't keep you safe, sort of sadly. 

 

Don't get me wrong I'm very much not a cloud is the best thing ever type, not even it's the defacto best choice. I actually have a blanket rule of "if it's IaaS then it's not the correct choice", IaaS pricing sucks so find a PaaS or SaaS solution or rent rack space. But I'm weary of 'industry trends' and sales engineers because their is always bias in the mix and just because something is a trend doesn't make it best fit for you.

We have had customers in the past go the whole hog completely having solutions cut off from the outside world. Thing is, costs become a problem when all that kit is duplicated. In one case some out of support kit was used well beyond it sell by date and it was only when, because there was no external monitoring, multiple disks failed in an array after a power outage they realised their choices were ill advised. So there has to be a balance struck between security from external interference and failure security.

 

Like you, I am weary of trends influencing decisions, especially when many can see those decisions are poor ones. Cloud is the buzz word of the moment, as is open systems. People see anything else, particularly mainframes, as outdated tech. Instead they should be open and looking at which tech is best suited for the task from performance, prices and security points of view. 

[Doobeedoo]

For long term cold storage and backups and yeah IOPS is definitely not it's thing. 

I wonder though, what could completely replace them one day, a type of storage that's not mechanical based and can have great capacity but with not much worse performance too. 

[TechyBen]

22 hours ago, manikyath said:

mehh.. randoms still wont be nearly fast enough to make a big dent in hard drive deployment for most use cases.

Yep. Reduce the cost and I'd use this as a home archive media. I've had a good run, but I know a lot of family/friends/people have external drives fail. Both SSD and HDD are more delicate to faults/vibrations than (AFAIK) a tape based backup. And most people using weekly/monthly backups are gonna be doing consecutive writes/reads on such a task.

If it's small too... could it be a competitor to SSD/HDD external drives?

 

But as always, marketing sees money in a market/area totally *not* fitting for the product, and tries to rebrand and get the engineering team on board with the car crash...

 

Rubber Band Co Marketing meeting: Oh, lots of money in interplanetary launches markets... can we use our bands to launch payloads???

[TechyBen]

11 hours ago, straight_stewie said:

This has me thinking:

 

Sony reports that the highest magnetic tape data density they've been able to achieve is 148Gb/in².

Seagate reports that their HAMR drives have a data density of 2Tb/in².

 

So let's say we want a state of the art 250GB drive. For hard drives that takes 1in² of platter area. For tape that takes ~13in² of tape area. That's a 1:13 storage density ratio.

So, in terms of space efficiency, tape still loses. By a long shot. However, I think you could get IOPS performance close to that of a harddrive. And here's why:

With a 1 inch wide tape, we need a roughly 13 inch long tape. If we use 2 inch diameter wheels on each end, we have two flat planes that are roughly 6 inches long on which to put read/write heads. This gives us a lot of space to use multiple heads. By combining a fast (for tape) reel speed with modern head speeds, and multiple heads, you might be able to approach the random read/write performance of even the most advanced hard drives.

Of course, I understand that that's not the point of tape drives, but it would make for an interesting engineering exercise.

Size of the HDD vs size of the tape system. Tape readers are larger than a single HDD, but a single HDD only accesses the platters *in* that HDD, you can have a single tape reader, and *many* tapes. Thus, at some point (server size settings) a tape system and an automated robot arm swapping tapes is smaller than a HDD array (or at the very least, less failures ).

 

IIRC a 7tb tape is about the size of a HDD... A Fujitsu LTO 8 is about 12TB and smaller than a HDD. Some HDDs now go up to close to 20tb though. So that's a better measure of "density" as it's taking in more of the overall system to compare to.

[straight_stewie]

9 hours ago, leadeater said:

You need to adjust your calculations for volume of space, tapes kill hard drives when you do that. Tapes don't need to be flat ridge surfaces.

I wasn't calculating for volume. I was calculating for areal density.

And my thought experiment was specifically to improve the random read/write performance of tape drives, with no regard for the archival performance. Random read/write performance does not care about volume density, it only cares about how quickly you can get to the data you want, hence, we can't run the tape to-and-fro from reel-to-reel, we must continuously run the entire tape past the head, as disk drives do.

 

Hence:

11 hours ago, straight_stewie said:

Of course, I understand that that's not the point of tape drives, but it would make for an interesting engineering exercise.