Digital Tape Storage

[joshfrog]

hi guys 

so i work for a media company and where looking into digital tape storage to archive projects. 

we cant upload to the cloud because we have crap internet and its something we cant fix.

i don't know much about digital tape storage so any help would be useful thanks.

 

some of the questions i have are,

what types are there and whats the best option?

whats the difference between the types of cartages if any?

and what is a good brand for the storage because i see many brands i know of and many i dont?

 

any help would be great thanks

[leadeater]

LTO is the current standard for tape back and archive. LTO-7 is the newest version but is still rather expensive so likely better to go with LTO-6.

 

Many different vendors make tape libraries and drives etc. Have a look at Quantum and HPE.

[Jarsky]

Check out tape libraries by the likes of HPE, Quantum and IBM. 

We use some massive libraries, but I also manage some smaller ones like the IBM TS3200 which uses the LTO-6 tapes. Quantum make some similar libraries to this as well. 

 

Generally you'll want to make your backups to disk, then to the tapes. 

We use various backup solutions for different cases and different infrastructure but primarily we use TSM (Tivoli Storage Manager) and Brightstor ArcServe. 

No doubt there are a lot of cheaper solutions out there though. 

 

You may wish to also look into tape storage companies as well if you require safe offsite storage. They will generally give you a schedule of inventory of your tapes, do dropoff/pickup and hold them in secure vaults. 

[leadeater]

5 hours ago, Jarsky said:

Brightstor ArcServe

I'm sorry you have to use that, I really am 

[Erkel]

The problem with tape is restore and management, if you using it archive files.

I personally would not go there,  Get a decent HDD based archive server and use tape to back it up, ie offline storage for DR.

 

Tape can be used in the way that you want, but has historically been the domain of giant systems as a secondary storage tier and is fully automated.   You can have multiple petabytes per rack now, so they are out of fashion.

 

What volumes are we talking about?    We talking about hundreds of terabyte or petabytes?  

[joshfrog]

thanks for your help guys 

the way out back-ups work at the moment is we have our main server a qnap server then every hour that backs up to a 2nd qnap server. once a video projects finished we dump the project on to external hard drive and delete from both servers. for many projects thats the only copy we have and we keep them in the office because we need to go back to them from time to time.

i have being worried for a while that if there was a fire or someone broke in to out office and stole anything they saw we could lose not only the video we are working on but also old projects.

 

where a small non profit media company so we dont have buckets of money to invest in a 3rd back up option that we need.

our best option would be cloud storage to back up our servers to but our office is in a bad spot for internet where we get about 1mb up and down so it would take takes to just back up whats on our server now let alone new footage as it comes in.

 

im looking in to tape because i saw the price of tape and how cheap it can be next to external hard drives. we already back up to external hard drives but as said we keep them in the office so i have being trying to find a system thats cheap and something my boss could take home so its offsite.

 

out server at the moment holds about 16tb of storage and during a year we may go though 200tb or more and where about to go 4k so any advice or ideas would be a big help thanks guys 

[theBadGeek]

I work in a small video and photo department for a hospital system, which sounds a little similar to your setup. Backing up to external drives is a nightmare is the waiting. I worked at a video production company that did that before I got there and continued once I started. Despite my regular insistence that we get a tape storage solution, they were quite happy with having a wall of external drives with years of projects and resources sitting in them until one drive after another began to fail and a massive amount of assets were lost. After that we upgraded to a LTO3 system (was years ago) and never lost another file in archive.

 

Were I am now, we use LTO to make long term archives for our projects on a regular basis. We're Mac based so we use Bru for the software and a HP LTO4 drive for the hardware. It works very, very well for us. We have limited storage space (which I'm trying to solve) and this allows me to manage our limitations without a huge hit to our speed. While writing to tape can take hours (at 1.3TBs per tape, I don't use compression), recovering individual projects or files is speedy enough not to adversely effect our overall speed of delivery. And Bru makes finding assets fairly easy. I've used LTO for years now (close to a decade) having it be a very reliable option. Below is a link to the LTO6 version of what we use now (with a Thunderbolt adapter), it's been rock solid.

 

https://www.bhphotovideo.com/bnh/controller/home?A=details&O=&Q=&ap=y&c3api=1876%2C{creative}%2C{keyword}&gclid=Cj0KEQjwx6TJBRCWtsiXpI7bhOYBEiQA1en3F7LmuuDYr2JYnBgNJuwLv99zUVy5G84p2wr7A0npw8MaAtCs8P8HAQ&is=REG&m=Y&sku=1217933

 

[NZLaurence]

I agree with the LTO drive for backup. However remember if you only have it in one place its not a backup.

 

In the past we have done a HDD for the last year. But when a project was finished, archive it to two tapes. We were using one tape per project and as they were no more then a 100gb a pop we used LTO3 tapes i think. About $10 a tape, coated into the project. One copy stayed onsite the other went offsite.

[Ramaddil]

If the data is that mission critical, isolating it to a single point of failure is a bad idea.  Ideally you should have at least three copies of your data.  I personally have three for my personal files, two servers and one set of drives that are stored off site and retrieved for bi weekly backups.  

[theBadGeek]

NZLaurence and Ramaddil are absolutely right. Even with LTO, best practice is doing a double copy, one for local use, the other for offsite storage. One thing that frustrates me where I am now is we don't do that, against my advice. Also something that needs to be understood, even as rock solid as LTO is, no storage is forever. If you are archiving assets that need a very long shelf life (speaking 5+ years), a best practice is to duplicate that archive every few years onto fresh media. It can be an added headache and expense, but tape can fail, it's just less likely to than other media (if stored properly). 

[samiscool51]

On 2017-5-20 at 4:43 PM, leadeater said:

(bump)

On 2017-5-20 at 10:54 AM, Jarsky said:

(bump)

 

 

 

[Mikensan]

If after completing a project you delete it from your 2x NAS devices and stick it onto an external hard drive, that's essentially cold storage, not really a backup. I do understand your dilemma though, only so much storage available and only so much of a budget.

 

I don't think you'll be able to get the video files to compress on to tape, and with LTO6 you're only getting 2.5TB. So if you have a need to archive 200TB/yr that's about 80 tapes... It's definitely going to cost some money every year.

 

I don't know how feasible it is, but one idea is to take these external disks to another site to put them onto the cloud. Be it at home or another office. You could even automate it so you just hookup the disk and it starts pulling data off it and shoves it into the cloud. Do that once a week or once a month, pretty solid solution. Although 200tb/ a year is going to cost a large sum of money for cloud storage too.

 

Long story short, you're going to need more revenue or a really nice sponsor.