What is the best HDD for an amateur archivist?

[Bleda412]

I am currently in the market a drive that will hold data for before offloading it to MDISCs. From what I've read, a hard drive is better than an SSD for archival grade storage. Is that true? I've also read that Seagate is the best company to go with right now. I've been looking at the Exos series and the Barracuda. Do you have any suggestions? Does capacity have any impact on the safety of my data?

 

Secondary question:

It seems best to me to go with MDISC for long-term storage. I was thinking of going with Blu-rays, but I wasn't sure of what size to go with, uncertain if that would impact reliability. Then, I saw this guy saying that a consumer's only choice is to go with DVDs because consumer Blu-ray burners are too faulty. Is that true? If not, what is a very good external Blu-ray burner that can handle MDISC?

[HQuan]

Hard drive is 4-6x efficient and better than SSD in data archive, long-time storage or heavy files. Capacity does not affect to the safety of data. What affect is the brand design and you.

 

There's no hard drive for amateur archivist. But if you archive thing for long time, use MDSIC like you said, or hard drive. Western Digital is best for you.

Yep, MDSIC is the best for long-term storage and not use often. Just stand right above DVD-RW.

Blu-ray in fact have more failure chance than red-ray (joke). But that not means blu-ray is bad. In fact you can fix a broken blu-ray by just adjust the potentiometer in some case. 

BLU-RAY CAN'T HANDLE MDSIC.

[Blue4130]

2 hours ago, HQuan said:

Hard drive is 4-6x efficient and better than SSD in data archive, long-time storage or heavy files. Capacity does not affect to the safety of data. What affect is the brand design and you.

 

There's no hard drive for amateur archivist. But if you archive thing for long time, use MDSIC like you said, or hard drive. Western Digital is best for you.

Yep, MDSIC is the best for long-term storage and not use often. Just stand right above DVD-RW.

Blu-ray in fact have more failure chance than red-ray (joke). But that not means blu-ray is bad. In fact you can fix a broken blu-ray by just adjust the potentiometer in some case. 

BLU-RAY CAN'T HANDLE MDSIC.

Tape is king. If you are serious about archiving, go lto tape. 

[RayLeech]

2 hours ago, Bleda412 said:

First:

a drive that will hold data for before offloading it to MDISCs.

 

Secondary question:

MDISC for long-term storage. ...  this guy saying that a consumer's only choice is to go with DVDs because consumer Blu-ray burners are too faulty. Is that true?

First: For the hard drive, It depends on how long you're storing data until you create mdisc. If you're wait for a month or couple months, almost any reasonable drive is fine, including ssd's. That assumes you have the original and one copy until you burn the mdisc(s).

 

Second: DVD/Blu-ray. The guy has a lot of good information, just a terrible presentation! I found the article more to my liking. I create mdiscs for my long-term archives. Mostly irreplaceable photos/videos, my business documents and data. I have a combination of mdisc DVD and BluRay, depending on the volume of data.  I found both are equally reliable, after 10 years (DVD) and 6+ years (BR).  Like anything other media, its possible to burn coasters, but if the mdisc passed the verify stage, I've not had any trouble with them after.

 

About the burner: I have an external ASUS USB3 burner and a stack of internal burners (mainly LG). Look for the logo on the burner and the compatibility list if you are concerned with a particular drive.

 

About disk size: use the one that is appropriate for a single burn. For example, I burn photos/videos only once a year (the week between Christmas and NewYear). Most years, that means a pair of 25gig BR mdisc. During the plandemic, I had significantly less, so a pair of DVD mdisc were just fine.

 

Lastly, hard drives are a fine medium for backup and archival, if you're smart about it. I use three enterprise quality drives per rotation. One backup off-site in a vault, two backups on-site but off-line (beyond the on-line PC/NAS copies). When it comes time to rotate media, Duplicate live data to HD1, duplicate HD1 to HD2, send HD2 to the vault and retrieve HD3 from the vault. Duplicate HD1 to HD3 when it arrives. Two on-site (off-line) backups and one off-site. If you're data is really important, its not that much of an inconvenience. About your youtube guy: if you throw your stuff on a drive (any drive) and just abandon it for 8+ years, it must not be that important. Even good data tapes should be verified over a period of time.

 

 

 

 

[HQuan]

43 minutes ago, Blue4130 said:

Tape is king. If you are serious about archiving, go lto tape. 

Agree..!

But nowadays finding a tape reader is pretty hard, and if you did not store data for 50+ year, it waste of money,

 

Edited December 30, 2021 by HQuan

[RayLeech]

2 hours ago, HQuan said:

Agree..!

But nowadays finding a tape reader is pretty hard...

 

Finding them is easy. Paying for them is a bit harder ($3k-$8k)

 

[wseaton]

3 hours ago, Blue4130 said:

Tape is king. If you are serious about archiving, go lto tape. 

Tape is not king. Ive worked with tape on all platforms including booting AS400s off them. LTO units are expensive, clunky, impossible to repair, and becoming scarce. You can backup all you want to tape, and then realize your heads were slightly off and other units have trouble reading the cartridge. I refuse to archive to tape unless the client commits to having a spare unit verified and stored in a box in case the first one fails. Pretty much any computer on the planet can read a SATA or blu ray.

[Blue4130]

3 hours ago, wseaton said:

Tape is not king. Ive worked with tape on all platforms including booting AS400s off them. LTO units are expensive, clunky, impossible to repair, and becoming scarce. You can backup all you want to tape, and then realize your heads were slightly off and other units have trouble reading the cartridge. I refuse to archive to tape unless the client commits to having a spare unit verified and stored in a box in case the first one fails. Pretty much any computer on the planet can read a SATA or blu ray.

Your not wrong, but the mediums archival is still king. 

 

As for Amateur level, tape is out I guess due to cost alone. 

[HQuan]

AS A AMATEUR, go with hard drive, MDISC like you say and............................................. nothing more. Like I said on the first comment.

[Bleda412]

22 hours ago, RayLeech said:

First: For the hard drive, It depends on how long you're storing data until you create mdisc. If you're wait for a month or couple months, almost any reasonable drive is fine, including ssd's. That assumes you have the original and one copy until you burn the mdisc(s).

 

Second: DVD/Blu-ray. The guy has a lot of good information, just a terrible presentation! I found the article more to my liking. I create mdiscs for my long-term archives. Mostly irreplaceable photos/videos, my business documents and data. I have a combination of mdisc DVD and BluRay, depending on the volume of data.  I found both are equally reliable, after 10 years (DVD) and 6+ years (BR).  Like anything other media, its possible to burn coasters,a

 

About the burner: I have an external ASUS USB3 burner and a stack of internal burners (mainly LG). Look for the logo on the burner and the compatibility list if you are concerned with a particular drive.

 

About disk size: use the one that is appropriate for a single burn. For example, I burn photos/videos only once a year (the week between Christmas and NewYear). Most years, that means a pair of 25gig BR mdisc. During the plandemic, I had significantly less, so a pair of DVD mdisc were just fine.

 

Lastly, hard drives are a fine medium for backup and archival, if you're smart about it. I use three enterprise quality drives per rotation. One backup off-site in a vault, two backups on-site but off-line (beyond the on-line PC/NAS copies). When it comes time to rotate media, Duplicate live data to HD1, duplicate HD1 to HD2, send HD2 to the vault and retrieve HD3 from the vault. Duplicate HD1 to HD3 when it arrives. Two on-site (off-line) backups and one off-site. If you're data is really important, its not that much of an inconvenience. About your youtube guy: if you throw your stuff on a drive (any drive) and just abandon it for 8+ years, it must not be that important. Even good data tapes should be verified over a period of time.

 

 

 

 

Thank you for this. I agree, that YouTuber's delivery was very abrasive.

When you say, "but if the mdisc passed the verify stage, I've not had any trouble with them after," I am relieved. I'm going to be offloading more than a few photos, and I want to have a number of copies, so Blu-ray is much preferred. With that in mind, would you suggest the 25GB MDISC size, the lowest Blu-ray size, or does size not matter when it comes to making clean burns and long-term survivability? One more question about survivability. I had my 25GB MDISC spindle fall off of a guitar case onto the ground (1 ft). They should be fine, right?

What program and speed do you use to burn MDISCs? Compression was mentioned to me before, which adds another complicating factor. Does that make sense here? I'm storing various types of media (pictures, videos, audio, books). Generally, I want the data to be as accessible as possible (think time capsule).

 

EDIT: I'm using an 8TB Barracuda HDD, bigger than I need but good price and shipping speed compared to other Barracuda options. Is that a good drive for my purposes? Should I be going with something like an Exos instead? This is the result of my test of my new Barracuda.

[RayLeech]

Hi Bleda412

Quote

I'm going to be offloading more than a few photos, and I want to have a number of copies, so Blu-ray is much preferred. With that in mind, would you suggest the 25GB MDISC size ... making clean burns and long-term survivability

My suggestion is use the right size media for the burn job. I use a lot of 25g and 50g BR mdisc, along with some DVD mdisc. I have a spindle of 100g mdisc, but haven't used them yet. The key is to make sure you have a good burner and verify your disc after burn. Any burn software worth its salt has a verify option. Xcopy/Robocopy have it. Just verify you didn't burn a coaster!

 

Quote

25GB MDISC spindle fall off of a guitar case

It is probably ok. You didn't say if they already contained data. Use one to read (write) to put your mind at ease. If they already contained data, I would suggest you treat your backups more like a fragile newborn child. Here, discs are all transported in a cd carrier, even from room to room. HDs are put in a latched, padded case. Media is organized and stored on sturdy shelves/cabinets. Everything climate controlled, away from potential water hazards, sunlight, etc. You reap what you sow (so to speak), so how 'nuts' you get is up to you.

 

Quote

Compression was mentioned to me before, which adds another complicating factor.

This is where a lot of people get screwed, including myself! I used a backup program a very long time ago that had ceased being produced. Years later when I needed a couple files, realized I never backed up the software, and had a hell of a time trying to restore!  If you encrypt anything, make sure you have a secure and redundant way to store the encryption passwords. I can't tell you how many people (several of them professionals who should have known better) wanted help retrieving data they encrypted and forgot the password (basically game over). If you're going to do anything beside straight file systems, make sure the exact software version you used (along with any registration key) is backed up somewhere, preferably on the media.

 

After my little fiasco, I usually do straight filesystem or ISO in the burn software for insensitive data. For sensitive data, encrypt with something like 7Zip/WinZip. Unless you're archiving military secrets, it's probably good enough for your bank records. Often I'll use 7Z, so I burn the install pack and the compressed file on disk.

 

Quote

8TB Barracuda HDD

Again, unless you're talking 'years unplugged', they all work fine. My long-term off-line media are all enterprise, mainly because I 'feel' better. I have no data to support it. As I mentioned in last reply, rotate your media, periodically verify.

 

Quote

Generally, I want the data to be as accessible as possible (think time capsule).

You might consider copying your data to a NAS for quick access, and off-line for backup. Keeping the data live somewhere makes verification easy. It might be overkill for your current project, so just a suggestion.

 

Quote

Missing Question: Organization

The thing you never mentioned is organization". Make your self a spreadsheet, or at worst a paper copy of what you archived and when. Nothing is more frustrating than needing to get a backup from a box of unlabeled (or poorly labeled) media! Speaking from experience here. I was called to a client site to restore a crashed system. They literally handed me a cardboard box with 6 hard drives and many dozens of dvds. The drives were numbered 1 to 6, but he didn't remember which one he used last, and all the DVDs were labeled 'Backup'. Bottom line, anything you do is better than nothing. At minimum identify the disc, contents, and date.

 

Best of luck with your project!

 

 

[Bleda412]

4 hours ago, RayLeech said:

Hi Bleda412

My suggestion is use the right size media for the burn job. I use a lot of 25g and 50g BR mdisc, along with some DVD mdisc. I have a spindle of 100g mdisc, but haven't used them yet. The key is to make sure you have a good burner and verify your disc after burn. Any burn software worth its salt has a verify option. Xcopy/Robocopy have it. Just verify you didn't burn a coaster!

 

It is probably ok. You didn't say if they already contained data. Use one to read (write) to put your mind at ease. If they already contained data, I would suggest you treat your backups more like a fragile newborn child. Here, discs are all transported in a cd carrier, even from room to room. HDs are put in a latched, padded case. Media is organized and stored on sturdy shelves/cabinets. Everything climate controlled, away from potential water hazards, sunlight, etc. You reap what you sow (so to speak), so how 'nuts' you get is up to you.

 

This is where a lot of people get screwed, including myself! I used a backup program a very long time ago that had ceased being produced. Years later when I needed a couple files, realized I never backed up the software, and had a hell of a time trying to restore!  If you encrypt anything, make sure you have a secure and redundant way to store the encryption passwords. I can't tell you how many people (several of them professionals who should have known better) wanted help retrieving data they encrypted and forgot the password (basically game over). If you're going to do anything beside straight file systems, make sure the exact software version you used (along with any registration key) is backed up somewhere, preferably on the media.

 

After my little fiasco, I usually do straight filesystem or ISO in the burn software for insensitive data. For sensitive data, encrypt with something like 7Zip/WinZip. Unless you're archiving military secrets, it's probably good enough for your bank records. Often I'll use 7Z, so I burn the install pack and the compressed file on disk.

 

Again, unless you're talking 'years unplugged', they all work fine. My long-term off-line media are all enterprise, mainly because I 'feel' better. I have no data to support it. As I mentioned in last reply, rotate your media, periodically verify.

 

You might consider copying your data to a NAS for quick access, and off-line for backup. Keeping the data live somewhere makes verification easy. It might be overkill for your current project, so just a suggestion.

 

The thing you never mentioned is organization". Make your self a spreadsheet, or at worst a paper copy of what you archived and when. Nothing is more frustrating than needing to get a backup from a box of unlabeled (or poorly labeled) media! Speaking from experience here. I was called to a client site to restore a crashed system. They literally handed me a cardboard box with 6 hard drives and many dozens of dvds. The drives were numbered 1 to 6, but he didn't remember which one he used last, and all the DVDs were labeled 'Backup'. Bottom line, anything you do is better than nothing. At minimum identify the disc, contents, and date.

 

Best of luck with your project!

 

 

Thank you for this reply too, very helpful stuff. The discs have never been written on. I will test them as soon as I get an MDISC burner. I am aware that disc burning software has a verification feature, but is there a way to verify hard drives?

When I rotate drives out of storage, how long do they have to be turned on? Is there a way to "keep them fresh" without really using it, other than to check if everything is okay?

I would have asked about organization, which is a concern of mine, but I have not been sure of how to address it, especially in a conversation like this. For instance, if I download a book from the Internet Archive by author A, and then a few years later, I am able to find a little known book by him that I want to save. One file is on one disc and another file is on another disc. Is the best way of dealing with such organizational issues is to group things the best I can, and when I add more data in the future, I keep a manifest of my archive to look at whenever I want to dig up something from a particular author or subject?

Those are all my questions. I think I am going to get started archiving pretty soon. I have a friend studying to be an electrical engineer who was interested in helping me, but he's got difficult classes, a job, and an internship, so I haven't heard from him in a couple months. For more than a year, I've been hindered by my lack of tech knowledge and my search for perfection, the enemy of the good. Thank you so much for your time and expertise.

[RayLeech]

58 minutes ago, Bleda412 said:

but is there a way to verify hard drives?

If you are using backup software to copy to HD, use that software's verify. If you are copying manually, use Xcopy/RoboCopy (both have a 'verify' option on the command line).

Quote

When I rotate drives out of storage, how long do they have to be turned on? Is there a way to "keep them fresh" without really using it, other than to check if everything is okay?

There has been a lot written about that in this forum (maybe a year ago), and on the net.  Spin up your drives every year or two seems to be the consensus. Store all your media in a climate controlled area. For me, it depends on what's on the media. My most important HD, I spin up every 6mo and give a cursory review, every 1yr do a verify and/or refresh. Less important HD, spin up every 1yr, refresh every 3 year. For the mdisc, I spin them every 2 yr or so and just do a quick verify.  Whatever timespan you pick, just don't go into "neglect mode". I still have files that were originally stored on 8" floppy disk from 40+ years ago (and a dozen other media formats) that are still readable in my archive today. It just requires a bit of attention.

 

Quote

organization ... One file is on one disc and another file is on another disc. Is the best way of dealing with such organizational issues is to group things the best I can, and when I add more data in the future, I keep a manifest of my archive to look at whenever I want to dig up something from a particular author or subject?

I try to organize things the best I can. Pictures/Video/Audio/RawData I burn by date. Other 'things' like your book example are a different challenge. Since you already keep a manifest, you already have the start for your catalog system. Just track archive media where you can find it.  You could also use a catalog program like WinCatalog (there are several).

 

Quote

I've been hindered by my lack of tech knowledge and my search for perfection, the enemy of the good. Thank you so much for your time and expertise.

'search for perfection', famous last words   I've been doing this for decades, and still don't have a "perfect plan" for most things in life. All that matters is your plan is good enough to achieve your goals.

 

You're very welcome.