NAS, DAS or into the case?

[grangervoldemort]

Going to use 2x 10TB WD Elements as backup. I will manually backup new files every week or so and by manually I mean simple drag and drop.

They will be kept physically disconnected from the PC other than when backing up to them.

 

> Should I be wary of magnetic depolarisation (happens to all drives; I just don't know in what time period it tends to happen) of the bits thus loss of all data on the backup WD Elements?

 

Perhaps ya'll can think of a better backup method that is automated BUT is simple to do. No having to learn complex software.

 

Building a new PC. Need 2x internal 10TB minimum drives.

 

Will be moving over 11TB of data to the new drives.

Have a Bluray ODD and a Lian Li SATA HDD Powerswitch that also sits in a 5.25 bay. 

 

NOTE: I can get 20% off WD products from their website.

 

Trying to decide what is going to be the best path to take for the 2x 10TB internal drives:

 

Option 1) Choose a case that has min 2x 5.25 bays and fit the Lian Li HDD Powerswitch along with my Bluray ODD.

Then put 2x 10TB WD RED's or 2x 10TB Seagate Barracuda Pro (these are NON NAS) which probably makes more sense as my PC is not on 24/7 and I only need access to the drives when grabbing the files.

 

Workflow will be something like this:

HDD switched on.

Files transferred from cameras to HDD.

HDD powered off. Hate noise.

Files transferred from cameras to NVME or a SSD (not sure which to get; apparently NVME have lower lifespans than SSD's).

Files on NVME/SSD os where the video editing will take place.

Once work is complete HDD will be powered back up and the new edited version transferred to it. All files on NVME will then be deleted.

> Once a week files will be backed up to the WD Elements manually and by manually I mean simple drag and drop.

 

Option 2) Buy a NAS unit and get 2x WD RED's. No RAID. NAS will never be on 24/7 and only on when I want to use it following the above workflow. This way I could get a smaller case if I wanted to and it opens up case options which means far more aesthetic choice.

 

Far more options if I get rid of the Bluray ODD BUT I really want to keep the ODD. What can I do for this? I was thinking of buying an external enclosure but that is just too much Then I have more cables for power and USB and I need to find a place to put the damn thing. I do not have a Bluray collection I have The Martian on Bluray. But I would like to possibly buy more blurays in the future; then I could hook the PC up to a TV and surround and watch on a big screen. Possibly.

 

I could also buy one more WD RED and do RAID. Or I could get two more RED's and have 2x redundancy. (Expensive)

 

Option 3) Buy a DAS and get 2x WD RED's or the Barracuda Pro's. Same workflow as above.

 

Option 4) Buy a single 10TB/12TB WD RED or Barracuda Pro inside the case, and use existing drives naked by buying a HDD USB dock. I would have to plug in the drive I wanted to use at the time into the dock. (Potential risk of static damage to the PCB under the drives this way).

 

Existing drives are internal unless otherwise stated: 2x 1TB, 2X 2TB, 1x 4TB external drive, 500GB SSD that is my OS drive but videos have to be saved on there as I have no more space on the other drives, and now this SSD is pretty much full, 2x 64GB SD cards, 128GB Micro SD, and now the S10 phone I bought has 512GB internal space. Phew.

 

Would also have to buy plastic cases to store the naked drives into.

The single 10TB/12TB drive would be for the newly edited videos only.

[Guest]

I used to roll with 6-10 HDDs in my case. Knocking it down to two and going with a NAS once 10GbE became affordable to do in the house was the best thing ever. Case choice was so limited over 2 HDDs. You could keep it super duper basic and use something as simple as Windows File History to backup your files. Why are you adverse to leaving the drives on 24/7? In my opinion, making the process so manual leads to more problems. It is doable though.

 

Edit: Actually my eyes skipped over the workflow section, I see what you mean now. It's a strange workflow, so I'll give it some thought and come back.

[grangervoldemort]

10 minutes ago, Den-Fi said:

SNIP

Why 10GbE? Will Ethernet not be fast enough for a RAID for me given my workflow atm?

[Guest]

1 minute ago, grangervoldemort said:

Why 10GbE? Will Ethernet not be fast enough for a RAID for me given my workflow atm?

I'm just impatient. With my current setup, when I archive projects I do so to a 4TB cache with a transfer speed of ~1.1GB/s that then flushes to the main storage overnight. I'd be limited to around 109MB over gigabit.

[grangervoldemort]

14 minutes ago, Den-Fi said:

I'm just impatient. With my current setup, when I archive projects I do so to a 4TB cache with a transfer speed of ~1.1GB/s that then flushes to the main storage overnight. I'd be limited to around 109MB over gigabit.

Imagine I am incredibly dumb. I don't know how you get slower speeds without 10GbE and what 10GbE is and what the setup is. Imagine I am your grandma. Please explain again imagining the above. Thanks.

 

Is there a better way to do my workflow?

 

I think this workflow is good right now as I do not need the drives on all the time. That would waste electricity and create noise. I only need the drives on as stated in the workflow as far as I am aware, which as you can see isn't a long amount of time at all (haven't ever edited a video yet, I have been filming for 5 years mainly on my phones - long story).

I edited my first post slightly. Give it a re-read will you.

[Electronics Wizardy]

44 minutes ago, grangervoldemort said:

Imagine I am incredibly dumb. I don't know how you get slower speeds without 10GbE and what 10GbE is and what the setup is. Imagine I am your grandma. Please explain again imagining the above. Thanks.

10gbe is just faster networking, normal ethernet is 1gbe, 10gbe ethernet is 10x the speed.

 

44 minutes ago, grangervoldemort said:

Is there a better way to do my workflow?

Id get the nas and put the drives in there, or just leave then on in your desktop. Turing then on and off seems like too much hassle. Also you can have the drives spindown automatically, so unplugging or having a switch seems pointless.

 

1 hour ago, grangervoldemort said:

Should I be wary of magnetic depolarisation (happens to all drives; I just don't know in what time period it tends to happen) of the bits thus loss of all data on the backup WD Elements?

No, doesn't happen that much, and your using the drives pretty often.

 

 

 

[grangervoldemort]

Just now, Electronics Wizardy said:

Id get the nas and put the drives in there, or just leave then on in your desktop. Turing then on and off seems like too much hassle. Also you can have the drives spindown automatically, so unplugging or having a switch seems pointless.

 

Windows/programs often makes the drives spin backup. I have indexing on, on all my drives.

Also if a hacker/virus got in and the drive was physically switched off, it can't do anything to it.

[Electronics Wizardy]

Just now, grangervoldemort said:

Windows/programs often makes the drives spin backup. I have indexing on, on all my drives.

Also if a hacker/virus got in and the drive was physically switched off, it can't do anything to it.

If your have a hacker/virus, then just restore backups.

 

Id really wouldn't worry about leaving drives spinning them, they often don't like being turned on and off a lot, and the hours won't hurt them. They also don't make much noise or use much power idling.

 

[grangervoldemort]

3 minutes ago, Electronics Wizardy said:

They also don't make much noise or use much power idling.

 

You haven't met my current setup then. The case randomly starts vibrating and the amount it vibrates varies over time; and I mean within seconds the vibration level varies. It is SO SO annoying. 
It can become a loud buzz.

 

It's this damn case.

 

Also hard drives make a fair bit of sound spinning anyway which is what they are doing when idling. 10TB drives must be much louder than average.

[Electronics Wizardy]

1 minute ago, grangervoldemort said:

You haven't met my current setup then. The case randomly starts vibrating and the amount it vibrates varies over time; and I mean within seconds the vibration level varies. It is SO SO annoying. 
It can become a loud buzz.

 

It's this damn case.

 

Also hard drives make a fair bit of sound spinning anyway which is what they are doing when idling. 10TB drives must be much louder than average.

I think you should find why your case is vibration, a cause shouldn't do that. That isn't a hdd issue.

 

Maybe you should just get a nas, get the hdds away from the main pc.

[grangervoldemort]

7 minutes ago, Electronics Wizardy said:

I think you should find why your case is vibration, a cause shouldn't do that. That isn't a hdd issue.

 

Maybe you should just get a nas, get the hdds away from the main pc.

You are ignoring everything I wrote AFTER telling you my case vibrates.

 

''Also hard drives make a fair bit of sound spinning anyway which is what they are doing when idling. 10TB drives must be much louder than average.''

 

I have tried to find what vibrates for years. It is not something I am willing to try to fix anyway even if I did find the cause as it would be a DIY job requiring skill and knowledge. 

My case is old and the matte finish is very sticky now and horrible looking.

I am tired of this case. A new case will be refreshing to me aesthetically.

 

You aren't answering logically based on my first post.

[Electronics Wizardy]

2 minutes ago, grangervoldemort said:

You are ignoring everything I wrote AFTER telling you my case vibrates.

 

''Also hard drives make a fair bit of sound spinning anyway which is what they are doing when idling. 10TB drives must be much louder than average.''

 

I have tried to find what vibrates for years. It is not something I am willing to try to fix anyway even if I did find the cause as it would be a DIY job requiring skill and knowledge. 

My case is old and the matte finish is very sticky now and horrible looking.

I am tired of this case. A new case will be refreshing to me aesthetically.

 

You aren't answering logically based on my first post.

I did suggest a nas, that seems like the best option here. Its moves the hdds somewhere else, getting rid of any noise.

 

Whats your backup plan? How are you backing up this array?

 

 

 

[grangervoldemort]

1 minute ago, Electronics Wizardy said:

I did suggest a nas, that seems like the best option here. Its moves the hdds somewhere else, getting rid of any noise.

 

Whats your backup plan? How are you backing up this array?

Clearly you have not read my first post. As I suspected.

[Electronics Wizardy]

1 minute ago, grangervoldemort said:

Clearly you have not read my first post. As I suspected.

I did read your post.

 

1 hour ago, grangervoldemort said:

NVME have lower lifespans than SSD'

NVMe is a protocol for ssds, doesn't affect life at all

 

 

 

Here is what id do for your workflow.

 

Use a ssd for editing drive and copy the camera to here. Have this automatically backed up to the nas.

 

Then copy projects to the nas when your done with them.

 

Have the nas automatically backup to the cloud.

 

That would be much simple, require much less manual changes and plugging and unplugging of drive and copying.

[grangervoldemort]

5 minutes ago, Electronics Wizardy said:

Here is what id do for your workflow.

 

Use a ssd for editing drive and copy the camera to here. Have this automatically backed up to the nas.

 

Then copy projects to the nas when your done with them.

It is better to make to separate copies in case somehow the first copy is corrupted. If I follow your method, then the copied file to the SSD IF corrupted would also be copied over to the NAS.

 

Why would I need to copy projects to the NAS when done if it's all happening automagically as you wrote?

Quote

 

Have the nas automatically backup to the cloud.

 

That would be much simple, require much less manual changes and plugging and unplugging of drive and copying.

And who's gonna pay for cloud? You? I have 17Mbps upload as well. Not exactly fast.

 

For the backup I HAVE to connect and disconnect. The point of a backup is to be DISCONNECTED. Unless you know a way around this.

 

 

[Electronics Wizardy]

Just now, grangervoldemort said:

Why would I need to copy projects to the NAS when done if it's all happening automagically as you wrote?

Automatic backup, the copying would be when your doing with a project.

 

1 minute ago, grangervoldemort said:

It is better to make to separate copies in case somehow the first copy is corrupted. If I follow your method, then the copied file to the SSD IF corrupted would also be copied over to the NAS.

You can also do that, but I have imported many videos with very few issues. How often are files corrupted on import. If its common then id check for a bad card or reader. Corruption on import should be very rare.

 

1 minute ago, grangervoldemort said:

And who's gonna pay for cloud? You? I have 17Mbps upload as well. Not exactly fast.

Yea, gotta pay for backups, And the cloud is much better at keeping the data safe then your external hdds as its stored in many locations, and fault tolerant.

 

Upload speed should be fine. How much data are you making. You can let it run over night.

 

2 minutes ago, grangervoldemort said:

For the backup I HAVE to connect and disconnect. The point of a backup is to be DISCONNECTED. Unless you know a way around this.

Thats what the cloud does for your, snapshots and the data can't just be deleted.

 

[grangervoldemort]

2 minutes ago, Electronics Wizardy said:

Automatic backup, the copying would be when your doing with a project.

And how is this done?

2 minutes ago, Electronics Wizardy said:

Yea, gotta pay for backups, And the cloud is much better at keeping the data safe then your external hdds as its stored in many locations, and fault tolerant.

Cloud is much more expensive over time than WD Elements drives. Although the advantage is it's totally offsite. BUT in the event I need to get the data back - then what? 11TB of data would take FOREVER to download on a 68Mbps connection and I may not have that speed when I move from here (living with my parents). But the cost of cloud is insane.

 

20TB = £1,599.80 a year.

Read more: https://www.which.co.uk/reviews/cloud-storage/article/how-to-choose-the-best-cloud-storage-service - Which?

 

[Electronics Wizardy]

1 minute ago, grangervoldemort said:

And how is this done?

Lots of backup programs can do this for you. Id probalby just use something like the built in windows backup, or something like robocopy.

 

 

 

Yea cloud is expensive, but its really the best way to do with backups.

 

Id probably go b2 https://www.backblaze.com/b2/cloud-storage-pricing.html

 

For restores, most services will ship your drives with your data if something goes really bad.

 

If your not gonna do cloud, id just have the nas sync with a hdd, and rotate that hdd offsite every week or two.

[grangervoldemort]

1 minute ago, Electronics Wizardy said:

Yea cloud is expensive, but its really the best way to do with backups.

 

Id probably go b2 https://www.backblaze.com/b2/cloud-storage-pricing.html

 

For restores, most services will ship your drives with your data if something goes really bad.

Can't afford it. Backblaze non existent in the UK.

 

1 minute ago, Electronics Wizardy said:

If your not gonna do cloud, id just have the nas sync with a hdd, and rotate that hdd offsite every week or two.

Poor explanation. Explain again please. What do you mean have a NAS sync with a HDD? Where is that HDD? Rotate - wtf? What? What does this mean in this context?

[Electronics Wizardy]

Just now, grangervoldemort said:

Poor explanation. Explain again please. What do you mean have a NAS sync with a HDD? Where is that HDD? Rotate - wtf? What? What does this mean in this context?

You can setup the nas to sync  with a external hdd, so when ever a change is made on the nas, it will also update the backup hdd.

 

Then get a couple of backup hdds, and have one offsite, and one onsite. Then every few weeks, swap them, so the offsite one is now onsite and vice versa.

[grangervoldemort]

1 minute ago, Electronics Wizardy said:

You can setup the nas to sync  with a external hdd, so when ever a change is made on the nas, it will also update the backup hdd.

 

Then get a couple of backup hdds, and have one offsite, and one onsite. Then every few weeks, swap them, so the offsite one is now onsite and vice versa.

If the NAS gets infected so does the HDD. Good luck finding a HDD that is over 20TB to store all the data stored on the multiple drives in the NAS.

[Electronics Wizardy]

Just now, grangervoldemort said:

If the NAS gets infected so does the HDD. Good luck finding a HDD that is over 20TB to store all the data stored on the multiple drives in the NAS.

No it won't infect the hdd, as there is also one offite that will have its data restored if there is an issue.

 

You can use multiple drives, or get an enclosure with 2 drives in it.

[grangervoldemort]

Just now, Electronics Wizardy said:

No it won't infect the hdd, as there is also one offite that will have its data restored if there is an issue.

Explain. 

Just now, Electronics Wizardy said:

You can use multiple drives, or get an enclosure with 2 drives in it.

Sounds very expensive.

[Electronics Wizardy]

2 minutes ago, grangervoldemort said:

Sounds very expensive.

It would be the same cost as having the hdds for backups. onsite, so really not that much.

 

But storing data safely is expensive. So if you want your data to be safe, you gotta spend money. There isn't a cheap way to store a ton of data safely. Can you compress the data? Do you need it all.

 

2 minutes ago, grangervoldemort said:

Explain. 

I said earlier, but one hdd would be onsite, and one offsite.

5 minutes ago, Electronics Wizardy said:

Then get a couple of backup hdds, and have one offsite, and one onsite. Then every few weeks, swap them, so the offsite one is now onsite and vice versa.

 

[grangervoldemort]

1 hour ago, comander said:

NAS. Physically away from your work area. You probably do NOT need 10Gbe if you're using hard drives

Consider more drives in RAID1, RAID5 or RAID10. RAID5 with 3 drives is likely the right choice. 

Also, consider just using Windows file history for automatic backups OR just map to the NAS directly. 

My work area is a tiny bedroom. So.... I will not be able to get the RAID away from me. At most 1 meter away in direct line of sight.

 

RAID is expensive. The enclosure and the additional drives that are used for RAID and are not available for storage. Considering I will have a backup, do I really need RAID?

 

Please take a look at my first post and reply carefully please. Thank you.