Adding a NAS for Video Editing Office

[voxeditor]

Budget: $1500

Country: USA

Programs: Final Cut Pro, Hedge (Media offload checksum software)

Other details: Current equipment includes: 2019 i9 8-core iMac with Radeon Pro 580X 8GB GPU and 32GB 2667MHz DDR4 RAM, a G-Technology RAID Studio with populated with two 4TB drives in RAID 1 configuration and a Linksys MR8300. I mainly shoot two or three camera setups in 4K in ProRes. On average, I accumulate around 200GB of data per shoot. Generally its interviews and multicam live music performance. I will not be upgrading past 4k resolution and 10-bit color media. So my data amounts shouldn't change all that much over the next 10 years.

I need a storage solution for my at home office where I edit freelance video projects. I do not plan to edit from the NAS, it will primarily be a backup of media and a backup of our household computers. I've narrowed down my NAS to a 5-bay Synology DS1520+ NAS populated with 5 6TB WD Red HDD configured in RAID 6. That gives me a total useable capacity of 18TB. I chose RAID 6 as I want as much capacity as possible, but with one more layer of data security over RAID 5. The reason I'm leaning towards a 5 bay over a 4 or even a 6 bay from Synology is because I get the highest possible useable capacity (considering my RAID 6 configuration) for the least amount of money. The NAS will be connected to my home network via two 1GbE for failover support, that will be used for wireless backups of two MacBook Airs. My iMac will be connected directly to my network via 1GbE and that's how I'll connect to the NAS.

 

Media Workflow:

1. Connect camera media (SSD) to iMac and copy media via Hedge software to the NAS and DAS RAID simultaneously.

2. Edit video project located on RAID with automatic backup of Final Cut Pro project file to NAS throughout the day. 

3. After project completion, create a backup of media and project files to an external USB HDD stored in separate offsite location. Empty the DAS RAID for next project. Final storage locations: NAS and external USB HDD. (I know adding a cloud service is ideal, but this is more of a favor to my clients as they also receive a copy of the footage that they are responsible for.)

 

Questions:

1. Does all of the above sound reasonable? Would you recommend any different connectivity options? 

2. Can you convince me that RAID 5 is more than adequate given my use case? If I were to go RAID 5, I'd likely purchase a 4-bay NAS with 6TB drives for a total of 18TB useable space and save the money for when I added expansion bays.

3. Are there any other offerings from other manufacturers that I should consider? Western digital seems overpriced, though redundant power would be amazing. QNAP seems reasonably priced but I've had a 2-bay NAS from them that failed right after the 2 year mark (RAID controller died) so I'm hesitant to use them. Also, their 2.5GbE offering would be wasted on my intended setup, unless I bought an additional adapter for my iMac which means the cost just increased. At the end of the day, the only time I'd see any bottleneck is during media copying, and with the Hedge software, I'd be able to use the simultaneously transfer to the slower NAS while still utilizing the full speed of the DAS RAID.

 

LINKS:

Synology DS1520+ 5-Bay NAS My top pick.

Synology DS920+ 4-Bay NAS My pick if you can convince me RAID 5 is fine.

Synology DS1621+ 6-Bay NAS Just for comparison.

QNAP TS-451D2 NAS 

QNAP TS-653D NAS

 

Thanks in advance for your feedback. This has been a long time coming. I've gone back and forth between DAS and NAS and even building my own system. But I think I'm in the final stretch. I'm just the kind of buyer who hates to spend money, but usually loves the outcome because I've spent so much time researching it. I just want to make sure I'm not making any foolish mistakes and forgetting some detail that will make a big impact.

[Electronics Wizardy]

6 minutes ago, voxeditor said:

6TB WD Red HDD configured in RAID 6.

Don't use wd red drives in a nas as there smr, you want red plus or ironwolf here.

 

7 minutes ago, voxeditor said:

1. Does all of the above sound reasonable? Would you recommend any different connectivity options? 

 

yea no real issues. I have a few synology nas boxes ive used for clients and they generally work well, and support is fairly good.

 

8 minutes ago, voxeditor said:

. Can you convince me that RAID 5 is more than adequate given my use case? If I were to go RAID 5, I'd likely purchase a 4-bay NAS with 6TB drives for a total of 18TB useable space and save the money for when I added expansion bays.

Id probalby go raid 5 here with 4 hdds, then get the 5 bay nas so you can expand later on if needed. Id also be tempted to start out with something like 2x16tb drives, and add the other 3 as needed, as I feel like your gonna chew through 18tb faster than you think.

 

Also since you  have a offsite backup as well, so the nas is never the only copy the slight improvement of raid6 isn't really needed here

 

10 minutes ago, voxeditor said:

Western digital seems overpriced, though redundant power would be amazing.

WD seems to not have great software reputation in the pas so Id avoid them. The dual psu really won't matter here, psu failure is pretty rare these days. And most nas and computers are pretty good with a unexpected power outage too.

 

13 minutes ago, voxeditor said:

Synology DS1520+ 5-Bay NAS My top pick.

Synology DS920+ 4-Bay NAS My pick if you can convince me RAID 5 is fine.

Synology DS1621+ 6-Bay NAS Just for comparison.

If these are the only options, Id go 6 bay, it seems to be only like 30 bucks more here. But Id personally get the 5 or 6 bay, and get 16tb drives, start with 2 and add more as needed.

[voxeditor]

This is tremendous feedback; exactly what I had hoped for. Thank you so much.
 

20 minutes ago, Electronics Wizardy said:

Id probalby go raid 5 here with 4 hdds, then get the 5 bay nas so you can expand later on if needed. Id also be tempted to start out with something like 2x16tb drives, and add the other 3 as needed, as I feel like your gonna chew through 18tb faster than you think.

 

Also since you  have a offsite backup as well, so the nas is never the only copy the slight improvement of raid6 isn't really needed here

If I'm understanding correctly, I can start with a partially populated NAS and then add drives (so long as they match in capacity) later on, thus expanding the total capacity with every additional drive. I didn't know that. In my mind, I think it would make the most sense to start with three 10TB hdds configured in RAID 5 and a 5-bay NAS. That seems like a reasonable balance of cost and capacity to start with and it is at the upper limit of my budget. I didn't mention it, but I was trying to leave room in the budget to buy at least one spare drive to have on hand for a rebuild if needed.

In the case of your 2x16TB setup, what would the RAID config be? I think the Synology would only be able to use them in RAID 0 or 1 which, but I could be wrong. RAID 1 would be fine except that it costs more due to the capacity limitations, and that is my primary concern.
 

 

40 minutes ago, Electronics Wizardy said:

54 minutes ago, voxeditor said:

Synology DS1520+ 5-Bay NAS My top pick.

Synology DS920+ 4-Bay NAS My pick if you can convince me RAID 5 is fine.

Synology DS1621+ 6-Bay NAS Just for comparison.

If these are the only options, Id go 6 bay, it seems to be only like 30 bucks more here. But Id personally get the 5 or 6 bay, and get 16tb drives, start with 2 and add more as needed.

Finally, your point about the prebuilt 6-bay is a good one. It also allows for a 10GbE card, so if in the future I wanted to edit directly from the NAS, that's an option. Plus, I've got two spare 4TB ironwolf drives for my DAS, so I'd be able to utilize those for rebuilds which is a nice financial incentive. 

 

Thanks again for your feedback. It's given me more to consider and I learned something new, so that's a win!

 

PS.

42 minutes ago, Electronics Wizardy said:

56 minutes ago, voxeditor said:

Western digital seems overpriced, though redundant power would be amazing.

WD seems to not have great software reputation in the pas so Id avoid them. The dual psu really won't matter here, psu failure is pretty rare these days. And most nas and computers are pretty good with a unexpected power outage too.

Fair enough. I will have the NAS on a UPS with a surge protector, along with the modem and router. My iMac, DAS RAID and powered speakers run off a separate UPS and surge protector too. Overkill? Absolutely.

[gloeckle]

I pretty much double all of the WIZARDRY above!
Had QNAP for 4 years with 2 outages caused by QNAP updates but good support

Had Synology for 6 years with no outages and outstanding support. Also since DSM7 they did a really good job cleaning the OS. It is really good to see that some of our money actually goes into improving already sold products!

About HDDs smr etc.... what the wizard said!

I would add a cheaper alternative: Toshiba Enterprise Capacity MG09ACA is a good deal.

 

With your budget I also would look into refurbished servergrade hardware like "HP DL380 gen9 12x LFF" + unraid for example.

Which gives you all possible scenarios and freedom - in exchange with your time and will to get into it of course

[voxeditor]

That's a great to hear about Synology improving the OS for users who have already made a purchase. I like that.

So with HDDs, we want to use PMR instead of SMR? I did leave out that the Synology comes with Red Plus drives. But Ironwolf is also an option. And as I already have to spare ironwolfs, I'll likely go that route.

5 minutes ago, gloeckle said:

I pretty much double all of the WIZARDRY above!
Had QNAP for 4 years with 2 outages caused by QNAP updates but good support

Had Synology for 6 years with no outages and outstanding support. Also since DSM7 they did a really good job cleaning the OS. It is really good to see that some of our money actually goes into improving already sold products!

About HDDs smr etc.... what the wizard said!

I would add a cheaper alternative: Toshiba Enterprise Capacity MG09ACA is a good deal.

 

With your budget I also would look into refurbished servergrade hardware like "HP DL380 gen9 12x LFF" + unraid for example.

Which gives you all possible scenarios and freedom - in exchange with your time and will to get into it of course

Just in general, your raise a good point about refurbished equipment. For some reason that never came to mind. I'll do some digging for through the used market. I think I'll avoid the more complicated scenarios though. I first learned about unraid from LTT. I spent a solid week researching it and determined I didn't have the ability to reliably operate it. That's why it's so great that we have options, right?
 

[gloeckle]

Spoiler

Common recording methods for hard disks (CMR and SMR)

The basic recording of hard disks takes place in such a way that a write head writes the data to platter – there it is stored. A distinction is made between two common recording methods: CMR and SMR. In general, one speaks of CMR, i.e. conventional magnetic recording, when hard drives with pure PMR (perpendicular magnetic recording) technology are used.

SMR, or Shingled Magnetic Recording, is the second recording method. PMR is also used in this method – similarities and differences to CMR are discussed in more detail below in the text.

 

CMR

PMR, or Perpendicular Magnetic Recording, is a method in which magnetization takes place horizontally and vertically on platters. PMR is the successor to Longitudinal Magnetic Recording (LMR) and achieves a data density three times higher than its predecessor LMR – the read / write head has been optimized for this and the vertical positioning also contributes to this. This procedure was developed in order to be able to provide more than 750 GB with the common 3.5 inch hard drives. With PMR it is possible to write directly to the final storage location of the data on the hard disk without the data first having to be rewritten on the hard disk.

 

SMR

SMR, the Shingled Magnetic Recording, writes data overlapping, because it is based on the concept that the write head is larger and less filigree than the read head. Therefore, the write head writes very close to the previous web, so that it remains readable. However, the data of the following lane are overwritten, which means that the next lane may have to be written again after a successful recording process. If things don’t go optimally, this “rewriting” has to continue. This may slow down the writing process drastically. The advantage of this technology, however, is that the storage density is increased compared to PMR.

In addition to the standard controller cache, SMR hard disks also have an on-disk cache in the outer tracks. This is where PMR technology comes into play. After the recording, the data is transferred from the controller to the SMR area as soon as possible. On the one hand, this process is more space-saving in the end, but it also shows the big difference to CMR technology: If the faster on-disk cache is filled before the controller can transfer the data to the SMR area, then the cache must first emptied or rewritten before new data can be saved again. For this reason, with large files it can happen that the transfer rate drops after a certain amount of data because the cache is full and must first be cleared again.

When does SMR make sense, when does CMR?

SMR hard disks are a good choice if they are primarily used as pure data storage or if a large hard disk is to be used for a PC on which data is stored. They offer more storage capacity and are more energy efficient than CMR, which makes them ideal for archiving tasks. Basically, SMR hard disks are not particularly well suited if the hard disk is to be constantly and permanently doing write operations, as this can lead to a cache overflow. In this case, a hard disk with a CMR recording method should definitely be used.

CMR hard disks are a good choice when data is to be stored at high transfer rates or particularly large amounts of data. This includes a large number of activities from music streaming and image processing to use for NAS servers.

source and even more info: https://www.reichelt.com/magazin/en/guide/smr-cmr-which-hard-drive-is-best-for-which-purpose/

short: only use CMR and don't mix in the same chassy!

 

If you are coming back to refurbished servergrade HW and you are living in the EU, have a look here:
https://www.serverschmiede.com/

Nice support, 1 year warranty, fair prices, good build quality (they assemble and update everything you configure)

 

No time but got more money? I would go with Synology. Because of already mentioned reasons and a lot of community apps to fulfil most needs. And you can find easy tutorials on pretty much anything around it.

Oh, one more experience that I made: WD NAS My Cloud

I configured one of those abominations at a friends place - Jesus Christ! What a horrible Product! Please do not even think about it! *uses-MIB-stick

Another memory just popped xD I had a cheap Zyxel. Works, cheap, is ok. Nearly no apps or extra features. This might have changed the past few years - but for those prices i doubt it. And would I trust it past 2 years? meh....

[brob]

1 hour ago, voxeditor said:

Can you convince me that RAID 5 is more than adequate given my use case? If I were to go RAID 5, I'd likely purchase a 4-bay NAS with 6TB drives for a total of 18TB useable space and save the money for when I added expansion bays.

 

I can convince you that RAID 5 is not a good solution and RAID 6 is not much better. It's well explained in  https://www.zdnet.com/article/raidfail-dont-use-raid-5-on-small-arrays/.

 

RAID is not backup and counting on a single array as backup is less than ideal. But if you are going to do it, use a scheme like RAID 1+0. Sure it requires more HDD, but calculate the cost of losing a RAID 5/6 array vs a few hundred for some additional mirror drives.

 

[voxeditor]

42 minutes ago, gloeckle said:

Oh, one more experience that I made: WD NAS My Cloud

I configured one of those abominations at a friends place - Jesus Christ! What a horrible Product! Please do not even think about it! *uses-MIB-stick

Another memory just popped xD I had a cheap Zyxel. Works, cheap, is ok. Nearly no apps or extra features. This might have changed the past few years - but for those prices i doubt it. And would I trust it past 2 years? meh....

Haha, noted. Yeah, I think I have to go with Synology at this stage. I'm also trying to get this purchased in the next few days as I live in a state without sales tax, and I move to a state with sales tax next week! Plus, it would be nice to have all of my data backed up on three drives for a cross country move. (One copy is staying at my parents')

[voxeditor]

36 minutes ago, brob said:

I can convince you that RAID 5 is not a good solution and RAID 6 is not much better. It's well explained in  https://www.zdnet.com/article/raidfail-dont-use-raid-5-on-small-arrays/.

I'm glad you brought this up. I just was reading up on this. I'm surprised that I hadn't heard anything about it until today. It's odd that companies like Lacie and G-Technology would sell products configured by default in RAID 5 considering the "risk", but I guess they just want to show off their high speed read/writes. That being said, I do have backups, as mentioned above, so in the event of a total failure I can rebuild from the backups. It would just be a huge pain in the butt. RAID 10 is looking more and more appealing just because I want the least amount of headache when it comes to rebuilding. In my use case, I'm purely focused on capacity and redundancy. RAID 10 is now likely my preferred configuration.

I do think that this particular article is a bit too doomsday-esque. Here's an interesting read as a follow up: http://www.raidtips.com/raid5-ure.aspx 
 

[gloeckle]

I do not share this opinion about raid5 (or any raid5 based manufacturer wizardry).

I mean can it happen that .... ? Yes. Can I be the one with this error? Yes.

Can a meteor hit you next Tuesday? Yes.

How likely is it? Very very unlikely. I mean read errors do not occur out of the blue.

 

Assumption:

Even as a poweruser / small business, how likely is it that Synology DSM7 SMART tests / HDD monitoring doesn't let you know that HDDx is starting to fail. Read error count goes up or reallocated sectors rise. The moment this occurs you know it. Do you see it 1 day, even 1 week later, nothing has changed. the Software of your NAS or even the HDD itself has rebuild / reallocated that data already and will tell you about it. So you can have a look and take measures / replace an HDD which is counting upwards in failures.

 

A RAID5(+) with just 1 single spare will last a very long time, Given no outside force causes it to fail like a fire or hammer

And concerned users do an offsite backup anyway...

 

When does a RAID5(+) actually fail from the inside?

Again considering a RAID5 config with just 1 spare HDD and lets say 4 usable HDDs:

The worst case is 1 HDD goes unrecoverably offline >> the software starts the rebuild on the hotspare >> during this rebuild a second drive goes unrecoverably offline = DATA loss

The odds that this will happen, without any external force is extremely unlikely. Can it happen? Yes but again: offsite backup.

 

I know someone who knows some who .... works in datacentres since 9 years and that guy has never seen HDDs fail, in the same machine, just by themselves in such short time periods. Exceptions: faulty hardware around it like cables / backplanes / storage controller, faulty batch of drives or again: outside force like fire or playing football with it.

But he has regularly NICs or onboard LAN ports with the exact same MAC addresses!

I admit, we are talking enterprise grade HW and very good conditions.

 

Of course, the other options are valid too. But IMHO not worth the space and money. Jesus... I did not stop typing did I?

[voxeditor]

I'll take the reassurance
I've had one HDD fail since I began computing, and had one drive that was DOA. Considering the literal hundreds of drives that I've used over the years with different companies I've worked for, it's very reassuring.

 

Alright, so I think this is my final conclusion:

I will get a Synology DS1520+ 5-bay NAS. I'll populate it with three 10TB IronWolf HDDs and then add drives as I need them. I'll also purchase a fourth 10TB to have as a hot spare. 

 

The final question is: what RAID configuration? I've now read up on Synology's SHR config, and that seems very useful. Especially considering I have two spare 4TB drives that I can use in conjunction with the 10TB drives. Otherwise, those 4TB are effectively useless to me. Is there any downside to utilizing those 4TB drives in SHR config? When I do eventually need more space, am I going to have to rebuild my RAID since I'd be pulling one of the 4TB and replacing with a 10TB? I assume the answer is yes, in which case I'm not willing to go through the hassle. I'd rather just lose the money on my spare drives.

[brob]

1 hour ago, gloeckle said:

How likely is it? Very very unlikely. I mean read errors do not occur out of the blue.

 

 

Do the math. Even with enterprise hardware the probability is higher than you might think. https://www.ibm.com/support/pages/re-evaluating-raid-5-and-raid-6-slower-larger-drives

 

Presumably a NAS box can do rebuilds faster than Intel motherboard RAID, but with 6TB drives rebuilds will be measured in days, not minutes or hours.

 

It's just a matter of knowing the risks so one can make informed decisions.

 

I'm risk adverse, so I would likely go with 4 larger drives and RAID 10. But it is a more expensive option.

[gloeckle]

6 hours ago, voxeditor said:

I'll take the reassurance

 

The final question is: what RAID configuration? I've now read up on Synology's SHR config

I am a bit confused.... You wrote that while I wrote my post? Synologys SHR is a manufactured altered RAID5+
My Opinion to that see above.

5 hours ago, brob said:

 

Do the math. Even with enterprise hardware the probability is higher than you might think. https://www.ibm.com/support/pages/re-evaluating-raid-5-and-raid-6-slower-larger-drives

Intertesting short readup. TY! What he describes as RAID-6 is basically what I mean when I say RAID5+ or manufacturer altered RAID5. The terms are getting out of hand above RAID5 because a lot of manufacturers call it differently in their implementations. Synologys SHR for example. Sure, I did not read the a 30 pager of synology and compare it to IBMs raid6 (I guess 250 pager) but I think it is close enough for us bare mortals

 

5 hours ago, brob said:

It's just a matter of knowing the risks so one can make informed decisions.

Very true my friend.