Need help to Build/Buy NAS

[Kuchma777]

Hi, 

i looked thought threads but did not find anything i am looking for.

 

My Company is looking to get a NAS/Backup solution a security system data storage.

 

Basically we need to store about  200 TB of security footage. So we need something robust as little as possible bugs, and least maintenance.  We have to have ability to have hot swappeble hard drives. Preferably directly in to the PC and not over network. 

 

And we need 5 of these budged is 10k per "NAS"  

[Jurrunio]

So it's 200TB storage per NAS? or 40TB per NAS?

[Kuchma777]

Its, 200 TB per NAS

 

[Windows7ge]

Storing this much data with reliability and dependability in mind I'd look into a Dell or HP server solution with hardware support. If anything stops working within warranty they'd send a technician out to you asap and fix it.

 

I like doing custom builds both PC and Server but you're out of the scope of what's reasonable. You'll want insurance with that much storage. There are server rack mount chassis. If you used 20 x 10TB disks then you'll need a 4U case with front removable drive bays.

[Kuchma777]

Well 4U case is ok . 
Is there any preferences over manufacturers?

Also any solutions that would record all of this not over network,  but rather something like USB or Thunderbolt ?

[DrunkenPanda]

Who decided on this arbitrary number of 200TB per NAS?

 

I ask because my entire movie collection on my home server of 650 low-compression 1080p movies takes up about 1.37TB. If you were to average out the movies at say... 1 hour 45 minutes a piece, then you could run a single camera for 47-1/2 days and only use 1.37 TB of space. I'd assume you'd be storing the footage in high-compression and without 5.1 dolby digital surround, so you'll cut that data ratio by 1/3 at least and still be in 1080p. So that runs a single 1080p camera for 142 days.

 

Now maybe you have 45 cameras running and you want to store the footage in 1080p completely uncompressed and with uncompressed sound, which probably would need 200TB for a month of storage. Most security footage is saved in SD without sound. Most places keep video surveillance for 30 days before rewriting over it. Unless you're business is a casino or a bank, there's no reason to use 1080p footage... and unless you're planning on being actively involved in a lawsuit against current employees, there's no real reason to keep footage past 30 days.

 

I'm not trying to belittle the use-case, but I feel like it needs to be addressed. By my math 200TB will store 6,934 days (about 19 years) of single camera 1080p movie quality, low-compression footage with 5.1 dolby digital sound (which is obviously overkill for security footage). If it WAS a single camera recording at that excessive quality (I get that it's probably not), the drives would start failing 12 years before the server was full. For the use case, we could assume high-compression storage without sound, and we'd be at ~57 years for that same single camera. Of course most places store security footage in SD, so that gives you another 4x multiplier to put that single camera @ 480p to 228 years of footage. *All of that is quickly approximated... but I think it illustrates the point.

 

I assume we'd be using 10TB drives... I'll also assume you want some kind of data failsafe, so if you want an active 200TB in RAID0, you'd need 40x 10TB drives to achieve this. Each drive is ~$400... that comes out to $16,040 just for the base of the drives, and you'll likely want 5-10 extras for hot-swapping and drive failure scenarios. So... you're walking into this at 60% over your budget before you've bought extra drives, 4U case, raid cards, ECC ram, server motherboard, CPU, power supply, cooling, etc.

 

In conclusion... I would reevaluate what you really need. After that, if it's actually determined you actually need THAT much hot-swappable redundant space, call Amazon and see what they charge.

[Kuchma777]

Look, you are more or less right about first paragraph, but i did not give any details and specifics for a reason. I have equipment that is writing almost 24/7 on alot of cameras. And you never know what environment i am in .

Besides i do not want to run a raid0

[DrunkenPanda]

I meant RAID1, single mirrored parity... anyways

 

Even without any redundancy, the drives alone are $8,020... 

 

I won't ask any questions I don't need an answer for to help you. (Though the secrecy of filming and storing THAT much footage is kind of creepy lol)

 

A. What file format/compression ratio are you storing the footage?

B. What resolution is the footage?

C. How many cameras?

D. How long do you need to store the footage for?

 

I ask because I don't think you budgeted enough for the system @ 10k per build, so if we can determine the data incoming and reevaluate what you really need to handle it, then maybe we can fit it into a 10k budget. RAID cards are expensive, ECC is expensive, 4U cases are expensive, server motherboards are expensive, and you'll need a decent CPU to process all the incoming data/fit the server mobo socket.

[unijab]

I would look into GlusterFS

[Lurick]

1 hour ago, DrunkenPanda said:

I meant RAID1, single mirrored parity... anyways

 

Even without any redundancy, the drives alone are $8,020... 

 

I won't ask any questions I don't need an answer for to help you. (Though the secrecy of filming and storing THAT much footage is kind of creepy lol)

 

A. What file format/compression ratio are you storing the footage?

B. What resolution is the footage?

C. How many cameras?

D. How long do you need to store the footage for?

 

I ask because I don't think you budgeted enough for the system @ 10k per build, so if we can determine the data incoming and reevaluate what you really need to handle it, then maybe we can fit it into a 10k budget. RAID cards are expensive, ECC is expensive, 4U cases are expensive, server motherboards are expensive, and you'll need a decent CPU to process all the incoming data/fit the server mobo socket.

Additionally, depending on the amount of data coming into the server, a budget for 10Gb NICs, although not terribly expensive, could need to be considered as well.

[GDRRiley]

a 45 drives q30 with 8tb drives would do well but we need more info. 

[Windows7ge]

1 hour ago, CUDA_Cores said:

A second good alternative would be supermicro. Supermicro uses an open platform style (so few things are completely proprietary) so all sorts of hardware can be installed with ease. Supermicro will cost more, but it sounds like it would be more appropriate for what OP is looking for. 

Some time ago I started a thread discussing the money that can be saved and the flexibility by building your own server. My intended audience was home users and small business who only needed a place to store some basic data...people didn't interpret my thread that way. Long story short it turned into a giant clusterf*** of everybody saying I was wrong. So...yeah, this guy. 5 servers, 200TB each, specifically for security footage...I wouldn't recommend building it themselves. Although a mod on the forum did open my eyes to the idea that there is a middle path. Buy a bare bones server from Dell or HP then install your own RAM, Drives, RAID controllers, etc. It would give you hardware support, allow you to use component brands you trust, and reduce the cost of the overall server.

 

Also I just realized 20x 10TB disks would result in one giant RAID0...he'll need more drives than that. I also wonder if he just needs 200TB worth of disks. Or 200TB of usable space because windows measures capacity differently. 1TB is equal to 931.(something)GB usable. If he needs 200TB usable he'll need even more drives.

[Kuchma777]

I appreciate your help. 
Everything is also driven by the cost,

why should we pay for this https://www.bhphotovideo.com/c/product/1232625-REG/digital_watchdog_dw_bjp2u24t_blackjack_p_rack_2u_8_bay.html   $6000.

If inside its Running Intel i7 4790, 16GB RAM, Supermicro X10SAE?

Sound to me like way overkill just because they are giving me a warranty.

We are  swapping drives every 3 month, and keeping them in the office.

By the specs that i see in this guys builds, i really dont need anything crazy.

 

So now my question is, in there a way to have 20 Disk, in the same machine without Raid, and redundancy. 

 

To sum up, i would just build a computer with the same spec put in the rack mounted case, and keep swamping HDD, that way we could cut cost in more then half , if above project doesnt work out.
 

About "creepy" part, for all it is , a residential apartment buildings with a lot of staff going on, the client wants 2 years of storage, just in case someone want to sue them because presumably something happens, that way  they could go back a check the footage. 
 

[GDRRiley]

get 45 drive machines they are cheap and good service 

[leadeater]

On 8/28/2017 at 4:04 PM, Windows7ge said:

Although a mod on the forum did open my eyes to the idea that there is a middle path. Buy a bare bones server from Dell or HP then install your own RAM, Drives, RAID controllers, etc.

[leadeater]

4 hours ago, Kuchma777 said:

I appreciate your help. 
Everything is also driven by the cost,

why should we pay for this https://www.bhphotovideo.com/c/product/1232625-REG/digital_watchdog_dw_bjp2u24t_blackjack_p_rack_2u_8_bay.html   $6000.

If inside its Running Intel i7 4790, 16GB RAM, Supermicro X10SAE?

Sound to me like way overkill just because they are giving me a warranty.

We are  swapping drives every 3 month, and keeping them in the office.

By the specs that i see in this guys builds, i really dont need anything crazy.

You'll need to think about an actual configuration that can support such a workflow as taking disks out of the system. You'll basically need the security camera software to support footage archival that also supports offline storage otherwise things will get tricky or even just not work, I see that you are already doing this so I'll assume you have that covered on the non hardware side of things.

 

However not knowing how you are doing it I can't say if this will also work for a bigger system since disks cannot just be removed and stored offline if they are in an array, all disks work together.

 

4 hours ago, Kuchma777 said:

So now my question is, in there a way to have 20 Disk, in the same machine without Raid, and redundancy. 

This follows on from above that I was going to say, this is mostly a software issue not a system/hardware issue. If the camera software you are using supports recording to multiple disks then potentially yes you might not have to use RAID or similar.

 

In a general sense if you need to do it cheaply with a low upfront cost I would configure each system with a dedicated short term footage array that can hold a month of footage then setup a footage archive process that moves this to a secondary disk or array if required due to size which you then remove from the system and put new disks in, repeat for 2 years.

 

My best advice is to actually consult with a security firm on this as they are experts. You don't have to buy anything off them software or hardware wise but you can pay them to come up with a design that will actually work and is likely something they have done before.

[Kuchma777]

23 hours ago, leadeater said:

You'll need to think about an actual configuration that can support such a workflow as taking disks out of the system. You'll basically need the security camera software to support footage archival that also supports offline storage otherwise things will get tricky or even just not work, I see that you are already doing this so I'll assume you have that covered on the non hardware side of things.

 

However not knowing how you are doing it I can't say if this will also work for a bigger system since disks cannot just be removed and stored offline if they are in an array, all disks work together.

 

This follows on from above that I was going to say, this is mostly a software issue not a system/hardware issue. If the camera software you are using supports recording to multiple disks then potentially yes you might not have to use RAID or similar.

 

In a general sense if you need to do it cheaply with a low upfront cost I would configure each system with a dedicated short term footage array that can hold a month of footage then setup a footage archive process that moves this to a secondary disk or array if required due to size which you then remove from the system and put new disks in, repeat for 2 years.

 

My best advice is to actually consult with a security firm on this as they are experts. You don't have to buy anything off them software or hardware wise but you can pay them to come up with a design that will actually work and is likely something they have done before.

The reason, we would like to have  all the 200TB on site, because it cost us money to come down every time.  

So my question about, how  do i connect 20 drives to a single machine,  assuming CPU,etc. And software is capable to deal with all the writhing.

[GDRRiley]

29 minutes ago, Kuchma777 said:

The reason, we would like to have  all the 200TB on site, because it cost us money to come down every time.  

So my question about, how  do i connect 20 drives to a single machine,  assuming CPU,etc. And software is capable to deal with all the writhing.

get compatible case and a HBA card, I would recommend 45 drives q30 

[leadeater]

9 hours ago, Kuchma777 said:

The reason, we would like to have  all the 200TB on site, because it cost us money to come down every time.  

So my question about, how  do i connect 20 drives to a single machine,  assuming CPU,etc. And software is capable to deal with all the writhing.

What OS is running the camera software? Windows?

 

Best option if that is the case is a good RAID card with BBU/write-back cache such as an LSI 9260/9266/9270/9271/9360/9361. You'll need either a case with a SAS expander backplane or a SAS expander card to multiply out the ports. For that many disks on a single RAID card this is a requirement.

 

You'll want to avoid most Windows software solutions for pooling disks for security camera recording since the risk of dropped frames is too high, while they do perform fine in most usages camera recording is rather sensitive to minor hiccups that would otherwise go unnoticed.

[Kuchma777]

On 9/17/2017 at 6:45 AM, leadeater said:

What OS is running the camera software? Windows?

 

Best option if that is the case is a good RAID card with BBU/write-back cache such as an LSI 9260/9266/9270/9271/9360/9361. You'll need either a case with a SAS expander backplane or a SAS expander card to multiply out the ports. For that many disks on a single RAID card this is a requirement.

 

You'll want to avoid most Windows software solutions for pooling disks for security camera recording since the risk of dropped frames is too high, while they do perform fine in most usages camera recording is rather sensitive to minor hiccups that would otherwise go unnoticed.

Thank you

 

[Dlog]

On 8/27/2017 at 12:51 AM, Kuchma777 said:

Hi, 

i looked thought threads but did not find anything i am looking for.

 

My Company is looking to get a NAS/Backup solution a security system data storage.

 

Basically we need to store about  200 TB of security footage. So we need something robust as little as possible bugs, and least maintenance.  We have to have ability to have hot swappeble hard drives. Preferably directly in to the PC and not over network. 

 

And we need 5 of these budged is 10k per "NAS"  

consider talking to a NetApp rep.

 

Side note I would be interested to see what NetApp data deduplication does with security footage. it's block by block so might end up saving you a ton of space. With static weekly server backups we see about 8:1 dedupe rate. 

[leadeater]

1 hour ago, Dlog said:

consider talking to a NetApp rep.

 

Side note I would be interested to see what NetApp data deduplication does with security footage. it's block by block so might end up saving you a ton of space. With static weekly server backups we see about 8:1 dedupe rate. 

It won't dedup all that well since the video is compressed already so the block matches will be very low. Unfortunately Netapp won't fit in the specified budget, might still be worth talking to them along with HPE though.

 

We put all our security footage on dedicated HPE DL380 servers even though we have a large amount of Netapp, we get very good HPE pricing and for the use case Netapp has no chance of being able to be price competitive. If other features were required like mirroring then they would be worth looking at.

 

Also 8:1 not bad, we get 95% dedup with Commvault for SQL backups with native compression off and 80% for NAS backups so slightly lower than what you are getting (88%). Are you using SnapVault?

[Dark]

It's difficult to give you proper recommendations when you start with:

"So we need something robust as little as possible bugs, and least maintenance"

 

And later state:

"So now my question is, in there a way to have 20 Disk, in the same machine without Raid, and redundancy. "

 

...

 

Cutting it at 10k per host is going to be a slight challenge to do right.

 

For example, 20x 10TB Ironwolf drives are ~$7000

 

A Supermicro SuperStorage Server 6048R-E1CR24L (24x 3.5 bays) is roughly $4k with the lowest options.

 

You could build up a 24 bay server if you wanted to do all the leg work: https://www.amazon.com/NORCO-Mount-Hot-Swappable-Server-RPC-4224/dp/B00BQY3916

 

An LSI Megaraid such as this may help get the storage bit taken care of: https://www.amazon.com/LSI-Logic-MegaRAID-9280-24i4e-Controller/dp/B003VP2EKK

 

You may very likely be able to do it for less, depending on which corners you want to cut (like getting a cheaper raid controller and a cheap sas expander).

[Dark]

It may be worth your time to reach out to a vendor or manufacturer in search of a discount, knowing that you'll need to purchase 1PB worth of hard drives.

[GDRRiley]

2 hours ago, Dark said:

For example, 20x 10TB Ironwolf drives are ~$7000

 

A Supermicro SuperStorage Server 6048R-E1CR24L (24x 3.5 bays) is roughly $4k with the lowest options.

that leaves you with 3k enough for 45drives q30

I agree, you should want something reliable and then not run raid. 15K each with 300TB pre raid is more realistic.