Storage solution for Video editing and archiving footage

[mejeffreyberg]

I'm looking for a solution for archiving footage and fast NAS storage for local network editing. But.. I'm getting a bit stuck in my configuration and OS selection.

 

My current idea;

• Dell T640 (Xeon Zilver processor, 32Gb RAM, Dual 10Gb LAN, 16 3.5' bays, IDRAC9 Enterprise, SSD) NAS SERVER
• Dell T640 (Xeon Zilver processor, 16Gb RAM, Dual 10Gb LAN, 16 3.5' bays, IDRAC9 Enterprise, SSD) BACKUP SERVER
• Cisco 2F10 switch (10 Gbit Switch)
• HighPoint SSD7120 NVMe RAID Controller (Extension for the NAS SERVER, I want to use this for caching read and write, not sure if this will work)
• WD RED PRO 6 TB Drives (I want to put this inside a RAID 6 configuration)

 

My current questions;

• Will the Dell T640 be able to handle good performances compared with QNAP TVS-x82T?

• What kind of OS should I use for scaling my server, high performances, and redundancy? (I was looking at FreeNAS)

• Will an NVMe RAID Controller with Samsung Evo 970's increase performance on Write as a Cache drive?

 

I also heard that FreeNAS uses software RAID. Does this mean it won't use the Hardware RAID Controller inside the T640? I heard DELL saying that their RAID controller is extremely powerful.

 

Also, I won't start with this machine fully packed with drives. I think I will start with 6 drives each machine (Little bit cutting the costs). But will it use the full 10 Gbit network?

 

And finally.. my last question; When copying files from A to B on the same NAS itself, will my computer 'Download' and 'Upload' those files to the NAS or is the infrastructure intelligence enough to find the shortest way and perform this task locally on the NAS?

 

I really appreciate any help you can provide.

[Electronics Wizardy]

24 minutes ago, mejeffreyberg said:

And finally.. my last question; When copying files from A to B on the same NAS itself, will my computer 'Download' and 'Upload' those files to the NAS or is the infrastructure intelligence enough to find the shortest way and perform this task locally on the NAS?

Depends on if you copy and paste or move, you can have it move instantly

 

26 minutes ago, mejeffreyberg said:

What kind of OS should I use for scaling my server, high performances, and redundancy? (I was looking at FreeNAS)

Well freenas doesn't support clustering.

 

Id probably go with something like centos or ubuntu server on these boxes, and have gluster or ceph manage the distrusted storage. You can also do storage spaces direct if you want windows. You won't want freenas here.

 

 

27 minutes ago, mejeffreyberg said:

I also heard that FreeNAS uses software RAID. Does this mean it won't use the Hardware RAID Controller inside the T640? I heard DELL saying that their RAID controller is extremely powerf

Freenas, and the other solutions above all use software raid, so you want to get a hba for the server, not the raid card.

 

28 minutes ago, mejeffreyberg said:

HighPoint SSD7120 NVMe RAID Controller (Extension for the NAS SERVER, I want to use this for caching read and write, not sure if this will work)

You can't easily cache the server this way on the client. If you want ssd caching, do it on the server side, not the client.

 

29 minutes ago, mejeffreyberg said:

WD RED PRO 6 TB Drives (I want to put this inside a RAID 6 configuration)

Why red Pros? If you want cheap, get those 8tb externals. If you want storage Id get 8tb reds, abou the same speed, and cheaper. You want more drives for more speeed.

 

29 minutes ago, mejeffreyberg said:

Cisco 2F10 switch (10 Gbit Switch)

Why that switch? Id personally get one with fiber due to the lower power and latency, but this should work fine.

 

 

This seems like a odd hardware choice your going towards. How many users do you have? If this is just you, id just do the storage locally, and just use the nas for backups.

 

 

 

Can you use used equipment? You can get the hardware much cheaper this way.

 

[mejeffreyberg]

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

And finally.. my last question; When copying files from A to B on the same NAS itself, will my computer 'Download' and 'Upload' those files to the NAS or is the infrastructure intelligence enough to find the shortest way and perform this task locally on the NAS?

Depends on if you copy and paste or move, you can have it move instantly

When I use copy and paste on the client side, will this move de data instantly?

 

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

What kind of OS should I use for scaling my server, high performances, and redundancy? (I was looking at FreeNAS)

Well freenas doesn't support clustering.

 

Id probably go with something like centos or ubuntu server on these boxes, and have gluster or ceph manage the distrusted storage. You can also do storage spaces direct if you want windows. You won't want freenas here.

I looked for Ceph before with CROIT or Ceph manager Dashboard. But I think installing will be very difficult. I also hear people saying that Ceph isn't good for high performances. They say that the speed is kinda low. 

 

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

I also heard that FreeNAS uses software RAID. Does this mean it won't use the Hardware RAID Controller inside the T640? I heard DELL saying that their RAID controller is extremely powerf

Freenas, and the other solutions above all use software raid, so you want to get a hba for the server, not the raid card.

HBA also a good choice for Ceph? So the RAID Controller won't be effective on this System? And the HBA will even have better Performance with Redundancy?

 

 

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

HighPoint SSD7120 NVMe RAID Controller (Extension for the NAS SERVER, I want to use this for caching read and write, not sure if this will work)

You can't easily cache the server this way on the client. If you want ssd caching, do it on the server side, not the client.

 

I will use the HighPoint SSD7120 NVMe RAID Controller for Caching on my server. I want it for very fast Write speed.

 

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

WD RED PRO 6 TB Drives (I want to put this inside a RAID 6 configuration)

Why red Pros? If you want cheap, get those 8tb externals. If you want storage Id get 8tb reds, abou the same speed, and cheaper. You want more drives for more speeed.

I want good performance hard drives for good value, so you recommend using the non-pro series? 

 

15 hours ago, Electronics Wizardy said:

16 hours ago, mejeffreyberg said:

Cisco 2F10 switch (10 Gbit Switch)

Why that switch? Id personally get one with fiber due to the lower power and latency, but this should work fine.

1

I'm now looking at Dell EMC Nerworking N1524

 

We now have 4 users and counting.

 

I think this entire configuration will fit in my budget, I just want performance. And if the price is 10k it is what it is.

 

[Electronics Wizardy]

39 minutes ago, mejeffreyberg said:

When I use copy and paste on the client side, will this move de data instantly?

 

I looked for Ceph before with CROIT or Ceph manager Dashboard. But I think installing will be very difficult. I also hear people saying that Ceph isn't good for high performances. They say that the speed is kinda low. 

 

HBA also a good choice for Ceph? So the RAID Controller won't be effective on this System? And the HBA will even have better Performance with Redundancy?

 

 

I will use the HighPoint SSD7120 NVMe RAID Controller for Caching on my server. I want it for very fast Write speed.

 

I want good performance hard drives for good value, so you recommend using the non-pro series? 

 

I'm now looking at Dell EMC Nerworking N1524

 

We now have 4 users and counting.

 

I think this entire configuration will fit in my budget, I just want performance. And if the price is 10k it is what it is.

 

If your goal is performance id get the main server with ssds only. Get some cheaper ssds like mx500s or 860evos. These will be much faster than any hdd raid system, esp since you have fast networking.

Id get a dell t440 with 48(or more)gb of ram, xeon 4112, dual 10gbe sfp+ nics, and the 16 2.5 bays. Then Id put a linux distro on it like ubuntu or centos, but something like freenas or windows server will also work. Id make a raid 10 for the ssd to get the max speed from them.

 

Id get something like this for the switch unless you need cisco https://www.newegg.com/Product/Product.aspx?Item=14P-000G-00120&Description=10gbe switch&cm_re=10gbe_switch-_-14P-000G-00120-_-Product

 

Id bond the 2x gbe from the server to the switch and that should be fine.

 

Then for backup, get something like the dell t340, fill it with 8+tb hdds, and have a program like rsync or lsyncd doing a realtime oneway sync from the main server. Then you can switch to the backup if needed, but this is much easier to setup than a cluster.