Fast and heavy storage for post production workflow

[zobos]

Hey guys,

 

I'm a video editor and recently started working on highend cameras getting many 4k 4:4:4 files (bit more than a gig per second). Also, one computer don't have the cpu power to treat 4k 4:4:4 files so I have to resample medias for this station.

What i'm getting at is that I'm running out of space and bandwidth given that I'm currently storing everything on external usb and firewire harddrives.

I think I need some sort of nas or server to hold HDDs and SSDs to feed the workstations I currently have and that could withstand some more workstation in the future, but the options are endless and I don't know the first thing on networks. 

 

Do you have any idea ?

 

 

[vanished]

I don't know what kind of budget you have, but I'm envisioning some kind of 24 SSD RAID server with multiple aggregated 10 Gbit links for multi-gigabyte per second transfer speeds

[zobos]

Haha that would be nice but I'm aiming under 1000$

[leadeater]

25 minutes ago, zobos said:

Hey guys,

 

I'm a video editor and recently started working on highend cameras getting many 4k 4:4:4 files (bit more than a gig per second). Also, one computer don't have the cpu power to treat 4k 4:4:4 files so I have to resample medias for this station.

What i'm getting at is that I'm running out of space and bandwidth given that I'm currently storing everything on external usb and firewire harddrives.

I think I need some sort of nas or server to hold HDDs and SSDs to feed the workstations I currently have and that could withstand some more workstation in the future, but the options are endless and I don't know the first thing on networks. 

 

Do you have any idea ?

 

 

A freeNAS server or Windows Storage Spaces will do what you need with SSD caching. 2 TB of SSD cache should suffice and then the rest can be HDDs.

 

10Gb will most definitely be required but can be done reasonably cheaply if you are going to have 4 or less workstations, just directly connect them rather than buying a 10Gb switch.

[zobos]

4 Would be the limit for the space we have anyway. Do you mean directly connecting them to the freeNAS ?

[leadeater]

1 minute ago, zobos said:

4 Would be the limit for the space we have anyway. Do you mean directly connecting them to the freeNAS ?

Yea, put in the server 2 dual port and a single port card in the workstations. There are cheap ones to be found on ebay that are good so the total cost shouldn't be too bad, not inexpensive though.

 

You'll want to use 10GBase-T (RJ45) or Direct Attached Copper (DAC) not fibre. I use Intel X540's but they are on the expensive side.

 

http://www.ebay.com/itm/Lot-of-2-Mellanox-ConnectX-2-Single-Port-SFP-10GBE-Network-Card-MNPA19-XTR-/301689994552

[zobos]

Oh ok that's very helpful.

 

And in this server, would the SSD cache the HDD ? How do I setup this ?

[tt2468]

Get an r710 off ebay, pack it full of crucial 240gb ssds, and get a couple dual port 10gb nics. Like linus did, create a san for storage. Use samba for the smb server. (as a bonus, smb multichannel will be coming to samba 4.4 in march!)

[leadeater]

2 hours ago, zobos said:

Oh ok that's very helpful.

 

And in this server, would the SSD cache the HDD ? How do I setup this ?

FreeNAS will do this for you, plenty of good documentation on how to set it up. You can also go all SSD but if you need a lot of storage the price will get stupid quickly.

 

On my Windows Storage Spaces setup that is 1TB SSD cached, 16TB raw total, the Auto Tier report says that with my current usage only 100GB is required to have 100% I/O SSD cached. For you I would expect that much more than 100GB is required since I only lightly use this storage pool. I also have a 2TB SSD only pool for my steam library.

 

Edit: Also as @Kyle Manning suggested getting a server off ebay is a very good choice and the prices are good. I would get a server with 3.5" HDD bays not 2.5" or one that has a mixture.

[zobos]

Alright well that is plently of good informations. Thank you all, Ebay here I come