High Computing Server and Storage Build Help needed

[sixjames1000]

Hi all,

 

We are planning on building up a small server + SAN setup that allows our small company to work. We produce video and 3d content. We are looking to:

 

1- Have a centralized SAN for storage (looking at around 60tb of space). Need to be able to grow this to 200tb eventually. We  are thinking on using Synology's rack mounted san storage with ironwolf disks.

2- Have a small server, that works as a file server to feed 10 workstations connected to a switcher. We are looking at a 32 cores, 64gb ram server with an internal 500mb ssd.
3- The same server does application serving (ie: Maya, Resolve). 

4- This same server does work as a license server for the applications aforementioned. 

5- Eventually we want to add a few more compute-centric  nodes (probably Epyc based) for rendering. 

 

Could you helps us with a mockup for the main items needed for a build like this? Like what are the components for such a workflow and what would you generally pick?  It would be super helpful. 

 

We want to ensure a fast transfer back and forth between the workstations and the server/san (probably around 300mb/s would be ideal).

 

Thanks in advance!

 

 

 

 

 

[Electronics Wizardy]

1. By san what protocols are you using? You probably want more nas protocols here if there being used by workstations, like smb of cifs

 

2. Why have the server and san seprate, Are you gettin ga cluster? Id just have one box that works as a nas. What are all those cores for? File servers normally don't need much cpu power.

 

3. So people are running the app on the remote system? So you want VDI? Why not run the app on the workstations?

 

 

What network do the workstations have? What os?

 

If I was bulding this, id get a few systems like dell r7515s, put a hypervisor on them like hyper-v, then have them all have like 50tb of hdds and like 8tb of ssds. Setup something like storage spaces direct if you want to grow, so then you have clustered storage you can easily grow and be redundant if a node fails. Then get a switch with a lot of 10gbe ports, and maybe some faster ports for the server, maybe this guy https://mikrotik.com/product/crs326_24s_2q_rm

 

Id really hire someone that knows what their doing here, turning this and keeping it working right can be a challenge, and Its well worth the cost to get someone that knows how it do it well.

[GDRRiley]

a pair of 45 drives Q30 when they offer epyc may make sense.

gives you 2 machines with most of the 64 cores free and a few PCIE slots for a pair of GPUs along with 40gb network and raid cards.

 

you could go with a single larger S45 and put in 36 drives set up in 3 12 drive raid 6 stripped. I'd have the other half of the row set up for SSDs giving you a live project server with something like 14 in a raid 50. Both can have one hot spare and I'd keep a few extra HDD on hand. you may have space for 2 GPUs in that config.

 

the best all around single box solution (I don't like the risk but the cut cost maybe worth it) R282-Z93 with 12 HDD in a raid 60 and then when you need more space using a diskshelf with an add in hba/raid card. that will allow 3 GPUs, 128 cores a single M.2 for os/caching  and 40gb networking.

[sixjames1000]

11 hours ago, Electronics Wizardy said:

1. By san what protocols are you using? You probably want more nas protocols here if there being used by workstations, like smb of cifs

 

2. Why have the server and san seprate, Are you gettin ga cluster? Id just have one box that works as a nas. What are all those cores for? File servers normally don't need much cpu power.

 

3. So people are running the app on the remote system? So you want VDI? Why not run the app on the workstations?

 

 

What network do the workstations have? What os?

 

If I was bulding this, id get a few systems like dell r7515s, put a hypervisor on them like hyper-v, then have them all have like 50tb of hdds and like 8tb of ssds. Setup something like storage spaces direct if you want to grow, so then you have clustered storage you can easily grow and be redundant if a node fails. Then get a switch with a lot of 10gbe ports, and maybe some faster ports for the server, maybe this guy https://mikrotik.com/product/crs326_24s_2q_rm

 

Id really hire someone that knows what their doing here, turning this and keeping it working right can be a challenge, and Its well worth the cost to get someone that knows how it do it well.

Hey, 

 

thanks for the quick rundown! Super helpful. Let me start with the last point:

-- "Id really hire someone that knows what their doing here"

 

Totally  Just want to get a grasp and get acquainted with some basics.

2- The idea behind using a SAN is becuase all this workstations will be pulling/pushing lots of data into and from the storage, and Im worried that bandwidth and slowness would be a problem. I understand that a SAN operates sort of like on its own network and reducing the latency between each computer would be nice. Ultimately all we need is a setup that allows ~300mb/s interaction with the server for about 5 workstations. Ideally we'd want to be able to scale that (eventually) to having more workstations and a few extra compute servers.

3- no no I mean, running on the workstation but *installed* on the server. apologies, my bad

 

Any other thoughts would be supe helpful, and if you know of any online trusted resources for hire (or a platform to find them) that would be super

 

Thanks for the help thus far!

[Electronics Wizardy]

6 hours ago, sixjames1000 said:

- The idea behind using a SAN is becuase all this workstations will be pulling/pushing lots of data into and from the storage, and Im worried that bandwidth and slowness would be a problem. I understand that a SAN operates sort of like on its own network and reducing the latency between each computer would be nice. Ultimately all we need is a setup that allows ~300mb/s interaction with the server for about 5 workstations. Ideally we'd want to be able to scale that (eventually) to having more workstations and a few extra compute servers.

Normally sans are block storage, and don't really work when connectd to deskops. I think a nas would work much better here, and would easily hit the needed speed.

6 hours ago, sixjames1000 said:

- no no I mean, running on the workstation but *installed* on the server. apologies, my bad

 

Why do that? Id just install the programs on the workstations them selves. The programs aren't that big, and All the workstations have boot ssds right?

6 hours ago, sixjames1000 said:

Any other thoughts would be supe helpful, and if you know of any online trusted resources for hire (or a platform to find them) that would be super

 

hunt around for small buiness IT places, Its probaby easier and better to hire someone. There is a lot of learning that would go into getting this setup right.

[BloodKnight7]

Why dont you try a HCI solution? Something with either vSAN or S2D? You can build those clusters with standard servers. Other more "comercial solutions" would involve Nutanix or VxRail. I have found that for single offices HCI is cheaper in te longrun. 60TB is not much and could be easily achievable with 3 or 4 nodes.

[AbsoluteFool]

20 hours ago, sixjames1000 said:

Hey, 

 

thanks for the quick rundown! Super helpful. Let me start with the last point:

-- "Id really hire someone that knows what their doing here"

 

Totally  Just want to get a grasp and get acquainted with some basics.

2- The idea behind using a SAN is becuase all this workstations will be pulling/pushing lots of data into and from the storage, and Im worried that bandwidth and slowness would be a problem. I understand that a SAN operates sort of like on its own network and reducing the latency between each computer would be nice. Ultimately all we need is a setup that allows ~300mb/s interaction with the server for about 5 workstations. Ideally we'd want to be able to scale that (eventually) to having more workstations and a few extra compute servers.

3- no no I mean, running on the workstation but *installed* on the server. apologies, my bad

 

Any other thoughts would be supe helpful, and if you know of any online trusted resources for hire (or a platform to find them) that would be super

 

Thanks for the help thus far!

With the specs you are comming up with people could think you where a hollywood editor or CGI person..

 

A SAN is useless for desktops, you can't simply just "plug" and play this to "compute" devices then remote it from desktops. This will be a horrible solution..

Your best bet is:

1. A "mega" (depends how you define it) NAS, that all desktops can access at once. This cal also be configured with JBOD setup for maximal transfer speeds, however you will need to keep an extra copy somewhere else then.

2. A sort of cluster with serval nodes.

 

A SAN will work best if you planned to hava 1 server serve serval virtual desktops. Other way just stay away. They are really not worth the hassle.