Cluster computing and Render cluster help

[Mr. Cucumber]

I have space for 1 server cabinet in my house... it's in a closet.... I have installed an air conditioner in there and temperature monitoring and all that jazz ... So i have a 38U server rack ... Anymore than that is too much for me

... So I want to build a compute cluster 

I want to have 12 2u compute node and 1 head node... I also want to add 2 4u render node... And please tell me if i need two network switches... So i have 4u of real estate left... Maybe 2 1u switches if needed... And can i get away with like 1 big 10GbE nas for all the storage?

So for the compute unit... I want to use the latest  i9-9980xe (I'll start buying in may next year) cpu and and enough ram... I want to use desktop motherboards... And non ecc ram...  

Do i need any gpus? And if i do i want to go with 1080ti ... 2 per system in sli and how should i arrange the head node?

I want all the top specs... 

And for the render cluster ... Should i go with threadripper ...i also want 1080ti 2 per chassis ( 2080ti is burning literally... My friends blew up a few days early... Founders edition so I don't want to take any chances...) Or i can go with any good gpus that come out then... So my question is.. that how do i set up and configure the compute cluster as one powerful system and how do i combined the the two 4u rendering machines as one.. . Like a render cluster ... With combined or parallelize the machines for faster rendering? And can these systems use one nas as their storage or do i need two of them?

[NelizMastr]

Might be a good idea to write down both for us and for yourself what it is that you're trying to achieve here. What kind of workloads will you be running? Any specific applications?

 

Networking wise, I'd reserve about 4-6U. 2U for a good rackmount 10GbE switch and the rest for cable management and aesthetics. You'll need at least 2U space beneath the switch to route the cables backwards to the rear of your nodes. Unless all machines will hit the NAS at once with massive file sizes, it should be alright. 

 

I'd stay away from the 9980XE for a render farm. It's just a poor value chip. For $300 less, you get almost twice the cores with Threadripper. For performance parity, assuming the render software scales to all threads, you'll need both less money and less nodes.

[AbsoluteFool]

2 hours ago, Mr. Cucumber said:

I have space for 1 server cabinet in my house... it's in a closet.... I have installed an air conditioner in there and temperature monitoring and all that jazz ... So i have a 38U server rack ... Anymore than that is too much for me

... So I want to build a compute cluster 

I want to have 12 2u compute node and 1 head node... I also want to add 2 4u render node... And please tell me if i need two network switches... So i have 4u of real estate left... Maybe 2 1u switches if needed... And can i get away with like 1 big 10GbE nas for all the storage?

So for the compute unit... I want to use the latest  i9-9980xe (I'll start buying in may next year) cpu and and enough ram... I want to use desktop motherboards... And non ecc ram...  

Do i need any gpus? And if i do i want to go with 1080ti ... 2 per system in sli and how should i arrange the head node?

I want all the top specs... 

And for the render cluster ... Should i go with threadripper ...i also want 1080ti 2 per chassis ( 2080ti is burning literally... My friends blew up a few days early... Founders edition so I don't want to take any chances...) Or i can go with any good gpus that come out then... So my question is.. that how do i set up and configure the compute cluster as one powerful system and how do i combined the the two 4u rendering machines as one.. . Like a render cluster ... With combined or parallelize the machines for faster rendering? And can these systems use one nas as their storage or do i need two of them?

I hate to seem like the blunt person here. But from what you are writing it seems to me that you've done very little reading on clusters. Not all applications work in clusters, so first you should figure out what application you want to run, and other things you might want to use it for.

 

Clusters preform best if all nodes are identical at the hardware level. 

In clusters you have controller nodes and compute nodes. There is nothing called "render nodes". You can however build a two node cluster to be a stand alone render system. Also to my knowledge rendering works best with GPUs.

[Mr. Cucumber]

Actually I have built PC's for a long tine now.. liquid cooling.. doing stupid experiemnts and actually making them work... Putting all that aside... This is actually my first time into doing such a thing.... Which @AbsoluteFool has pointed at (thanks by the way )... I have recently completed my masters degree on computational chemistry ... And our University lacks any service for processing student models...so everyone is Forced to do complete their project and models on their own machine ...which take a lot of time ... And I've even seen some of my friends PC's have been actually damaged due to thermal issues... So I want to host a service.. for a little amount of money to render their projects and do some of my own research in simulations and other such workloads... And I have recently grown an interest into 3d modeling ...cad...and (mainly my firiends will use it.... ) But I have a really like really top tier PC workstation...but... When I start rendering.... I have nothing to do but stare at my barely functional pc...and wait... I don't want that .. I want to offload the rendering to the render cluster and cut down render times so that I don't have to wait while something is processing... And yes... I am new to this... Someone please suggest me any books or any articles about this subject which will help me to understand the concept from the ground up and get up and running.... And that i don't have to do this half blind.. thanks

[Mr. Cucumber]

@NelizMastr then what should I go with? And doesn't threadripper run hotter than the 9980xe... So any suggested thermal solutions? And yes... You're right..... threadripper is a way better option...

[NelizMastr]

5 minutes ago, Mr. Cucumber said:

@NelizMastr then what should I go with? And doesn't threadripper run hotter than the 9980xe... So any suggested thermal solutions? And yes... You're right..... threadripper is a way better option...

The TR2990WX is actually somewhat easier to cool than a 9980XE at full whack, so that won't be an issue. They're pretty composed. 

[Mr. Cucumber]

9 minutes ago, NelizMastr said:

The TR2990WX is actually somewhat easier to cool than a 9980XE at full whack, so that won't be an issue. They're pretty composed. 

And any help about me getting a good grasp at how I can do what I want to do ...any books/articles/videos? ... And.. should I use liquid metal?... I have some thermal grizzly conductonaut ... Is it safe?

[NelizMastr]

8 minutes ago, Mr. Cucumber said:

And any help about me getting a good grasp at how I can do what I want to do ...any books/articles/videos? ... And.. should I use liquid metal?... I have some thermal grizzly conductonaut ... Is it safe?

No use for liquid metal here, no. You won't be delidding these chips as they have soldered heatspreaders. Can't help you with books or research on the subject, as I don't have any in depth knowledge about your use case.

[AbsoluteFool]

1 hour ago, Mr. Cucumber said:

Actually I have built PC's for a long tine now.. liquid cooling.. doing stupid experiemnts and actually making them work... Putting all that aside... This is actually my first time into doing such a thing.... Which @AbsoluteFool has pointed at (thanks by the way )... I have recently completed my masters degree on computational chemistry ... And our University lacks any service for processing student models...so everyone is Forced to do complete their project and models on their own machine ...which take a lot of time ... And I've even seen some of my friends PC's have been actually damaged due to thermal issues... So I want to host a service.. for a little amount of money to render their projects and do some of my own research in simulations and other such workloads... And I have recently grown an interest into 3d modeling ...cad...and (mainly my firiends will use it.... ) But I have a really like really top tier PC workstation...but... When I start rendering.... I have nothing to do but stare at my barely functional pc...and wait... I don't want that .. I want to offload the rendering to the render cluster and cut down render times so that I don't have to wait while something is processing... And yes... I am new to this... Someone please suggest me any books or any articles about this subject which will help me to understand the concept from the ground up and get up and running.... And that i don't have to do this half blind.. thanks

I'd suggest reading everything that wikipedia has about clusters, this will also be a ponting direction of what else you would want to learn to actually complate the task you want to do. I know youtube have some "cluster Raspberry pi" guides. That's basicly the same subject, althought i do recomend you do this with two nodes before even attempting to do it on a higer scale.

 

Clustering is a very uniqe subject in the compute world. And is mostly used to create million dollars supercomputers.

[AbsoluteFool]

Also before i forget.. Most clusters use a type of linux opperative system. So might be a good idea to think about what OS you want to use. And learn about the command line for the system you're going to use. Will be the best start.

[Mr. Cucumber]

@AbsoluteFool yes... Thanks for the help... I have some experience from Ubuntu and kali... From my college days ... And I have a fair collection of parts (motherboards cpus gpus rams I have 2 3 even 4 identical parts) so should I build two identical systems and learn how to use it? Will it even work?

[Electronics Wizardy]

6 hours ago, Mr. Cucumber said:

@NelizMastr then what should I go with? And doesn't threadripper run hotter than the 9980xe... So any suggested thermal solutions? And yes... You're right..... threadripper is a way better option...

How about used servers? Something like a used r730 will be much faster than a threadripper or i9 and you can get ecc and much more ram, and they have nice management features.

 

For software, id run a hypervisor on all the system and do it all in vms. You lose a bit of performance, but its much easier to setup and reconfigure.

[AbsoluteFool]

7 hours ago, Mr. Cucumber said:

@AbsoluteFool yes... Thanks for the help... I have some experience from Ubuntu and kali... From my college days ... And I have a fair collection of parts (motherboards cpus gpus rams I have 2 3 even 4 identical parts) so should I build two identical systems and learn how to use it? Will it even work?

That's what i would have done before i scale it up to 10 and more nodes. It's alot easier to make a two node cluster to get the basic idea, instead of jumping strait into a 10-12 node cluster.

If it will work depends entierly on you and the applications you want to run.

[Mr. Cucumber]

8 hours ago, Electronics Wizardy said:

How about used servers? Something like a used r730 will be much faster than a threadripper or i9 and you can get ecc and much more ram, and they have nice management features.

 

For software, id run a hypervisor on all the system and do it all in vms. You lose a bit of performance, but its much easier to setup and reconfigure.

Unfortunately in our country decommissioned server gear is rare.... And it's kinda illegal to sell and buy used servers here... I don't know why... I don't make the laws...