Managing a render farm...

[Scruffie]

Hey guys, how's it going?

 

I've recently been tasked with expanding an animation render farm with multiple nodes connected to it. It is a CPU based render farm (infrastructure is heavily invested is CPU rendering, GPU rendering would require a paradigm shift. Too expensive ATM) with about 25 nodes hooked into it. We currently use AWS's Deadline to manage render provisioning and queuing. The current configuration works quite well actually, however, I'm rapidly running into a management/time bottleneck. 

 

If any node needs any sort of software updates it is currently handled manually. Which is fine, on a one off basis. however, if an entire suite needs to be upgraded across the farm this suddenly becomes a massive time sink that will only become worse as the farm expands in size. Is there some sort of management solution out there that can handle this? I understand you can deploy software packages through group policy, however I'm not sure you can deploy software the requires a unique serial number through GP.

 

Basically my wish list would be to have some sort of management console to deploy/update software packages and licensing management and perhaps also manage windows settings on the side (windows updates, IP configuration, etc.)

 

Hoping someone knows of a solution, or can point me in the right direction. 

[unijab]

ansible

[Electronics Wizardy]

what os?

 

Probably ansible/foreman or ad for windows.

 

Or just make a image of the config you like and just reimage all the nodes if updates are needed.

[leadeater]

For Windows PowerShell DSC would be the way to go, Linux then Ansible is a very easy tool and quick startup with low learning time and doesn't require software agents on the server (uses ssh). Puppet is a good option if you need to manage the OS over long term and different software versions and ensuring each server always stays in the defined state it should be in, Puppet always remediates the server so out of band one off configuration changes is not a good or idea and typically doesn't work at all.

[Scruffie]

21 hours ago, Electronics Wizardy said:

what os?

 

Probably ansible/foreman or ad for windows.

 

Or just make a image of the config you like and just reimage all the nodes if updates are needed.

Unfortunately there is hardware differences between the nodes. As certain blocks of the farms were purchased separately. and I'm running windows

[Salv8 (sam)]

On 6/9/2019 at 8:53 AM, Scruffie said:

Unfortunately there is hardware differences between the nodes. As certain blocks of the farms were purchased separately. and I'm running windows

you could use blender..

it's free, comes in an Msi format so you can install and manage it's install via group policy (couldn't find any info about managing the program itself though) supports network rendering via a plugin, and supports most 3d project file formats by itself and i think you can add more via some community plugins.

[Electronics Wizardy]

On 6/8/2019 at 3:53 PM, Scruffie said:

Unfortunately there is hardware differences between the nodes. As certain blocks of the farms were purchased separately. and I'm running windows

Are you on a domain?  Id put them all on a domain, then its really easily to push out scripts and lots of tools to monitor the nodes and software on them. Im kinda a fan of lansweeper for a basic setup, but you can also go sccm and other management and deplyment systems.

[leadeater]

12 hours ago, Electronics Wizardy said:

Are you on a domain?  Id put them all on a domain, then its really easily to push out scripts and lots of tools to monitor the nodes and software on them. Im kinda a fan of lansweeper for a basic setup, but you can also go sccm and other management and deplyment systems.

PowerShell DSC in pull configuration would be way easier here, it's very simple and not actually a script language like PowerShell and more a descriptor file of what the configuration should be like and it'll conform to it. Really nice and no significant backing infrastructure required to make it work, Windows Server role and the rest is inbuilt in to the Windows OS.

[djlithium]

Have you considered Squidnet? www.squidnetsoftware.com 

[Scruffie]

On 8/24/2019 at 11:48 PM, djlithium said:

Have you considered Squidnet? www.squidnetsoftware.com 

That's what we're using Deadline for...