Home GPU Render farm - thoughts?

[BunnyToes]

Budget (including currency): $10,000 US

Country: USA

Games, programs or workloads that it will be used for: C4D, Octane, Redshift

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

 

This is purely hypothetical - but if you wanted to set up a small GPU render farm at home, where would you start?  I'm finding advice in the area of which CPU to use, memory, etc, but as far as how I would set this up, I can't find much outside of rack units that seem like complete overkill for what I need.  The main objective with this project would be sending render-ready files to render on the farm so I can get back to other work.  The less time rendering means projects with GPU-demanding scenes delivered faster.  I'd probably start with two RTX 3090's - which by the way, are selling where I live at no premium as scalpers are finding out the hard way that gamers do not want these cards.  Anyway, is there something in between mining GPU racks and enterprise-level rack-mounted cases, something with PCIe slots, that would fit what I need?  

[Jurrunio]

Do you need that much VRAM? 3080's much cheaper per dollar for performance.

[BunnyToes]

4 minutes ago, Jurrunio said:

Do you need that much VRAM? 3080's much cheaper per dollar for performance.

Depends on the scene - also the number of cores for octane render does make a big difference

[eeeee1]

Then why not use quadros

[BunnyToes]

13 minutes ago, eeeee1 said:

Then why not use quadros

Well, the hardware side of 3D rendering is something I'm just now exploring in depth - but they are very expensive even compared to RTX 3000 cards, and have both fewer cores and lower memory bandwidth.  Am I missing something you think?

[BunnyToes]

13 minutes ago, eeeee1 said:

Then why not use quadros

Well, the hardware side of 3D rendering is something I'm just now exploring in depth - but they are very expensive even compared to RTX 3000 cards, and have both fewer cores and lower memory bandwidth.  Am I missing something you think?

[eeeee1]

Just now, BunnyToes said:

Well, the hardware side of 3D rendering is something I'm just now exploring in depth - but they are very expensive even compared to RTX 3000 cards, and have both fewer cores and lower memory bandwidth.  Am I missing something you think?

well they got insane amounts of vram going for them.

[BunnyToes]

Just now, eeeee1 said:

well they got insane amounts of vram going for them.

Hm... Only the 8000 has more vram than the 3090, and hypothetically I can get an NVlink for 80 bucks that gives two 3090's a combined vram of the 8000, which costs $1400 more than two of them and has only 4600 cores...

[eeeee1]

Just now, BunnyToes said:

Hm... Only the 8000 has more vram than the 3090, and hypothetically I can get an NVlink for 80 bucks that gives two 3090's a combined vram of the 8000, which costs $1400 more than two of them and has only 4600 cores...

don't forget ecc for all its worth

[BunnyToes]

1 minute ago, eeeee1 said:

don't forget ecc for all its worth

I thought ECC was for computational stuff like deep learning, and disabled for things like ray-path tracing in rendering?  I don't know.

[eeeee1]

Just now, BunnyToes said:

I thought ECC was for computational stuff like deep learning, and disabled for things like ray-path tracing in rendering?  I don't know.

well are you going to ray trace?

[BunnyToes]

As I understand - and to be honest, I really don't, that's why I'm here, lol - path tracing engines like the ones I render from use similar processes.

[Jurrunio]

35 minutes ago, BunnyToes said:

also the number of cores for octane render does make a big difference

yeah that's one of the advantage of getting more 3080s, 3 3080 will beat 2 3090 for example until they run of memory, of which I'm not sure what will happen or if it even matters.

 

28 minutes ago, eeeee1 said:

Then why not use quadros

Quadros are absolutely unnecessary. Rendering does not benefit from optimization specific to workstation cards. It's purely a math crunching workload.

 

10 minutes ago, BunnyToes said:

hypothetically I can get an NVlink for 80 bucks that gives two 3090's a combined vram of the 8000

no NVLink support on 3090. Yes it's phyiscally an NVLink connector in the wrong way, but as far as software is concerned it is only an SLI connector and is only used for SLI.

 

10 minutes ago, eeeee1 said:

don't forget ecc for all its worth

Rendering has no use for ECC. Unlike scientific calculations, rendering is way more forgiving to slight errors (since our eyes do not see in 0s and 1s) to the point that with enough pixels, no one will spot the errors.

 

9 minutes ago, eeeee1 said:

well are you going to ray trace?

ECC does not help with raytracing either, otherwise gaming RTX cards will also come with ECC memory.

[BunnyToes]

1 minute ago, Jurrunio said:

yeah that's one of the advantage of getting more 3080s, 3 3080 will beat 2 3090 for example until they run of memory, of which I'm not sure what will happen or if it even matters.

 

Quadros are absolutely unnecessary. Rendering does not benefit from optimization specific to workstation cards. It's purely a math crunching workload.

 

no NVLink support on 3090. Yes it's phyiscally an NVLink connector in the wrong way, but as far as software is concerned it is only an SLI connector and is only used for SLI.

 

Rendering has no use for ECC. Unlike scientific calculations, rendering is way more forgiving to slight errors (since our eyes do not see in 0s and 1s) to the point that with enough pixels, no one will spot the errors.

 

ECC does not help with raytracing either, otherwise gaming RTX cards will also come with ECC memory.

I just learned a lot.  Thank you!

[BunnyToes]

2 minutes ago, Jurrunio said:

yeah that's one of the advantage of getting more 3080s, 3 3080 will beat 2 3090 for example until they run of memory, of which I'm not sure what will happen or if it even matters.

 

Quadros are absolutely unnecessary. Rendering does not benefit from optimization specific to workstation cards. It's purely a math crunching workload.

 

no NVLink support on 3090. Yes it's phyiscally an NVLink connector in the wrong way, but as far as software is concerned it is only an SLI connector and is only used for SLI.

 

Rendering has no use for ECC. Unlike scientific calculations, rendering is way more forgiving to slight errors (since our eyes do not see in 0s and 1s) to the point that with enough pixels, no one will spot the errors.

 

ECC does not help with raytracing either, otherwise gaming RTX cards will also come with ECC memory.

Also, Octane doesn't use SLI, so on larger scenes it would probably max out at 24gb, but I think that would be ok - at that point it goes out-of-core, according to the internet and Octane manual - so I'm thinking 3090's would be the way to go that could handle larger files with more ease.  More cores on the other hand have an almost linear relationship to render times in Octane, until you hit vram limits, also according to the internet.

[Jurrunio]

1 minute ago, BunnyToes said:

Also, Octane doesn't use SLI, so on larger scenes it would probably max out at 24gb, but I think that would be ok - at that point it goes out-of-core, according to the internet and Octane manual - so I'm thinking 3090's would be the way to go that could handle larger files with more ease.  More cores on the other hand have an almost linear relationship to render times in Octane, until you hit vram limits, also according to the internet.

I just dont think it's that big of a problem since render farms at home isnt a new thing, in the past people are perfectly good with 1080Ti and 2080ti which are 11GB cards. I dont think the extra 1GB here matters that much but need more reading. so far seems like out of core is for textures and is perfectly fine to run out of VRAM as long as there's still space for data other than textures that should be in VRAM.

[BunnyToes]

2 minutes ago, Jurrunio said:

I just dont think it's that big of a problem since render farms at home isnt a new thing, in the past people are perfectly good with 1080Ti and 2080ti which are 11GB cards. I dont think the extra 1GB here matters that much but need more reading. so far seems like out of core is for textures and is perfectly fine to run out of VRAM as long as there's still space for data other than textures that should be in VRAM.

Dude, thanks for chatting me through this.  Very helpful.