Why are Blender CPU benchmarks usually limited to 3D rendering?

[leoz96]

Hi everyone! new member here, thanks for letting me in, ive recently started watching a lot of benchmarking videos and ive noticed that most Blender CPU benchmarks include only 3D rendering as a workload, while i wouldnt disagree that render time is a good way to measure the general performance of cpus, i feel like its not really accurate as a representation of a usual workload.

Most people who do 3D rendering with Blender (Or really most rendering programs) will use a GPU for this task, since in almost all cases GPUs will be faster than CPUs, especially with the recent implementation of RT hardware. Theres a lot of tasks in Blender that take advantage of high end CPUs like smoke simulations, fluid simulations, physics simulations, particles, sculpting, modelling, remeshing, etc but these are almost never adressed in reviews. I know its hard to include most of this information in a single review, and its easier to use the already established benchmarks, but i feel like more information in this area would be valuable for Blender users or really most modelling program users like Houdini, C4D and Maya.

[Jurrunio]

I wonder if it's because it doesnt affect the CPU performance hiearchy? So while the scores are different, the rankings and differences between CPUs are still the same.

[kucharczykt]

13 minutes ago, leoz96 said:

Hi everyone! new member here, thanks for letting me in, ive recently started watching a lot of benchmarking videos and ive noticed that most Blender CPU benchmarks include only 3D rendering as a workload, while i wouldnt disagree that render time is a good way to measure the general performance of cpus, i feel like its not really accurate as a representation of a usual workload.

Most people who do 3D rendering with Blender (Or really most rendering programs) will use a GPU for this task, since in almost all cases GPUs will be faster than CPUs, especially with the recent implementation of RT hardware. Theres a lot of tasks in Blender that take advantage of high end CPUs like smoke simulations, fluid simulations, physics simulations, particles, sculpting, modelling, remeshing, etc but these are almost never adressed in reviews. I know its hard to include most of this information in a single review, and its easier to use the already established benchmarks, but i feel like more information in this area would be valuable for Blender users or really most modelling program users like Houdini, C4D and Maya.

I don't think that blender benchmarks are neccicarilly to show preformence for blender users and more to show the overall power of the cpu

[leoz96]

12 minutes ago, Jurrunio said:

I wonder if it's because it doesnt affect the CPU performance hiearchy? So while the scores are different, the rankings and differences between CPUs are still the same.

I thought about it but upon looking up some tests people have previously done it seems that different cpu tasks dont seem to follow a defined pattern, some tasks like simulations perform better on cpus with more cores, while others like viewport performance and sculpting perform better on cpus with fewer, more powerful cores. I guess as kucharczykt mentions the benchmark is just used as a general performance indicator.

[piemadd]

13 minutes ago, kucharczykt said:

I don't think that blender benchmarks are neccicarilly to show preformence for blender users and more to show the overall power of the cpu

The thing is that the workloads they tend to use to stress test the CPU aren't really that relevant in the Blender workspace. Tho I can't blame them since I don't think most tech users are using blender on a daily basis.

[piemadd]

6 minutes ago, leoz96 said:

I thought about it but upon looking up some tests people have previously done it seems that different cpu tasks dont seem to follow a defined pattern, some tasks like simulations perform better on cpus with more cores, while others like viewport performance and sculpting perform better on cpus with fewer, more powerful cores. I guess as kucharczykt mentions the benchmark is just used as a general performance indicator.

Well cpus with more cores perform better in rendering, soooo

[Serin]

The trick is how you design the benchmarks for things that are perhaps a little more interesting than raw rendering power. They'd likely have to hire a vfx artist to create meaningful tests for most of this stuff.
I'd add viewport playback framerate and baking to the list. And probably a really stupidly high poly scene performance.

10/10 would like to see at least some of this stuff though. 

[PeachGr]

3 minutes ago, pierom_qwerty said:

Well cpus with more cores perform better in rendering, soooo

not even slightly. My 2060 super outperforms ryzen 7 3800x by about 3 times
The benchmarking is just for brute force comparison
To understand the deal, 3D artists usuualy stack 2 or more GPUs and sometimes use even Intel's CPU for the old decoding and the denoising (currently useless)

[piemadd]

Just now, PeachGr said:

not even slightly. My 2060 super outperforms ryzen 7 3800x by about 3 times
The benchmarking is just for brute force comparison
To understand the deal, 3D artists usuualy stack 2 or more GPUs and sometimes use even Intel's CPU for the old decoding and the denoising (currently useless)

I was talking about cpu vs cpu, not cpu vs gpu. My 2060 super way outperforms my 2700x. I get that the benchmarking is brute force, but they should also try running the render with 2 threads enabled to compare the clockspeed difference between cpus

[kucharczykt]

8 minutes ago, pierom_qwerty said:

The thing is that the workloads they tend to use to stress test the CPU aren't really that relevant in the Blender workspace. Tho I can't blame them since I don't think most tech users are using blender on a daily basis.

 I use blender so its definitely annoying somtimes.

[leoz96]

5 minutes ago, pierom_qwerty said:

Well cpus with more cores perform better in rendering, soooo

Yes generally cpus that perform well in the current 3D rendering benchmark are gonna perform well in simulations because they have more cores, but other aspects like eevee performance and mesh deformation are generally better on cpus with fewer, more powerful cores, so the general 3D rendering benchmark is not a good complete indicator. I agree with you tho, this information isnt really relevant to the avarage consumer but it could be valuable for 3d software users.

[tim0901]

Because Blender is an example of a tile-based renderer that's:

 

a) Free

b) Easy to setup and use in a review scenario - it has it's own benchmark built-in

 

Most people doing stuff in Blender will use GPU rendering yes. But the same isn't true amongst all tile-based renderers.

 

Also: GPU rendering for this kind of renderer isn't universally available. Arnold - the default renderer for Autodesk Maya - only released a GPU renderer in the 2020 release. Anyone running an older version of Maya (very common) is restricted to CPU rendering. Pixar's Renderman only released it's XPU rendering (mixed CPU and GPU) in 2019. Some renderers also don't have full feature parity between CPU and GPU rendering, so you may be limited by that.

[PeachGr]

6 minutes ago, pierom_qwerty said:

I was talking about cpu vs cpu, not cpu vs gpu. My 2060 super way outperforms my 2700x. I get that the benchmarking is brute force, but they should also try running the render with 2 threads enabled to compare the clockspeed difference between cpus

they do other tests like cinebench for single core.
I mean they can test performance by many applications but there is not a real point after a few tests.
You can check hardware unboxed if you want many benchmarks (they are now in floatplane)
I wanted to see how RADEON will do in Blender because if you run full AMD system, you wont have dinoiser by NVIDIA nor INTEL
and if there is a reasonable way to do it.
I also understand that its kinda niche use so i m surprised they even mention CPU performance there

[leoz96]

1 minute ago, tim0901 said:

Because Blender is an example of a tile-based renderer that's:

 

a) Free

b) Easy to setup and use in a review scenario - it has it's own benchmark built-in

 

Most people doing stuff in Blender will use GPU rendering yes. But the same isn't true amongst all tile-based renderers.

 

Also: GPU rendering for this kind of renderer isn't universally available. Arnold - the default renderer for Autodesk Maya - only released a GPU renderer in the 2020 release. Anyone running an older version is restricted to CPU rendering. Pixar's Renderman only released it's XPU rendering (mixed CPU and GPU) in 2019. Some renderers also don't have full feature parity between CPU and GPU rendering, so you may be limited by that.

Yes, i didnt want to imply that CPU Rendering benchmarks are absolutely useless! I personally use Keyshot very often and unless you have an RTX card its CPU only, with various features limited to the CPU mode, so in that sense CPU rendering benchmarks are very valuable to me, and considering that Blender is free and has very standarized tests it makes sense that it has become a popular benchmark for this. What i meant to say is that a more general benchmark of a 3D modelling or animation workload that includes simulations, baking, playback speed, sculpting, etc. could be very valuable when it comes to choosing a CPU for 3D artists.

[tim0901]

1 minute ago, leoz96 said:

Yes, i didnt want to imply that CPU Rendering benchmarks are absolutely useless! I personally use Keyshot very often and unless you have an RTX card its CPU only, with various features limited to the CPU mode, so in that sense CPU rendering benchmarks are very valuable to me, and considering that Blender is free and has very standarized tests it makes sense that it has become a popular benchmark for this. What i meant to say is that a more general benchmark of a 3D modelling or animation workload that includes simulations, baking, playback speed, sculpting, etc. could be very valuable when it comes to choosing a CPU for 3D artists.

Absolutely it would. Unfortunately most reviewers simply aren't equipped to run such a test, hell a lot of them probably don't know what half of that list means. Only those in the field really know what to look for, making it very difficult to properly benchmark. It would also probably be pretty difficult to standardize and repeat.

 

A lot of it also comes down to exactly what you're doing with the software. Just like video editing, each person's needs differ depending on exactly how you're using the software. Many sculpting tools (eg. extrusion) are very single-threaded in nature (Zbrush explicitly states this on their website), but others are more easily parallelized. Depending on which tools you use more, the conclusion as to which CPU you should buy may differ as well, and that's not something you'll ever be able to get across in a canned benchmark.

[leoz96]

16 minutes ago, tim0901 said:

Absolutely it would. Unfortunately most reviewers simply aren't equipped to run such a test, hell a lot of them probably don't know what half of that list means. Only those in the field really know what to look for, making it very difficult to properly benchmark. It would also probably be pretty difficult to standardize and repeat.

 

A lot of it also comes down to exactly what you're doing with the software. Just like video editing, each person's needs differ depending on exactly how you're using the software. Many sculpting tools (eg. extrusion) are very single-threaded in nature (Zbrush explicitly states this on their website), but others are more easily parallelized. Depending on which tools you use more, the conclusion as to which CPU you should buy may differ as well, and that's not something you'll ever be able to get across in a canned benchmark.

Yeah youre right, its definitely not something i would expect to see tackled on a youtube review, even on a written benchmark or graph it would probably be hard to get across. I guess the approach could be some documentation that generally describes the behaviour of each tool and process when it comes to cpu utilization, to let the users decide depending on their specific workload.

[LogicalDrm]

-> Moved to CPUs, Motherboards and Memory