Why are CPU benchmarks not discussed from a Developer perspective?

[specious]

Hey all! 

 

I've been a long time fan and am a developer. I have recently been thinking about building a new rig and was deciding between AMD/Intel. So when Linus' latest video popped up (Is AMD a Good Option in 2018?) I was excited.

 

However, as I watched the video, my long time gripe with all CPU Benchmark reviews came back. I am a software developer and I spend a lot of time compiling. When looking to build a new rig, I am explicitly interested in reducing compile times. However, I don't believe there is much information out there regarding a CPUs impact on compile time.

 

There are people who post to ask about building the best systems for development, however, I have not seen an indepth review by a group (Linus or otherwise) that investigates how different hardware performs against compilation time. I would love to see Linus' take on what is best for development.

 

I'm sure I'm not the only dev around wishing that hardware reviewers would do more than just gaming, animation and video editing benchmarking. What do ya'll think?

[ShadySocks]

Developers typically don't need machines as powerful as gamers. Also, compile time is very specific to the program and the compiler so it would be hard to benchmark.

[Alex Atkin UK]

Compiling is such a can of worms as I have seen on the occasions where I tried to compile something that it rarely maxes out any single CPU more, nor does it necessarily scale well to multiple cores.

 

My guess is this is more related to the fact compiling involves a lot of small IO operations which even an SSD is not sufficient to keep the CPU fed.  The bottleneck may even be down to the CPU cache itself, although I'm just guessing.

 

But as said above, as all code is different, there is really no one perfect solution.

[specious]

23 minutes ago, Alex Atkin UK said:

Compiling is such a can of worms as I have seen on the occasions where I tried to compile something that it rarely maxes out any single CPU more, nor does it necessarily scale well to multiple cores.

 

My guess is this is more related to the fact compiling involves a lot of small IO operations which even an SSD is not sufficient to keep the CPU fed.  The bottleneck may even be down to the CPU cache itself, although I'm just guessing.

 

But as said above, as all code is different, there is really no one perfect solution.

So there really is no way to figure out what hardware is generally better for dev? I find that hard to believe. 

 

[specious]

30 minutes ago, ShadySocks said:

Developers typically don't need machines as powerful as gamers. Also, compile time is very specific to the program and the compiler so it would be hard to benchmark.

I typically use react / react-native for Web/app stuff and  cocos2dx for game dev stuff. Generating an apk on android studio for games typically last between 25 - 40 minutes. 

 

As I run a local backend environment as well for game dev, i generally have docker running on my machine which consumes quite a bit of resources too.

 

So I do need a pretty beefy set up to support all these things

[RogerW]

1 hour ago, specious said:

I typically use react / react-native for Web/app stuff and  cocos2dx for game dev stuff. Generating an apk on android studio for games typically last between 25 - 40 minutes. 

 

As I run a local backend environment as well for game dev, i generally have docker running on my machine which consumes quite a bit of resources too.

 

So I do need a pretty beefy set up to support all these things

I’m interested in comparing these newer AMD and Intel chipsets/cpus for development as well. If we assume disk speed is similar, and some /most processes are CPU bound, is there in this case any research between the two? I see cheaper cost overall for AMD, but slightly higher single core performance (gaming, however). 

 

Any suggestions for investigating this?

[Guest]

well a benchmark suite that would just repeatedly measure some development tasks would certainly be interesting. It's true the requirements vary from technology to technology but that doesn't mean one could not just measure whats what to find the hardware that supports best what you do. I don't think game benchmarks are that different. There is no absolute truth, a lot depends on optimization.

 

A larger performance comparison is (for example, not trying to start a war...) on the gradle website and they also provide some instructions on how to repeat them.

 

I would believe similar measurements should be possible for - well anything. Compile java, or building node applications or running some TypeScript compiler... there are a lot of open source projects around, simply building them a few times and measuring should already give an indication what to expect from a certain hardware platform - to simplify what to look for - or where to invest more money (in memory, faster storage, more cpu's, faster network, other miracles, ...).

 

Don't know a suite that would do this - would certainly be interesting