CPU vs. GPU for scientific computing

[threadripperforscience]

Budget (including currency): ~10,000€

Country: Europe

Games, programs or workloads that it will be used for: Scientific computing/modelling/simulations (geo-spatial data)

 

I am considering to invest in a workstation to allow for larger/faster computations. While this could be in the context of machine learning (vanilla regression and classification), complexity of the models would vary and certainly I would not only (if at all) use deep-learning models. I know more complex deep-learning models are done on GPUs nowadays. I was hoping you could give your views on use-cases for GPU-heavy workstations vs. CPU-heavy workstations.

 

In particular, I am wondering whether it makes sense to build a Threadripper 3990x build with 248 GB memory, or if the money is better invested by taking a smaller CPU but instead 2 to 4 GPU cards (3080s?). I work mostly in R and Python. So far, I have generally split runs on the 12 threads I have and I was naivly thinking that the 128 threads of the threadripper would allow me to scale things up. However, I am not sure whether most computations could also be performed on GPUs and if those always outperform more CPU cores under certain conditions. Note, I often work with geo-spatial data that needs to be read into memory during computation steps. It might therefore be possible that memory becomes a bottleneck.

 

Thanks to anyone willing to share viewpoints or things to look out for

 

[igormp]

First of all, are you going to need double precision (fp64) for your models/simulations? If so, then going with a regular geforce is a bad idea since those lack DP hardware. You're better off with an A100 or any V100-based GPU (such as the Titan V or GV100).

 

Second, will your workload easily fit into a GPU's framebuffer? 2~3 3080s is only 40gb, maybe you'd be better off with some 3090s instead.

 

If you're doing mostly simulations, those tend to be more reliant on CPU and RAM instead of GPUs. But since you mentioned that you're going to use DL models, then a GPU would really be needed (trying to train something off a CPU would be really painful).

 

Another option would be to go for a CPU-heavy workstation, and use GPUs on the cloud whenever you need to use those, then evaluate if it the time you're spending using those is worthwhile the investment to have it locally.

[threadripperforscience]

Thanks for your feedback. I had neither considered the precision nor the framebuffer!

 

I did compute double precision floats in previous simulations. I cannot really answer your question on the framebuffer.

 

I think the main struggle I am facing is that I would like a generalist system. I have not used DL models, but only random forests, gradient boosting, support vectors, and so on. The simulation studies were entirely different from the machine learning topics. Here, I used stochastic simulations that computed problem-specific functions for different parameter values. The functions themselves were usually straightforward but the number of times I needed to compute them was cumbersome. E.g., for global sensitivity analyses the number of runs required can increase drastically if you have just a few more parameters. 

 

I guess my naive idea was that I would like a system that would give me flexibility to use the proper methods without worrying much about computing time. Can you think of modelling work, other than deep learning, that would benefit from GPU-based processing?

 

 

[igormp]

4 hours ago, threadripperforscience said:

but only random forests, gradient boosting, support vectors

Those can be accelerated by a GPU, usually.

 

5 hours ago, threadripperforscience said:

The functions themselves were usually straightforward but the number of times I needed to compute them was cumbersome. E.g., for global sensitivity analyses the number of runs required can increase drastically if you have just a few more parameters. 

But did each run take a long time? If so, maybe looking towards GPU implementations of your most used tasks (afaik, there are some GPU implementations for sensitivity analysis) and see if it'd be worth the hassle/money.

5 hours ago, threadripperforscience said:

Can you think of modelling work, other than deep learning, that would benefit from GPU-based processing?

I'm far from versed in modeling, but I do know that some stuff are GPU-accelerated nowadays, such as Monte Carlo and methods for ODE-solving.