Mostly C, Any programs that can automatically "adapt" existing source code to use a gpu instead of cpu?

[Poet129]

 

Like the title says.

[Sauron]

If it were that easy it would be built into every compiler. It is not. GPUs work differently and code has to be written specifically with GPUs in mind for it to work.

[Eigenvektor]

This is very difficult, if not impossible. As @Sauron said GPUs work very differently from CPUs. To take full advantage of a GPU, your problem needs to be massively parallel in nature. But that is only part of it:

 

I lack the vocabulary to properly describe the issue, but I will try my best. To solve a problem, you first need to understand it. To automate that, you first need to understand the problem solving process itself. You need to solve the problem of "how to solve the initial problem in every possible permutation".

 

To do what you are asking, you need to understand how to parallelize source code. Not just a particular piece of code, but any source code. Only when you are able to formally describe the process of doing that are you going to be able to write a program that can parallelize existing code.

 

You also need to realize that not everything can be parallelized and even if it can, it may not be worth it (i.e. draw sufficient advantage from it). Your program also needs to be able to recognize and handle such a case.

[JacobFW]

Might try discussing this on some forums dedicated to emulator development, as many emulators have to do this to some extent, by mapping the instructions and registers used by the original cpu & graphics chips to that of their modern equivalents, or by reverse engineering the code and rewriting portions of it so that it can run on modern hardware.

 

This is one of the things that dll's can be used for.  By keeping certain portions of the code separate, they can be updated independently of the original executable, to allow for better performance/compatibility as new hardware becomes available.  Even then however, you're still going to be limited by how exactly the code is modularized; which aspects your code is responsible for, and which aspects the dll developer is responsible for, and how forward thinking everyone is during development.

 

Unfortunately there is no one size, fits all solution.

[Eigenvektor]

40 minutes ago, JacobFW said:

Might try discussing this on some forums dedicated to emulator development, as many emulators have to do this to some extent, by mapping the instructions and registers used by the original cpu & graphics chips to that of their modern equivalents, or by reverse engineering the code and rewriting portions of it so that it can run on modern hardware.

Running code developed for a CPU on a GPU is far more than emulation. You can't just translate CPU instructions to GPU instructions and be done with it. At least if you want to get any kind of advantage out of it.

 

GPUs are optimized for a specific use case: Run a limited set of operations on a huge amount of data in parallel. Unless the task you want to solve fits this profile, a GPU is not well suited for it. Adjusting an existing program to make use of this is nontrivial and requires far more than a translation layer or a simple rewrite.

[trag1c]

You also have to keep in mind that unless you're running something like embedded hardware without an OS you have to deal with the fact that all a GPU application is a set of commands sent to a runtime which deals with a driver.

 

Nothing is compiled for a GPU with the exception of a few programmable pipeline stages which in turn are small programs (generally they're kept as small as possible) that are run for each piece/set of data in parallel. So you still need a cpu application to configure the pipeline, allocate memory, load data, send it through the pipeline and read/unload the result or reconfigure the pipeline and the send the data through repeatedly for how ever many passes are needed. This all assumes that the data is not dependent on each other and that is large enough  so that it can be processed in extremely parallel fashion.

[JacobFW]

8 hours ago, Eigenvektor said:

Running code developed for a CPU on a GPU is far more than emulation. You can't just translate CPU instructions to GPU instructions and be done with it. At least if you want to get any kind of advantage out of it.

 

GPUs are optimized for a specific use case: Run a limited set of operations on a huge amount of data in parallel. Unless the task you want to solve fits this profile, a GPU is not well suited for it. Adjusting an existing program to make use of this is nontrivial and requires far more than a translation layer or a simple rewrite.

Completely agree.  The point though is that those guys have to deal with the same generalized problem:  how do we make code developed for one system run well on another.

I can't think of a reason why this should be possible, but I'll admit, I'm nowhere near an expert in this field; those guys are.  If someone has developed a grand solution us average mages can't even dream of, it will likely be someone in that field

 

 

 

[JacobFW]

I just realized I misread the title.  For some reason I thought it was referring to executables and just ran with it.  Sorry if I confused anyone.

 

But the answer is still no.  The only way it would even be possible is if you basically have an AI which can do it for your (and if you had that why the hell would you waste it on a task like this).

 

Again, another good example of where using a DLL would be beneficial.

 

 

[Franck]

Nothing is automatic but there is library that can do as such. I do not know for C but for C# there is task libraries that can, by using special task command send to the accelerated GPU or CPU depending on what you have available. But behind the scene there is 2 code. Doing a loop on the CPU is not the same instruction as loop on the GPU but the code detect if you have the GPU then send the second set of instruction. Bu you don't have to manually type both version of the same code, it's abstracted. To me it's the closest we have come to what you want to do. And that technique is perfect in my eye. I write the code i want to perform and it runs on what's best available. Exact same principle as OpenGL can be ran using the CPU (software) or GPU (hardware) emulation. You just write one OpenGL command and the backend library code does the work to call the CPU or GPU instruction. So far those tools i know are all limited to Nvidia card all models GTX to Quadro for consumer grade and Tesla and GRID for enterprise grade.

[Poet129]

4 hours ago, Franck said:

Nothing is automatic but there is library that can do as such. I do not know for C but for C# there is task libraries that can, by using special task command send to the accelerated GPU or CPU depending on what you have available. But behind the scene there is 2 code. Doing a loop on the CPU is not the same instruction as loop on the GPU but the code detect if you have the GPU then send the second set of instruction. Bu you don't have to manually type both version of the same code, it's abstracted. To me it's the closest we have come to what you want to do. And that technique is perfect in my eye. I write the code i want to perform and it runs on what's best available. Exact same principle as OpenGL can be ran using the CPU (software) or GPU (hardware) emulation. You just write one OpenGL command and the backend library code does the work to call the CPU or GPU instruction. So far those tools i know are all limited to Nvidia card all models GTX to Quadro for consumer grade and Tesla and GRID for enterprise grade.

Do you remember what it is called?

[Franck]

7 minutes ago, Poet129 said:

Do you remember what it is called?

For C# the one that come to mind is AleaGPU (probably the most recent of the bunch). Didn't had time to check it out properly yet but i recall they have hundredfold performance gain in some aspect. Considering c# compiled performance is very close to any average C++ build it's still a very good bump in performance. It's nothing new these type of libraries. It's been out there since Nvidia Serie 200 at minimum wince i recall using one with servers equipped with 295 GTX

[Poet129]

50 minutes ago, Franck said:

For C# the one that come to mind is AleaGPU (probably the most recent of the bunch). Didn't had time to check it out properly yet but i recall they have hundredfold performance gain in some aspect. Considering c# compiled performance is very close to any average C++ build it's still a very good bump in performance. It's nothing new these type of libraries. It's been out there since Nvidia Serie 200 at minimum wince i recall using one with servers equipped with 295 GTX

Thank you, guess I should have mentioned I was willing to learn a new programming language just to be able to "seamlessly" use the gpu and cpu.

[Franck]

16 hours ago, Poet129 said:

Thank you, guess I should have mentioned I was willing to learn a new programming language just to be able to "seamlessly" use the gpu and cpu.

I found my notes on it and i have specially targeted that one because it implement the actual C++ CUDA library from Nvidia and it is completely cross platform.

[shadow_ray]

On 11/4/2020 at 1:38 PM, Franck said:

I found my notes on it and i have specially targeted that one because it implement the actual C++ CUDA library from Nvidia and it is completely cross platform.

it is completely cross platform, given that you've got an nvidia card and drivers are willing to work on that platform

[Franck]

31 minutes ago, shadow_ray said:

it is completely cross platform, given that you've got an nvidia card and drivers are willing to work on that platform

yeah, i would question very hardly an AMD card with CUDA cores anyhow

[shadow_ray]

Sorry, by platform i meant OS.

 

But theoretically speaking, the same or similar hardware is not required. Kernels written in CUDA C/C++  compile into a byte code called PTX, and the driver (on the fly) compiles it into machine code by targeting a specific architecture.

 

Actually this is somewhat similar to what Khronos Group is trying to achieve with SPIR, so everyone can write kernels/shaders in their favourite language and it should work with different back-ends.