Multi-threading with CUDA in C++

[dgm_eclipse]

I'm looking for some help with setting up a multi-threaded algorithm for some data analysis. Currently my code takes in data from a binary file, reads in sections, outputs these sections to the GPU for analysis, and prints each section once the analysis is done. However the speed of this code is crucial and various sections are being bottlenecked by initialization times and data transfers. I think that if I use threads on the processor I can have one thread reading in data, another sending the info to the GPU, and a third doing the printing to a file. This should be possible as CUDA allows for simultaneous data transfers however I have never done threaded coding before outside of the GPU.

 

Would someone be able to send me an example code or links to good websites about this? I am currently developing in Windows however will probably have to move this to Linux eventually which only makes this more complicated. 

Any advice would be greatly appreciated.

[FedeWar]

You can overlap data transfers and kernel execution using non default streams.

 

https://devblogs.nvidia.com/parallelforall/how-overlap-data-transfers-cuda-cc/

[dgm_eclipse]

Okay thank you for the help. I think that devblogs link will help a bit. The issue isn't with the CUDA threads though. I need my CPU to be reading data, writing previously analyzed data, and sending the most recently read data over to the GPU all at the same time in order to speed up my code.