Does anyone here work with or write multithreaded code?

[smjpl]

I am looking to know what languages and libraries you use and what you're using it for? Did you have to manage the threads or did the library take care of it? Things like that. I am  looking for peoples experience with writing multithreaded code and how they handled all the issues involved with it. Cheers.

 

 

 

[Deprecated]

Java makes it relatively easy and the Java API contains most of what you would use when using concurrency.

You get things from kicking off your own threads to creating thread pools that have customized behavior on how it uses the tasks its given.

While deadlocking and other issues are possible in java, knowing what your doing should prevent all that.

[gigabyte]

>.< Make a multi-threaded CPU miner? >_>

[smjpl]

Java makes it relatively easy and the Java API contains most of what you would use when using concurrency.

You get things from kicking off your own threads to creating thread pools that have customized behavior on how it uses the tasks its given.

While deadlocking and other issues are possible in java, knowing what your doing should prevent all that.

 

What sort of things were you making?

[LukeTim]

What exactly do you want to know?

I have done quite a bit of concurrent programming. Just yesterday I was writing a log parsing script in Python and found that the parsing was taking forever (it is a lot of data to sift through) so I created a thread pool and split the task up into smaller chunks. You parse equal portions of the logs in each thread, then get the results and concatenate them togther. I even tested it today to see that it had really made a difference. It did...

1 threads09:54Time taken: 90 minutes 04 seconds11:242 threads11:24Time taken: 55 minutes 12 seconds12:193 threads12:19Time taken: 42 minutes 38 seconds13:024 threads13:02Time taken: 31 minutes 23 seconds13:335 threads13:33Time taken: 26 minutes 02 seconds13:596 threads13:59Time taken: 20 minutes 50 seconds14:207 threads14:20Time taken: 16 minutes 24 seconds14:378 threads14:37Time taken: 16 minutes 29 seconds14:53

Really quite cool to see it pay off like that.

Anyway... Like I said. What do you want to know? The problems with concurrency really come when you need to have shared resources or even perform messaging between threads and processes... then you need to be thinking about mutexes and semaphores and whatnot. My Python example was simple because the threads could work completely independently.

What language are you interested in using it for?

[Ciccioo]

i'm a beginner, so i don't have "experience", but i'll post my 2 cents

 

i just finished a c++ program that needs to be kind of a local cache/accentrator for a web service: it periodically retrieves data from some sources over the internet (every 5 seconds, when possible) and it makes this data available to be read and used from php webpages

the use of multiple threads is not even a must in this case, but i thought it would have been better to keep the two things as seperate as possible

data races are trivial, just handled with std::mutex and std::unique_lock, while thread spawning is done with std::async, which saved a couple lines of code and made the code slightly cleaner-looking imho

[Deprecated]

What sort of things were you making?

A lot of things...

Any GUI is usually multithreaded because one thread needs to handle IO to the GUI and another will do any processing that needs to be done. For instance if you had a button that when clicked will do something for 5 seconds running on a single thread, once pressed the GUI will freeze until the processing is done and it can go back to handling IO.

Also at my work our servers need to process transactions really fast and creating applications that take advantage of a multicore environment can be very beneficial to investors and the business. Instead of throwing money at teams of people to re-engineer the application every year to get out an extra 25% performance, they can just invest into more hardware and the application will get more cores to use to do more work at the same time.

Just to name a couple

[LukeTim]

Also, here is a cool project I did in University to learn about asynchronous communication between processes/threads... so if you have one thread spitting out data faster than the other can process it, for example.

 

If I remember correctly, you build each file separately with gcc, and then run them as "game | ACM | delay" (it has been 2 and a half years so not 100% sure...) but anyway maybe you can get some ideas about how to do stuff... If you run game, it's just a little game where you play a fox and you have to catch a rabbit or something (they are represented by characters in the console window), and it should work perfectly. Run game and pipe the output to delay, however, and it becomes completely unplayable... because the delay delays the printing of the output to the screen... However, if you put the ACM in between the two processes, it becomes playable again because the communications module arbitrates between the two.

At least, that is how I remember it. I hope I am right...

[smjpl]

Snip

 

Anyway... Like I said. What do you want to know?

 

 

Pretty much just everything you said. I am just looking for loads of different peoples experience.

 

That was a data parallel example where there were no dependencies among the threads. I am assuming it scaled pretty well in that situation. What sort of speed ups were you getting and how many cores was it running on? My bad. I skipped over the part where you showed me everything I wanted to know. Whoops.

[LukeTim]

Pretty much just everything you said. I am just looking for loads of different peoples experience.

 

That was a data parallel example where there were no dependencies among the threads. I am assuming it scaled pretty well in that situation. What sort of speed ups were you getting and how many cores was it running on? My bad. I skipped over the part where you showed me everything I wanted to know. Whoops.

 

Some additional information... I was running on a Core i7. So it had 4 cores with hyperthreading. I think the great scaling was partly due to the fact that there is A LOT of memory accessing in parsing through strings with regex, so the hyperthreading really boosted the speed since when one thread went off to fetch some data another one could jump on and start working.

[smjpl]

Again I am just looking for everyone's experiences. I would love if you could share code examples or even any good papers comparing different techniques. I am researching the area and I'd just trying to broaden my view from a developer perspective. So people can think of this thread as a place to dump there multithreaded experiences. The more diverse the better. Anyone using Apple GCD, Intel TBB, OpenMP, MPI, Microsoft TPL or any functional programmers out there? Apparently the holy grail of parallel lies within the deterministic functional languages .

[smjpl]

A lot of things...

Any GUI is usually multithreaded because one thread needs to handle IO to the GUI and another will do any processing that needs to be done. For instance if you had a button that when clicked will do something for 5 seconds running on a single thread, once pressed the GUI will freeze until the processing is done and it can go back to handling IO.

Also at my work our servers need to process transactions really fast and creating applications that take advantage of a multicore environment can be very beneficial to investors and the business. Instead of throwing money at teams of people to re-engineer the application every year to get out an extra 25% performance, they can just invest into more hardware and the application will get more cores to use to do more work at the same time.

Just to name a couple

 

I have seen that in mobile app development (in my research, not myself personally). GCD on iOS seems to be a big one as all UI is handled on the main thread so any heavy processing or IO should be run on separate threads.

 

With Java, do you have to make sure you use thread safe libraries? What sort of things can you not use?

[fizzlesticks]

With Java, do you have to make sure you use thread safe libraries? What sort of things can you not use?

That depends on how they're being used. Of course anything that gets accessed by multiple threads should be thread safe.

The only thing I know off the top of my head is that ArrayList isn't thread safe while Vector is.

[Tam3n]

Well I do some basic c++ once in a while for fun. I've used OpenMP to speed up rendering of Mandelbrot fractal on all cores... I'd say try OpenMP for a start.

[Nuluvius]

Some that I can think of right now that do the majority of the heavy lifting for you and are pleasant to work with; We have TPL and asynchronous features in C# and the Boost Thread library for C++. The threading that's already in C++ isn't that great right now at least until C++ 11 > features maybe. Another library I have been working with recently for distributed multi threading is ØMQ. Though the .net wrapper is garbage (it has bugs and it's not maintained well) and we actually threw it away and wrapped it ourselves in the end.

 

Other's worth mentioning: C++ AMP, OpenCL & CUDA.