Jump to content

How to set Parallelism to Storm Topology which store data into child-parent relationship table?


I'm using a Apache Storm to collect some real time data from Twitter Api's. I have written a Storm Topology with a kafka-spout and 10 bolts. I'm trying to store the data into Postgres tables with child-parent relationship.

My problem:

0) As we know Storm works on parallelism all the data is inserted at once to all the tables where only few tables are getting the data.

1) i kept default count 1 parallelism to all my bolts.To insert data into all tables i kept 2 sec sleep method in each execute method of bolts.where all the data is entering into storm but it's taking long time to insert data into tables and getting Fatal error and next bolt is recovering itself.

I need solution for: How do i set parallelism count to my storm bolts ?

Thanks in advance


Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now