Re: Multiple output streams: all go through the same connection?

2016-01-06 Thread Annabel Melongo
John,  Storm has various streams groupings to achieve this. Read the "Streams Groupings" paragraph. If that doesn't answer your question, please let me know. |   | |   |   |   |   |   | | ConceptsThis page lists the main concepts of Storm and links to resources where you can find more information

Re: Multiple output streams: all go through the same connection?

2016-01-06 Thread John Yost
Awesome, thanks everyone! Annabel -> I should have clarified my question better, I meant to ask if there are multiple streams defined between two bolts whether both streams would go over the same connection. I have logic that will send one of two types of streams between two bolts depending upon w

Re: Multiple output streams: all go through the same connection?

2016-01-06 Thread Nathan Leung
Look at http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/. Pretty sure it's using one port for all of the communications. Also ZMQ is not a queue, especially not a brokered one, and storm uses netty by default in the latest versions. On Wed, Jan 6, 2016 at

Re: Multiple output streams: all go through the same connection?

2016-01-06 Thread Annabel Melongo
John, It depends on how you defined your topology. If you created a topology made of one spout, two streams and a bolt, then the data received from the spout is going to go down to the bolt via those two streams. Spouts and bolts are connected through streams. Thanks On Wednesday, January

Re: Multiple output streams: all go through the same connection?

2016-01-06 Thread Ankur Garg
Hi John , I am not aware of the storm internal working so I am not sure about this . But since no one has answered this so far , I will try my hand . I believe Spouts and Bolts communicate through an intermediate queue (ZMQ) . Assuming this it should always be a single connection to ZMQ . But Li

Multiple output streams: all go through the same connection?

2016-01-05 Thread John Yost
Hi Everyone, Had a co-worker ask me today if, I define two output streams for a spout or bolt, do they both go out over the same connection to a downstream bolt, or are there separate connections for each output stream. Anyone know? Thanks --John