I second Nathan. Bind your bolts to named streams (when building topology) and use the same stream names when emitting from the spout. ________________________________ From: Nathan Leung [[email protected]] Sent: Saturday, September 27, 2014 9:22 PM To: [email protected] Subject: Re: Direct Streams and control over tuples router
In your case I would recommend streams. Direct grouping shouldn't be used unless you have a good reason to. On Sat, Sep 27, 2014 at 9:56 PM, Bobby Evans <[email protected]<mailto:[email protected]>> wrote: Yes you are correct if you know the task Id you can send it to the exact one you want, or more than one if you desire. On Sep 27, 2014 8:05 PM, "Nick Katsipoulakis" <[email protected]<mailto:[email protected]>> wrote: Hey Andrew, At first, thank you for your immediate response. I figured that what you explain to me is the case in storm. Nevertheless, in the Javadoc of the OutputCollector class, I see a function called emitDirect(int taskId, java.util.List<java.lang.Object> tuple). Therefore, I suspect that if I know the task ID of each bolt, I can simulate the functionality I need by introducing an extra bolt in my topology. Do you believe this is going to work? If yes, do you know how I can get the task ID of a component in a storm topology? Thank you for your precious help. Regards, Nikos On Sat, Sep 27, 2014 at 8:42 PM, Andrew Xor <[email protected]<mailto:[email protected]>> wrote: Hi, The way storm works prohibits you from performing that functionality... unless you declare a separate stream for each bolt you want to have different tuples sent to it you will not be able to perform what you asked. If a bolt is subscribed to a source then all subscribed bolts to that source will receive *all* tuples emitted by that source. Finally storm groupings only apply with how the tuples are distributed among the threads spawned within each worker and not with the actual tuple emission distribution amongst the subscribed bolts. Hope this helps. Kindly yours, Andrew Grammenos -- PGP PKey -- <https://www.dropbox.com/s/2kcxe59zsi9nrdt/pgpsig.txt> https://www.dropbox.com/s/ei2nqsen641daei/pgpsig.txt On Sun, Sep 28, 2014 at 3:36 AM, Nick Katsipoulakis <[email protected]<mailto:[email protected]>> wrote: Hello, I recently started working with Storm and I want to understand better the way tuples are routed in a topology. Let us assume the following topology with Spout A and Bolts B and C: Spout-A---------->Bolt-B +-----------> Bolt-C Meaning that Spout A produces tuples for both bolts. In my use-case I want to control the tuples I am emitting to each bolt. In more detail, every time nextTuple() of Spout A is called, I want to control where the new tuple is going (and not be emitted in both bolts). Correct me if I am wrong, but I believe that I need to use direct grouping to achieve the aforementioned functionality. Am I in the right path about using direct grouping or should I follow a different approach? In addition, I was looking at Storm's documentation about Direct Grouping, but the link does not work (https://storm.incubator.apache.org/documentation/Direct-groupings.html). Where can I find more information about direct grouping? -- Nikolaos Romanos Katsipoulakis, University of Pittsburgh, PhD candidate, email: [email protected]<mailto:[email protected]> -- Nikolaos Romanos Katsipoulakis, University of Pittsburgh, PhD candidate, email: [email protected]<mailto:[email protected]> This message, including any attachments, is the property of Sears Holdings Corporation and/or one of its subsidiaries. It is confidential and may contain proprietary or legally privileged information. If you are not the intended recipient, please delete it without reading the contents. Thank you.
