Agree on populating the queue on another thread. When you say "try to pop the next message", do you mean that it should block indefinitely?
My colleagues suggest that it should not. Instead a wait strategy to be used in the yaml: https://storm.apache.org/releases/1.0.3/javadocs/org/apache/storm/spout/ISpoutWaitStrategy.html From: [email protected] Subject: Re: blocking ISpout::nextTuple() I believe the docs around spouts say nextTuple() should never block. In the spout implementations I've done, I typically spin off a separate work thread in the spout's open() method and push tuples into a shared concurrent queue. nextTuple() simply tries to pop the next message off of that shared queue. On Tue, Jul 18, 2017 at 8:32 AM, Ramin Farajollah (BLOOMBERG/ 731 LEX) <[email protected]> wrote: Hi, should a spout block on its source to produce a tuple? For example, should it read its source from a blocking queue before emit? In this case, the spout will indefinitely block before it can emit a tuple. I read about how ack and fail are processed on the same thread. 1) Will blocking in spout's nextTuple result in blocking the downstream bolts? 2) How about when a messageId is not specified? 3) How about when a tuple traverses multiple JVM instances on the same or cross machines? Kind regards << �gA mind is like a parachute. It doesn't work if it is not open�h Frank Zap >> << �gA mind is like a parachute. It doesn't work if it is not open�h Frank Zap >>
