Run your producer code in another thread to fill a LBQ, poll that with nextTuple instead.
You should never be blocking yourself inside a spout. Michael Rose (@Xorlev <https://twitter.com/xorlev>) Senior Platform Engineer, FullContact <http://www.fullcontact.com/> mich...@fullcontact.com On Fri, Jul 18, 2014 at 1:03 PM, Itai Frenkel <i...@forter.com> wrote: > Hello again, > > > Attached is a simplified reproduction (without the ShellSpout, but the > concepts are the same). > > > It seems that ack() and nextTuple() are always called on the same > thread. That means that there is an inherent tradeoff. > > Either nextTuple sleeps a few ms (and then the ShellSpout would serialize > alot of nextTuple messages) > > or nextTuple can sleep but then the ack is delayed. > > > Is there a way around this limitation? > > > Itai > ------------------------------ > *From:* Itai Frenkel <i...@forter.com> > *Sent:* Thursday, July 17, 2014 9:42 PM > *To:* user@storm.incubator.apache.org > *Subject:* Acking is delayed by 5 seconds (in disruptor queue ?) > > Hello, > > I have noticed that an ack takes 5 seconds to pass from the bolt to the > spout (see debug log below). It is a simple topology with 1 spout, 1 bolt > and 1 acker all running on the same worker. The spout and the bolt are > ShellSpout and ShellBolt respectively. > > It looks like the message is delayed in the LMAX disruptor​ queue. > How can I reduce this delay to ~1ms ? > > Regards, > Itai > > > 2014-07-17 18:30:30 b.s.t.ShellBolt [INFO] Shell msg: Sent process to > tuple 2759481868963667531 > 2014-07-17 18:30:30 b.s.d.task [INFO] Emitting: bolt __ack_ack > [-357211617823660063 -3928495599512172728] > 2014-07-17 18:30:30 b.s.t.ShellBolt [INFO] Shell msg: Bolt sent ack to > tuple 2759481868963667531 > 2014-07-17 18:30:30 b.s.d.executor [INFO] Processing received message > source: bolt:2, stream: __ack_ack, id: {}, [-357211617823660063 > -3928495599512172728] > 2014-07-17 18:30:30 b.s.d.task [INFO] Emitting direct: 3; __acker > __ack_ack [-357211617823660063] > 2014-07-17 18:30:35 b.s.d.executor [INFO] Processing received message > source: __acker:1, stream: __ack_ack, id: {}, [-357211617823660063] > 2014-07-17 18:30:35 b.s.d.executor [INFO] Acking message 1138 > > > >
