[ https://issues.apache.org/jira/browse/STORM-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958925#comment-15958925 ]
Robert Joseph Evans commented on STORM-292: ------------------------------------------- [~ujfjhz], You are right overflow has been on for all spouts and bolts for quite a while. Since security went in as a part of 0.10. If you are running into it please upgrade to at least 0.10 and preferable 1.0. > emit blocks the publishing bolt if disrupter queue is full > ------------------------------------------------------------- > > Key: STORM-292 > URL: https://issues.apache.org/jira/browse/STORM-292 > Project: Apache Storm > Issue Type: Bug > Components: storm-core > Affects Versions: 0.9.2-incubating > Reporter: Kishor Patil > Assignee: Robert Joseph Evans > Fix For: 0.10.0 > > > During testing, i notice once the disruptor queue is full, it blocks > (timed_wait ) the publishing bolt essentially creating slowdown live-lock. > Should the outputCollector.emit be a non-blocking call? > Also, configs for better control on the disruptor queue/buffer size seem to > not have any impact. > > "topology.executor.receive.buffer.size" > "topology.receiver.buffer.size" > "topology.executor.send.buffer.size" > "topology.transfer.buffer.size" > Below is example of topology that re-creates live lock scenario with > disruptor queue. > https://github.com/kishorvpatil/incubator-storm/blob/dqueue-full/examples/storm-starter/src/jvm/storm/starter/SplitJoinTopology.java > "Thread-9-splitjoinbolt" prio=10 tid=0x00007fe518a9f000 nid=0x64b5 waiting on > condition [0x00007fe5144f2000] > java.lang.Thread.State: TIMED_WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > at > java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349) > at > com.lmax.disruptor.SingleThreadedClaimStrategy.waitForFreeSlotAt(SingleThreadedClaimStrategy.java:129) > at > com.lmax.disruptor.SingleThreadedClaimStrategy.incrementAndGet(SingleThreadedClaimStrategy.java:81) > at com.lmax.disruptor.Sequencer.next(Sequencer.java:127) > at > backtype.storm.utils.DisruptorQueue.publish(DisruptorQueue.java:113) > at backtype.storm.disruptor$publish.invoke(disruptor.clj:51) > at > backtype.storm.daemon.executor$mk_executor_transfer_fn$this__4913.invoke(executor.clj:176) > at > backtype.storm.daemon.executor$mk_executor_transfer_fn$this__4913.invoke(executor.clj:183) > at > backtype.storm.daemon.executor$mk_executor_transfer_fn$this__4913.invoke(executor.clj:185) > at > backtype.storm.daemon.executor$fn__5141$fn__5155$bolt_emit__5184.invoke(executor.clj:683) > at > backtype.storm.daemon.executor$fn__5141$fn$reify__5190.emit(executor.clj:693) > at backtype.storm.task.OutputCollector.emit(OutputCollector.java:186) > at backtype.storm.task.OutputCollector.emit(OutputCollector.java:32) > at > backtype.storm.topology.BasicOutputCollector.emit(BasicOutputCollector.java:19) > at > storm.starter.bolt.SplitAndCountBolt.execute(SplitAndCountBolt.java:24) > at > backtype.storm.topology.BasicBoltExecutor.execute(BasicBoltExecutor.java:33) > at > backtype.storm.daemon.executor$fn__5141$tuple_action_fn__5143.invoke(executor.clj:634) -- This message was sent by Atlassian JIRA (v6.3.15#6346)