[ https://issues.apache.org/jira/browse/STORM-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405773#comment-16405773 ]
Roshan Naik commented on STORM-2970: ------------------------------------ [~ethanli] i think this issue was a side effect of worker thread not being spun up in multi-worker mode with RAS enabled. can we close this if not relevant anymore ? > JCQueue can be full and will throw IllegalStateException > -------------------------------------------------------- > > Key: STORM-2970 > URL: https://issues.apache.org/jira/browse/STORM-2970 > Project: Apache Storm > Issue Type: Bug > Reporter: Ethan Li > Priority: Major > > I ran > {code:java} > storm jar /tmp/storm-loadgen-2.0.0-SNAPSHOT.jar > org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2 > --counters 1 --rate 1000 -c topology.debug=true > {code} > and the topology was not running properly. And some exception showed up in > the worker.log: > {code:java} > 2018-02-23 15:13:29.939 o.a.s.u.Utils Thread-15-spout-executor[7, 7] [ERROR] > Async loop died! > java.lang.RuntimeException: java.lang.IllegalStateException: Queue full > at org.apache.storm.executor.Executor.accept(Executor.java:288) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:309) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.utils.JCQueue.consume(JCQueue.java:290) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.utils.JCQueue.consume(JCQueue.java:281) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:173) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor$2.call(SpoutExecutor.java:163) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.utils.Utils$2.run(Utils.java:352) > [storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] > Caused by: java.lang.IllegalStateException: Queue full > at java.util.AbstractQueue.add(AbstractQueue.java:98) ~[?:1.8.0_131] > at > org.apache.storm.daemon.worker.WorkerTransfer.tryTransferRemote(WorkerTransfer.java:112) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.daemon.worker.WorkerState.tryTransferRemote(WorkerState.java:526) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.ExecutorTransfer.tryTransfer(ExecutorTransfer.java:71) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.daemon.Task.sendUnanchored(Task.java:196) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutOutputCollectorImpl.sendSpoutMsg(SpoutOutputCollectorImpl.java:164) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutOutputCollectorImpl.emit(SpoutOutputCollectorImpl.java:75) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.spout.SpoutOutputCollector.emit(SpoutOutputCollector.java:51) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.loadgen.LoadSpout.fail(LoadSpout.java:135) > ~[stormjar.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor.failSpoutMsg(SpoutExecutor.java:362) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor$1.expire(SpoutExecutor.java:126) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor$1.expire(SpoutExecutor.java:119) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.utils.RotatingMap.rotate(RotatingMap.java:77) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at > org.apache.storm.executor.spout.SpoutExecutor.tupleActionFn(SpoutExecutor.java:298) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > at org.apache.storm.executor.Executor.accept(Executor.java:284) > ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT] > ... 7 more > {code} > More log from the worker.log: > https://gist.github.com/Ethanlm/bcab1289a3813af6d7135aa95b1aaa14 -- This message was sent by Atlassian JIRA (v7.6.3#76005)