[jira] [Comment Edited] (STORM-963) Frozen topology (KafkaSpout + Multilang bolt)

[Alex Sobrino (JIRA)]

    [ 
https://issues.apache.org/jira/browse/STORM-963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14659799#comment-14659799
 ] 

Alex Sobrino edited comment on STORM-963 at 8/6/15 11:02 AM:
-------------------------------------------------------------

Done. This is the result:

{code}
Attaching to process ID 7636, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.45-b02
Deadlock Detection:

No deadlocks found.

Thread 7761: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=100, line=1088 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7760: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7759: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7758: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7757: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7756: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7755: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.Reader.read(char[]) @bci=5, line=140 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer, 
char[]) @bci=8, line=2001 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer) 
@bci=7, line=1980 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.Reader, java.io.Writer) @bci=2, 
line=1957 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.InputStream, java.io.Writer, 
java.nio.charset.Charset) @bci=15, line=1907 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream, 
java.nio.charset.Charset) @bci=11, line=778 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream) @bci=4, line=759 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getErrorsString() @bci=11, line=138 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getProcessTerminationInfoString() @bci=19, 
line=180 (Interpreted frame)
 - backtype.storm.task.ShellBolt.die(java.lang.Throwable) @bci=21, line=280 
(Interpreted frame)
 - backtype.storm.task.ShellBolt.access$400(backtype.storm.task.ShellBolt, 
java.lang.Throwable) @bci=2, line=69 (Interpreted frame)
 - backtype.storm.task.ShellBolt$BoltHeartbeatTimerTask.run() @bci=81, line=305 
(Interpreted frame)
 - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
(Compiled frame)
 - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
frame)
 - 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
 @bci=1, line=180 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
@bci=37, line=294 (Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7754: (state = BLOCKED)
 - java.io.BufferedInputStream.available() @bci=0, line=409 (Compiled frame)
 - backtype.storm.utils.ShellProcess.logErrorStream() @bci=4, line=125 
(Compiled frame)
 - 
backtype.storm.utils.ShellProcess.writeBoltMsg(backtype.storm.multilang.BoltMsg)
 @bci=11, line=108 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=108, line=361 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7753: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7752: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.Reader.read(char[]) @bci=5, line=140 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer, 
char[]) @bci=8, line=2001 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer) 
@bci=7, line=1980 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.Reader, java.io.Writer) @bci=2, 
line=1957 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.InputStream, java.io.Writer, 
java.nio.charset.Charset) @bci=15, line=1907 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream, 
java.nio.charset.Charset) @bci=11, line=778 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream) @bci=4, line=759 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getErrorsString() @bci=11, line=138 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getProcessTerminationInfoString() @bci=19, 
line=180 (Interpreted frame)
 - backtype.storm.task.ShellBolt.die(java.lang.Throwable) @bci=21, line=280 
(Interpreted frame)
 - backtype.storm.task.ShellBolt.access$400(backtype.storm.task.ShellBolt, 
java.lang.Throwable) @bci=2, line=69 (Interpreted frame)
 - backtype.storm.task.ShellBolt$BoltHeartbeatTimerTask.run() @bci=81, line=305 
(Interpreted frame)
 - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
(Compiled frame)
 - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
frame)
 - 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
 @bci=1, line=180 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
@bci=37, line=294 (Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7751: (state = BLOCKED)
 - java.io.BufferedInputStream.available() @bci=0, line=409 (Compiled frame)
 - backtype.storm.utils.ShellProcess.logErrorStream() @bci=4, line=125 
(Compiled frame)
 - 
backtype.storm.utils.ShellProcess.writeBoltMsg(backtype.storm.multilang.BoltMsg)
 @bci=11, line=108 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=108, line=361 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7750: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7749: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7748: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7747: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7746: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7745: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7744: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7743: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7738: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7737: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7736: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7735: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7733: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.poll(long, 
java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=86, line=359 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7732: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.poll(long, 
java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=86, line=359 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7731: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7730: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7729: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7723: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7722: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7637: (state = BLOCKED)


Thread 7719: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7718: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7717: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7716: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.spout.SleepSpoutWaitStrategy.emptyEmit(long) @bci=4, line=36 
(Compiled frame)
 - backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke() 
@bci=1143, line=575 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7715: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Compiled frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_local_fn$fn__6748.invoke(java.lang.Object)
 @bci=180, line=108 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_fn$fn__6755.invoke(java.lang.Object, 
java.lang.Object) @bci=604, line=134 (Compiled frame)
 - 
backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__6489.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=65, line=258 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7714: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7713: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7712: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7711: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7710: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.put(java.lang.Object) @bci=56, 
line=350 (Compiled frame)
 - backtype.storm.task.ShellBolt.execute(backtype.storm.tuple.Tuple) @bci=54, 
line=145 (Compiled frame)
 - 
backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(java.lang.Object,
 java.lang.Object) @bci=354, line=633 (Compiled frame)
 - 
backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=357, line=401 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7709: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7708: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.put(java.lang.Object) @bci=56, 
line=350 (Compiled frame)
 - backtype.storm.task.ShellBolt.execute(backtype.storm.tuple.Tuple) @bci=54, 
line=145 (Compiled frame)
 - 
backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(java.lang.Object,
 java.lang.Object) @bci=354, line=633 (Compiled frame)
 - 
backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=357, line=401 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7707: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7706: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7705: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - org.apache.storm.netty.util.HashedWheelTimer$Worker.waitForNextTick() 
@bci=84, line=483 (Compiled frame)
 - org.apache.storm.netty.util.HashedWheelTimer$Worker.run() @bci=49, line=392 
(Compiled frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7704: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=100, line=1088 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7703: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Interpreted frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_local_fn$fn__6748.invoke(java.lang.Object)
 @bci=180, line=108 (Compiled frame)
 - backtype.storm.messaging.loader$mk_receive_thread$fn__1626$fn__1627.invoke() 
@bci=600, line=49 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7702: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Interpreted frame)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Interpreted frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Interpreted 
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Interpreted 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Interpreted frame)
 - sun.nio.ch.SelectorImpl.select() @bci=2, line=101 (Interpreted frame)
 - 
org.apache.storm.netty.channel.socket.nio.NioServerBoss.select(java.nio.channels.Selector)
 @bci=1, line=163 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioServerBoss.run() @bci=1, 
line=42 (Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7701: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run() @bci=1, 
line=89 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioWorker.run() @bci=1, line=178 
(Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7700: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Compiled frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - backtype.storm.daemon.executor$setup_metrics_BANG_$fn__6499.invoke() 
@bci=80, line=274 (Compiled frame)
 - backtype.storm.timer$schedule_recurring$this__1807.invoke() @bci=7, line=99 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=174, line=50 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7699: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7698: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7697: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7696: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7695: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.NioClientBoss.run() @bci=1, 
line=42 (Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7694: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run() @bci=1, 
line=89 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioWorker.run() @bci=1, line=178 
(Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7693: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.storm.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.storm.curator.framework.imps.CuratorFrameworkImpl$4.call() 
@bci=4, line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7692: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.storm.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7691: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.storm.zookeeper.ClientCnxnSocketNIO.doTransport(int, 
java.util.List, java.util.LinkedList, org.apache.storm.zookeeper.ClientCnxn) 
@bci=6, line=349 (Compiled frame)
 - org.apache.storm.zookeeper.ClientCnxn$SendThread.run() @bci=615, line=1081 
(Interpreted frame)


Thread 7690: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - 
org.apache.storm.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.storm.curator.framework.state.ConnectionStateManager.access$000(org.apache.storm.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.storm.curator.framework.state.ConnectionStateManager$1.call() 
@bci=4, line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7647: (state = BLOCKED)


Thread 7646: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Compiled frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=164 (Compiled frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted 
frame)


Thread 7645: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Object.wait() @bci=2, line=502 (Compiled frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Compiled 
frame)
{code}

It's from a worker process.

Nothing is written inside the logs :-\ And {{kafkaSpout}} offset is not being 
updated in Zookeeper.


was (Author: alexsobrino):
Done. This is the result:

{code}
Attaching to process ID 7636, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.45-b02
Deadlock Detection:

No deadlocks found.

Thread 7761: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=100, line=1088 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7760: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7759: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7758: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7757: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7756: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7755: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.Reader.read(char[]) @bci=5, line=140 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer, 
char[]) @bci=8, line=2001 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer) 
@bci=7, line=1980 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.Reader, java.io.Writer) @bci=2, 
line=1957 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.InputStream, java.io.Writer, 
java.nio.charset.Charset) @bci=15, line=1907 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream, 
java.nio.charset.Charset) @bci=11, line=778 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream) @bci=4, line=759 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getErrorsString() @bci=11, line=138 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getProcessTerminationInfoString() @bci=19, 
line=180 (Interpreted frame)
 - backtype.storm.task.ShellBolt.die(java.lang.Throwable) @bci=21, line=280 
(Interpreted frame)
 - backtype.storm.task.ShellBolt.access$400(backtype.storm.task.ShellBolt, 
java.lang.Throwable) @bci=2, line=69 (Interpreted frame)
 - backtype.storm.task.ShellBolt$BoltHeartbeatTimerTask.run() @bci=81, line=305 
(Interpreted frame)
 - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
(Compiled frame)
 - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
frame)
 - 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
 @bci=1, line=180 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
@bci=37, line=294 (Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7754: (state = BLOCKED)
 - java.io.BufferedInputStream.available() @bci=0, line=409 (Compiled frame)
 - backtype.storm.utils.ShellProcess.logErrorStream() @bci=4, line=125 
(Compiled frame)
 - 
backtype.storm.utils.ShellProcess.writeBoltMsg(backtype.storm.multilang.BoltMsg)
 @bci=11, line=108 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=108, line=361 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7753: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7752: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.Reader.read(char[]) @bci=5, line=140 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer, 
char[]) @bci=8, line=2001 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copyLarge(java.io.Reader, java.io.Writer) 
@bci=7, line=1980 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.Reader, java.io.Writer) @bci=2, 
line=1957 (Interpreted frame)
 - org.apache.commons.io.IOUtils.copy(java.io.InputStream, java.io.Writer, 
java.nio.charset.Charset) @bci=15, line=1907 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream, 
java.nio.charset.Charset) @bci=11, line=778 (Interpreted frame)
 - org.apache.commons.io.IOUtils.toString(java.io.InputStream) @bci=4, line=759 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getErrorsString() @bci=11, line=138 
(Interpreted frame)
 - backtype.storm.utils.ShellProcess.getProcessTerminationInfoString() @bci=19, 
line=180 (Interpreted frame)
 - backtype.storm.task.ShellBolt.die(java.lang.Throwable) @bci=21, line=280 
(Interpreted frame)
 - backtype.storm.task.ShellBolt.access$400(backtype.storm.task.ShellBolt, 
java.lang.Throwable) @bci=2, line=69 (Interpreted frame)
 - backtype.storm.task.ShellBolt$BoltHeartbeatTimerTask.run() @bci=81, line=305 
(Interpreted frame)
 - java.util.concurrent.Executors$RunnableAdapter.call() @bci=4, line=511 
(Compiled frame)
 - java.util.concurrent.FutureTask.runAndReset() @bci=47, line=308 (Compiled 
frame)
 - 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
 @bci=1, line=180 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run() 
@bci=37, line=294 (Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7751: (state = BLOCKED)
 - java.io.BufferedInputStream.available() @bci=0, line=409 (Compiled frame)
 - backtype.storm.utils.ShellProcess.logErrorStream() @bci=4, line=125 
(Compiled frame)
 - 
backtype.storm.utils.ShellProcess.writeBoltMsg(backtype.storm.multilang.BoltMsg)
 @bci=11, line=108 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=108, line=361 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7750: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7749: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7748: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7747: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7746: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7745: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call() @bci=4, 
line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7744: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7743: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(int, java.util.List, 
java.util.LinkedList, org.apache.zookeeper.ClientCnxn) @bci=6, line=349 
(Compiled frame)
 - org.apache.zookeeper.ClientCnxn$SendThread.run() @bci=604, line=1081 
(Interpreted frame)


Thread 7738: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.curator.framework.state.ConnectionStateManager.access$000(org.apache.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.curator.framework.state.ConnectionStateManager$1.call() @bci=4, 
line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7737: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7736: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7735: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7733: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.poll(long, 
java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=86, line=359 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7732: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.poll(long, 
java.util.concurrent.TimeUnit) @bci=62, line=467 (Compiled frame)
 - backtype.storm.task.ShellBolt$BoltWriterRunnable.run() @bci=86, line=359 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7731: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7730: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Compiled frame; 
information may be imprecise)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Compiled 
frame)
 - java.io.BufferedInputStream.read1(byte[], int, int) @bci=39, line=284 
(Compiled frame)
 - java.io.BufferedInputStream.read(byte[], int, int) @bci=49, line=345 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.readBytes() @bci=135, line=284 (Compiled frame)
 - sun.nio.cs.StreamDecoder.implRead(char[], int, int) @bci=112, line=326 
(Compiled frame)
 - sun.nio.cs.StreamDecoder.read(char[], int, int) @bci=180, line=178 (Compiled 
frame)
 - java.io.InputStreamReader.read(char[], int, int) @bci=7, line=184 (Compiled 
frame)
 - java.io.BufferedReader.fill() @bci=145, line=161 (Compiled frame)
 - java.io.BufferedReader.readLine(boolean) @bci=44, line=324 (Compiled frame)
 - java.io.BufferedReader.readLine() @bci=2, line=389 (Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readString() @bci=12, line=179 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readMessage() @bci=1, line=166 
(Compiled frame)
 - backtype.storm.multilang.JsonSerializer.readShellMsg() @bci=1, line=100 
(Compiled frame)
 - backtype.storm.utils.ShellProcess.readShellMsg() @bci=4, line=99 (Compiled 
frame)
 - backtype.storm.task.ShellBolt$BoltReaderRunnable.run() @bci=17, line=318 
(Compiled frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7729: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7723: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7722: (state = IN_NATIVE)
 - java.lang.UNIXProcess.waitForProcessExit(int) @bci=0 (Interpreted frame)
 - java.lang.UNIXProcess.lambda$initStreams$266() @bci=5, line=290 (Interpreted 
frame)
 - java.lang.UNIXProcess$$Lambda$9.run() @bci=4 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7637: (state = BLOCKED)


Thread 7719: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7718: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7717: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7716: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.spout.SleepSpoutWaitStrategy.emptyEmit(long) @bci=4, line=36 
(Compiled frame)
 - backtype.storm.daemon.executor$fn__6579$fn__6594$fn__6623.invoke() 
@bci=1143, line=575 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7715: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Compiled frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_local_fn$fn__6748.invoke(java.lang.Object)
 @bci=180, line=108 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_fn$fn__6755.invoke(java.lang.Object, 
java.lang.Object) @bci=604, line=134 (Compiled frame)
 - 
backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__6489.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=65, line=258 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7714: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7713: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7712: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7711: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7710: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.put(java.lang.Object) @bci=56, 
line=350 (Compiled frame)
 - backtype.storm.task.ShellBolt.execute(backtype.storm.tuple.Tuple) @bci=54, 
line=145 (Compiled frame)
 - 
backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(java.lang.Object,
 java.lang.Object) @bci=354, line=633 (Compiled frame)
 - 
backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=357, line=401 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7709: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7708: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.LinkedBlockingQueue.put(java.lang.Object) @bci=56, 
line=350 (Compiled frame)
 - backtype.storm.task.ShellBolt.execute(backtype.storm.tuple.Tuple) @bci=54, 
line=145 (Compiled frame)
 - 
backtype.storm.daemon.executor$fn__6647$tuple_action_fn__6649.invoke(java.lang.Object,
 java.lang.Object) @bci=354, line=633 (Compiled frame)
 - 
backtype.storm.daemon.executor$mk_task_receiver$fn__6570.invoke(java.lang.Object,
 java.lang.Object, java.lang.Object) @bci=357, line=401 (Compiled frame)
 - 
backtype.storm.disruptor$clojure_handler$reify__1605.onEvent(java.lang.Object, 
long, boolean) @bci=28, line=58 (Compiled frame)
 - backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(long, 
com.lmax.disruptor.EventHandler) @bci=125, line=125 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=39, line=99 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.daemon.executor$fn__6647$fn__6659$fn__6706.invoke() @bci=17, 
line=748 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7707: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(long,
 java.util.concurrent.TimeUnit) @bci=97, line=2163 (Compiled frame)
 - com.lmax.disruptor.BlockingWaitStrategy.waitFor(long, 
com.lmax.disruptor.Sequence, com.lmax.disruptor.Sequence[], 
com.lmax.disruptor.SequenceBarrier, long, java.util.concurrent.TimeUnit) 
@bci=58, line=87 (Compiled frame)
 - com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(long, long, 
java.util.concurrent.TimeUnit) @bci=21, line=54 (Compiled frame)
 - 
backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(com.lmax.disruptor.EventHandler)
 @bci=21, line=97 (Compiled frame)
 - 
backtype.storm.disruptor$consume_batch_when_available.invoke(java.lang.Object, 
java.lang.Object) @bci=12, line=80 (Compiled frame)
 - backtype.storm.disruptor$consume_loop_STAR_$fn__1618.invoke() @bci=17, 
line=94 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7706: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(java.lang.Object, long) 
@bci=20, line=215 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(long)
 @bci=78, line=2078 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=124, line=1093 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7705: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - org.apache.storm.netty.util.HashedWheelTimer$Worker.waitForNextTick() 
@bci=84, line=483 (Compiled frame)
 - org.apache.storm.netty.util.HashedWheelTimer$Worker.run() @bci=49, line=392 
(Compiled frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7704: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Compiled frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=100, line=1088 (Compiled frame)
 - java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take() 
@bci=1, line=809 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor.getTask() @bci=149, line=1067 
(Compiled frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=26, line=1127 (Compiled frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7703: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Interpreted frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - 
backtype.storm.daemon.worker$mk_transfer_local_fn$fn__6748.invoke(java.lang.Object)
 @bci=180, line=108 (Compiled frame)
 - backtype.storm.messaging.loader$mk_receive_thread$fn__1626$fn__1627.invoke() 
@bci=600, line=49 (Compiled frame)
 - backtype.storm.util$async_loop$fn__459.invoke() @bci=39, line=463 (Compiled 
frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7702: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Interpreted frame)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Interpreted frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Interpreted 
frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Interpreted 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Interpreted frame)
 - sun.nio.ch.SelectorImpl.select() @bci=2, line=101 (Interpreted frame)
 - 
org.apache.storm.netty.channel.socket.nio.NioServerBoss.select(java.nio.channels.Selector)
 @bci=1, line=163 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioServerBoss.run() @bci=1, 
line=42 (Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7701: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run() @bci=1, 
line=89 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioWorker.run() @bci=1, line=178 
(Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7700: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may 
be imprecise)
 - java.util.concurrent.locks.LockSupport.parkNanos(long) @bci=11, line=338 
(Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.waitForFreeSlotAt(long, 
com.lmax.disruptor.Sequence[], com.lmax.disruptor.util.MutableLong) @bci=34, 
line=99 (Compiled frame)
 - 
com.lmax.disruptor.AbstractMultithreadedClaimStrategy.incrementAndGet(com.lmax.disruptor.Sequence[])
 @bci=21, line=49 (Compiled frame)
 - com.lmax.disruptor.Sequencer.next() @bci=26, line=127 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publishDirect(java.lang.Object, boolean) 
@bci=8, line=174 (Interpreted frame)
 - backtype.storm.utils.DisruptorQueue.publish(java.lang.Object, boolean) 
@bci=69, line=167 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object, 
java.lang.Object) @bci=18, line=66 (Compiled frame)
 - backtype.storm.disruptor$publish.invoke(java.lang.Object, java.lang.Object) 
@bci=18, line=68 (Compiled frame)
 - backtype.storm.daemon.executor$setup_metrics_BANG_$fn__6499.invoke() 
@bci=80, line=274 (Compiled frame)
 - backtype.storm.timer$schedule_recurring$this__1807.invoke() @bci=7, line=99 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=174, line=50 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7699: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7698: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7697: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7696: (state = BLOCKED)
 - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - backtype.storm.utils.Time.sleepUntil(long) @bci=173, line=76 (Compiled frame)
 - backtype.storm.utils.Time.sleep(long) @bci=5, line=81 (Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790$fn__1791.invoke() @bci=220, line=57 
(Compiled frame)
 - backtype.storm.timer$mk_timer$fn__1790.invoke() @bci=54, line=42 
(Interpreted frame)
 - clojure.lang.AFn.run() @bci=1, line=24 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7695: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.NioClientBoss.run() @bci=1, 
line=42 (Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7694: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector)
 @bci=4, line=68 (Compiled frame)
 - 
org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector)
 @bci=1, line=415 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run() @bci=56, 
line=212 (Compiled frame)
 - org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run() @bci=1, 
line=89 (Interpreted frame)
 - org.apache.storm.netty.channel.socket.nio.NioWorker.run() @bci=1, line=178 
(Interpreted frame)
 - org.apache.storm.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 
(Interpreted frame)
 - org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, 
line=42 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7693: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=28, line=211 (Interpreted frame)
 - java.util.concurrent.DelayQueue.take() @bci=1, line=70 (Interpreted frame)
 - 
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop()
 @bci=10, line=781 (Interpreted frame)
 - 
org.apache.storm.curator.framework.imps.CuratorFrameworkImpl.access$400(org.apache.storm.curator.framework.imps.CuratorFrameworkImpl)
 @bci=1, line=57 (Interpreted frame)
 - org.apache.storm.curator.framework.imps.CuratorFrameworkImpl$4.call() 
@bci=4, line=275 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7692: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.LinkedBlockingQueue.take() @bci=29, line=442 
(Interpreted frame)
 - org.apache.storm.zookeeper.ClientCnxn$EventThread.run() @bci=9, line=494 
(Interpreted frame)


Thread 7691: (state = IN_NATIVE)
 - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 
(Compiled frame; information may be imprecise)
 - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=269 (Compiled frame)
 - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=79 (Compiled frame)
 - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=86 (Compiled 
frame)
 - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=97 (Compiled frame)
 - org.apache.storm.zookeeper.ClientCnxnSocketNIO.doTransport(int, 
java.util.List, java.util.LinkedList, org.apache.storm.zookeeper.ClientCnxn) 
@bci=6, line=349 (Compiled frame)
 - org.apache.storm.zookeeper.ClientCnxn$SendThread.run() @bci=615, line=1081 
(Interpreted frame)


Thread 7690: (state = BLOCKED)
 - sun.misc.Unsafe.park(boolean, long) @bci=0 (Interpreted frame)
 - java.util.concurrent.locks.LockSupport.park(java.lang.Object) @bci=14, 
line=175 (Interpreted frame)
 - 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await() 
@bci=42, line=2039 (Interpreted frame)
 - java.util.concurrent.ArrayBlockingQueue.take() @bci=20, line=403 
(Interpreted frame)
 - 
org.apache.storm.curator.framework.state.ConnectionStateManager.processEvents() 
@bci=13, line=208 (Interpreted frame)
 - 
org.apache.storm.curator.framework.state.ConnectionStateManager.access$000(org.apache.storm.curator.framework.state.ConnectionStateManager)
 @bci=1, line=42 (Interpreted frame)
 - org.apache.storm.curator.framework.state.ConnectionStateManager$1.call() 
@bci=4, line=110 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=42, line=266 (Interpreted frame)
 - 
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker)
 @bci=95, line=1142 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=617 
(Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame)


Thread 7647: (state = BLOCKED)


Thread 7646: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=143 (Compiled frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=164 (Compiled frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted 
frame)


Thread 7645: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be 
imprecise)
 - java.lang.Object.wait() @bci=2, line=502 (Compiled frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Compiled 
frame)
{code}

Nothing is written inside the logs :-\ And {{kafkaSpout}} offset is not being 
updated in Zookeeper.

> Frozen topology (KafkaSpout + Multilang bolt)
> ---------------------------------------------
>
>                 Key: STORM-963
>                 URL: https://issues.apache.org/jira/browse/STORM-963
>             Project: Apache Storm
>          Issue Type: Bug
>    Affects Versions: 0.9.4, 0.9.5, 0.9.6
>         Environment: - VMware ESX 5.5 
> - Ubuntu Server 14.04 LTS (kernel 3.16.0-41-generic)
> - Java (TM) SE Runtime Environment (build 1.8.0_45-b14)
> - Python 2.7.6 (default, Jun 22 2015, 17:58:13)
> - Zookeeper 3.4.6
>            Reporter: Alex Sobrino
>              Labels: multilang
>
> Hi,
> We've got a pretty simple topology running with Storm 0.9.5 (tried also with 
> 0.9.4 and 0.9.6-INCUBATING) in a 3 machine cluster:
> {code}kafkaSpout (3) -----> processBolt (12){code}
> Some info:
> - kafkaSpout reads from a topic with 3 partitions and 2 replications
> - processBolt iterates throught the message and saves the results in MongoDB
> - processBolt is implemented in Python and has a storm.log("I'm doing 
> something") just to add a simple debug message in the logs
> - The messages can be quite big (~25-40 MB) and are in JSON format
> - The kafka topic has a retention of 2 hours
> - We use the same ZooKeeper cluster to both Kafka and Storm
> The topology gets frozen after several hours (not days) running. We don't see 
> any message in the logs... In fact, the periodic message from s.k.KafkaUtils 
> and s.k.ZkCoordinator disapears. As you can imagine, the message from the 
> Bolt also dissapears. Logs are copy/pasted further on. If we redeploy the 
> topology everything starts to work again until it becomes frozen again.
> Our kafkaSpout config is:
> {code}
> ZkHosts zkHosts = new ZkHosts("zkhost01:2181,zkhost02:2181,zkhost03:2181");
> SpoutConfig kafkaConfig = new SpoutConfig(zkHosts, "topic", 
> "/topic/ourclientid", "ourclientid");
> kafkaConfig.scheme = new SchemeAsMultiScheme(new StringScheme());
> kafkaConfig.fetchSizeBytes = 50*1024*1024;
> kafkaConfig.bufferSizeBytes = 50*1024*1024;
> {code}
> We've also tried setting the following options
> {code}
> kafkaConfig.forceFromStart = true;
> kafkaConfig.startOffsetTime = kafka.api.OffsetRequest.EarliestTime(); // Also 
> with kafka.api.OffsetRequest.LatestTime();
> kafkaConfig.useStartOffsetTimeIfOffsetOutOfRange = true;
> {code}
> Right now the topology is running without acking the messages since there's a 
> bug in kafkaSpout with failed messages and deleted offsets in Kafka.
> This is what can be seen in the logs in one of the workers:
> {code}
> 2015-07-23T12:37:38.008+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:37:39.079+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:37:51.013+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:37:51.091+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:38:02.684+0200 s.k.ZkCoordinator [INFO] Task [2/3] Refreshing 
> partition manager connections
> 2015-07-23T12:38:02.687+0200 s.k.DynamicBrokersReader [INFO] Read partition 
> info from zookeeper: GlobalPartitionInformation{partitionMap={0=kafka1:9092, 
> 1=kafka2:9092, 2=kafka3:9092}}
> 2015-07-23T12:38:02.687+0200 s.k.KafkaUtils [INFO] Task [2/3] assigned 
> [Partition{host=kafka2, partition=1}]
> 2015-07-23T12:38:02.687+0200 s.k.ZkCoordinator [INFO] Task [2/3] Deleted 
> partition managers: []
> 2015-07-23T12:38:02.687+0200 s.k.ZkCoordinator [INFO] Task [2/3] New 
> partition managers: []
> 2015-07-23T12:38:02.687+0200 s.k.ZkCoordinator [INFO] Task [2/3] Finished 
> refreshing
> 2015-07-23T12:38:09.012+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:38:41.878+0200 b.s.t.ShellBolt [INFO] ShellLog pid:28364, 
> name:processBolt I'm doing something
> 2015-07-23T12:39:02.688+0200 s.k.ZkCoordinator [INFO] Task [2/3] Refreshing 
> partition manager connections
> 2015-07-23T12:39:02.691+0200 s.k.DynamicBrokersReader [INFO] Read partition 
> info from zookeeper: GlobalPartitionInformation{partitionMap={0=kafka1:9092, 
> 1=kafka2:9092, 2=kafka3:9092}}
> 2015-07-23T12:39:02.691+0200 s.k.KafkaUtils [INFO] Task [2/3] assigned 
> [Partition{host=kafka2:9092, partition=1}]
> 2015-07-23T12:39:02.691+0200 s.k.ZkCoordinator [INFO] Task [2/3] Deleted 
> partition managers: []
> 2015-07-23T12:39:02.691+0200 s.k.ZkCoordinator [INFO] Task [2/3] New 
> partition managers: []
> 2015-07-23T12:39:02.691+0200 s.k.ZkCoordinator [INFO] Task [2/3] Finished 
> refreshing
> 2015-07-23T12:40:02.692+0200 s.k.ZkCoordinator [INFO] Task [2/3] Refreshing 
> partition manager connections
> 2015-07-23T12:40:02.695+0200 s.k.DynamicBrokersReader [INFO] Read partition 
> info from zookeeper: GlobalPartitionInformation{partitionMap={0=kafka1:9092, 
> 1=kafka2:9092, 2=kafka3:9092}}
> 2015-07-23T12:40:02.695+0200 s.k.KafkaUtils [INFO] Task [2/3] assigned 
> [Partition{host=kafka2:9092, partition=1}]
> 2015-07-23T12:40:02.695+0200 s.k.ZkCoordinator [INFO] Task [2/3] Deleted 
> partition managers: []
> 2015-07-23T12:40:02.695+0200 s.k.ZkCoordinator [INFO] Task [2/3] New 
> partition managers: []
> 2015-07-23T12:40:02.695+0200 s.k.ZkCoordinator [INFO] Task [2/3] Finished 
> refreshing
> 2015-07-23T12:41:02.696+0200 s.k.ZkCoordinator [INFO] Task [2/3] Refreshing 
> partition manager connections
> 2015-07-23T12:41:02.699+0200 s.k.DynamicBrokersReader [INFO] Read partition 
> info from zookeeper: GlobalPartitionInformation{partitionMap={0=kafka1:9092, 
> 1=kafka2:9092, 2=kafka3:9092}}
> 2015-07-23T12:41:02.699+0200 s.k.KafkaUtils [INFO] Task [2/3] assigned 
> [Partition{host=kafka2:9092, partition=1}]
> 2015-07-23T12:41:02.699+0200 s.k.ZkCoordinator [INFO] Task [2/3] Deleted 
> partition managers: []
> 2015-07-23T12:41:02.699+0200 s.k.ZkCoordinator [INFO] Task [2/3] New 
> partition managers: []
> 2015-07-23T12:41:02.699+0200 s.k.ZkCoordinator [INFO] Task [2/3] Finished 
> refreshing
> 2015-07-23T12:42:02.735+0200 s.k.ZkCoordinator [INFO] Task [2/3] Refreshing 
> partition manager connections
> 2015-07-23T12:42:02.737+0200 s.k.DynamicBrokersReader [INFO] Read partition 
> info from zookeeper: GlobalPartitionInformation{partitionMap={0=kafka1:9092, 
> 1=kafka2:9092, 2=kafka3:9092}}
> 2015-07-23T12:42:02.737+0200 s.k.KafkaUtils [INFO] Task [2/3] assigned 
> [Partition{host=kafka2:9092, partition=1}]
> 2015-07-23T12:42:02.737+0200 s.k.ZkCoordinator [INFO] Task [2/3] Deleted 
> partition managers: []
> 2015-07-23T12:42:02.737+0200 s.k.ZkCoordinator [INFO] Task [2/3] New 
> partition managers: []
> 2015-07-23T12:42:02.737+0200 s.k.ZkCoordinator [INFO] Task [2/3] Finished 
> refreshing
> {code}
> and then it becomes frozen. Nothing is written into the nimbus log. We've 
> checked the offsets in ZooKeeper and they're not updated:
> {code}
> {"topology":{"id":"218e58a5-6bfb-4b32-ae89-f3afa19306e1","name":"our-topology"},"offset":12047144,"partition":1,"broker":{"host":"kafka2","port":9092},"topic":"topic"}
> cZxid = 0x100028958
> ctime = Wed Jul 01 12:22:36 CEST 2015
> mZxid = 0x100518527
> mtime = Thu Jul 23 12:42:41 CEST 2015
> pZxid = 0x100028958
> cversion = 0
> dataVersion = 446913
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 183
> numChildren = 0
> {code}
> Any ideas of what we could be missing?
> PS: This was sent to the Storm user's mailing list and got 0 replies :\



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)