Deadlock in ActiveMQ Failover Worker
------------------------------------
Key: AMQ-897
URL: https://issues.apache.org/activemq/browse/AMQ-897
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 4.0.1
Environment: Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006
i686 i686 i386 GNU/Linux
Dell PE 850 with 4 GB RAM and 1x dual core 3GHz Pentium D
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)
Reporter: Michael P Lum
Full thread dump Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode, sharing):
"ActiveMQ Transport: tcp://XXX/XXX:61616" prio=1 tid=0x08091510 nid=0x2f4c
waiting for monitor entry [0xafdc9000..0xafdca130]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportInterupted(DemandForwardingBridgeSupport.java:134)
- waiting to lock <0x3654b290> (a
org.apache.activemq.network.DemandForwardingBridgeSupport$2)
at
org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98)
at
org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98)
at
org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:223)
at
org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:53)
at
org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:111)
at
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94)
at
org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120)
at
org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149)
at
org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:100)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:156)
at java.lang.Thread.run(Thread.java:595)
"Thread-7" daemon prio=1 tid=0x085263d8 nid=0x2f4b waiting for monitor entry
[0xafe4b000..0xafe4b1b0]
at
org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:335)
- waiting to lock <0x365793b0> (a java.lang.Object)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x36579610> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startRemoteBridge(DemandForwardingBridgeSupport.java:235)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$4.run(DemandForwardingBridgeSupport.java:197)
"ActiveMQ Transport: tcp:///127.0.0.1:40496" daemon prio=1 tid=0x0851c070
nid=0x2f3c runnable [0xafac3000..0xafac3fb0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 9847682" daemon prio=1 tid=0x0851bc30
nid=0x2f3b in Object.wait() [0xafb44000..0xafb45030]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x37849a98> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///127.0.0.1:40386" daemon prio=1 tid=0x0855b060
nid=0x2b0e runnable [0xb06db000..0xb06dc1b0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 11854491" daemon prio=1 tid=0x085110d8
nid=0x2b0d in Object.wait() [0xafbc5000..0xafbc5e30]
at java.lang.Object.wait(Native Method)
- waiting on <0x36406180> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36406180> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///127.0.0.1:40385" daemon prio=1 tid=0x0855bfe0
nid=0x2b09 runnable [0xb0af6000..0xb0af6eb0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 14732323" daemon prio=1 tid=0x085822e8
nid=0x2b07 in Object.wait() [0xb0bf8000..0xb0bf8f30]
at java.lang.Object.wait(Native Method)
- waiting on <0x36408358> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36408358> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///127.0.0.1:40381" daemon prio=1 tid=0x0855aec0
nid=0x2af1 runnable [0xb0a75000..0xb0a75fb0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 2804837" daemon prio=1 tid=0x0850ad00
nid=0x2af0 in Object.wait() [0xb0cfc000..0xb0cfd030]
at java.lang.Object.wait(Native Method)
- waiting on <0x3640a530> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x3640a530> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///10.7.93.117:47626" daemon prio=1 tid=0x08092be8
nid=0x2ad9 runnable [0xafcc7000..0xafcc81b0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 5165021" daemon prio=1 tid=0x08092600
nid=0x2ad8 in Object.wait() [0xafd48000..0xafd48e30]
at java.lang.Object.wait(Native Method)
- waiting on <0x3640c708> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x3640c708> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///XXX:49524" daemon prio=1 tid=0x08094ef0 nid=0x2a9d
runnable [0xafecb000..0xafecbfb0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 10884088" daemon prio=1 tid=0x08093c20
nid=0x2a9c in Object.wait() [0xaff4c000..0xaff4d030]
at java.lang.Object.wait(Native Method)
- waiting on <0x3640e8e0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x3640e8e0> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport: tcp:///XXX:49523" daemon prio=1 tid=0x08093e90 nid=0x2a9b
runnable [0xaffcd000..0xaffce0b0]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 30810736" daemon prio=1 tid=0x08526ca8
nid=0x2a9a in Object.wait() [0xb004e000..0xb004f130]
at java.lang.Object.wait(Native Method)
- waiting on <0x36410ab8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36410ab8> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Scheduler" daemon prio=1 tid=0x08524730 nid=0x2a99 waiting for
monitor entry [0xb00cf000..0xb00d01b0]
at
org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportInterupted(DemandForwardingBridgeSupport.java:134)
- waiting to lock <0x3654b290> (a
org.apache.activemq.network.DemandForwardingBridgeSupport$2)
at
org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98)
at
org.apache.activemq.transport.TransportFilter.transportInterupted(TransportFilter.java:98)
at
org.apache.activemq.transport.failover.FailoverTransport.handleTransportFailure(FailoverTransport.java:223)
at
org.apache.activemq.transport.failover.FailoverTransport.access$300(FailoverTransport.java:53)
at
org.apache.activemq.transport.failover.FailoverTransport$1.onException(FailoverTransport.java:111)
at
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:94)
at
org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:120)
at
org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:149)
at
org.apache.activemq.transport.InactivityMonitor.readCheck(InactivityMonitor.java:101)
at
org.apache.activemq.transport.InactivityMonitor.access$000(InactivityMonitor.java:35)
at
org.apache.activemq.transport.InactivityMonitor$1.run(InactivityMonitor.java:51)
at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:431)
at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.runAndReset(FutureTask.java:198)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:189)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:213)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport: tcp:///XXX:49522" daemon prio=1 tid=0x08520f28 nid=0x2a98
runnable [0xb0150000..0xb0150e30]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 9363478" daemon prio=1 tid=0x0851d400
nid=0x2a97 in Object.wait() [0xb01d1000..0xb01d1eb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x36412dd0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36412dd0> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Scheduler" daemon prio=1 tid=0x08527a88 nid=0x2a96 in Object.wait()
[0xb0252000..0xb0252f30]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:432)
at
edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301)
at
edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)
- locked <0x36410b38> (a java.lang.Object)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=1 tid=0x08527360 nid=0x2a95 in Object.wait()
[0xb02d3000..0xb02d3fb0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:432)
at
edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301)
at
edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)
- locked <0x36410b38> (a java.lang.Object)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport: tcp:///XXX:49521" daemon prio=1 tid=0x0851fc98 nid=0x2a94
runnable [0xb0354000..0xb0355030]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Connection Dispatcher: 15779934" daemon prio=1 tid=0x0851f950
nid=0x2a93 in Object.wait() [0xb04d7000..0xb04d80b0]
at java.lang.Object.wait(Native Method)
- waiting on <0x364150a8> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x364150a8> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Scheduler" daemon prio=1 tid=0x083acc68 nid=0x2a90 in Object.wait()
[0xb03d5000..0xb03d61b0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:432)
at
edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301)
at
edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)
- locked <0x36410b38> (a java.lang.Object)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Scheduler" daemon prio=1 tid=0x083ac700 nid=0x2a8f in Object.wait()
[0xb0456000..0xb0456e30]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:432)
at
edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:301)
at
edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)
- locked <0x36410b38> (a java.lang.Object)
at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:590)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:477)
at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:674)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport: tcp://XXX/10.7.93.117:61616" prio=1 tid=0x083ad3d0
nid=0x2a8d runnable [0xb0558000..0xb0558f30]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:48)
at
org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:55)
at java.io.DataInputStream.readInt(DataInputStream.java:353)
at
org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:274)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:142)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Failover Worker: 2891371" daemon prio=1 tid=0xb0e0f1c0 nid=0x2a8c in
Object.wait() [0xb05d9000..0xb05d9fb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x36417380> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36417380> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Connection Dispatcher: 21350489" daemon prio=1 tid=0xb0e0d788
nid=0x2a8b in Object.wait() [0xb065a000..0xb065b030]
at java.lang.Object.wait(Native Method)
- waiting on <0x36417400> (a java.lang.Object)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:91)
- locked <0x36417400> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Failover Worker: 5862378" daemon prio=1 tid=0xb0e0ea00 nid=0x2a89
waiting for monitor entry [0xb076f000..0xb0770130]
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:43)
- waiting to lock <0x36579610> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportResumed(DemandForwardingBridgeSupport.java:164)
- locked <0x3654b290> (a
org.apache.activemq.network.DemandForwardingBridgeSupport$2)
at
org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102)
at
org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102)
at
org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:175)
- locked <0x365793b0> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Connection Dispatcher: 12115695" daemon prio=1 tid=0xb0e0a718
nid=0x2a88 in Object.wait() [0xb07f0000..0xb07f11b0]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at
edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch.await(CountDownLatch.java:179)
- locked <0x382fb880> (a
edu.emory.mathcs.backport.java.util.concurrent.CountDownLatch)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.waitStarted(DemandForwardingBridgeSupport.java:836)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:462)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$1.onCommand(DemandForwardingBridgeSupport.java:116)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:97)
at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:63)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:76)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x36417648> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:211)
at
org.apache.activemq.broker.AbstractConnection.processDispatch(AbstractConnection.java:581)
at
org.apache.activemq.broker.AbstractConnection.iterate(AbstractConnection.java:597)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"ActiveMQ Transport Status Monitor: vm://localhost" daemon prio=1
tid=0xb0e08090 nid=0x2a87 waiting on condition [0xb0871000..0xb0871e30]
at java.lang.Thread.sleep(Native Method)
at
org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport Status Monitor: stomp" daemon prio=1 tid=0xb0e01fb0
nid=0x2a86 waiting on condition [0xb08f2000..0xb08f2eb0]
at java.lang.Thread.sleep(Native Method)
at
org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport Server: tcp://XXX:61613?wireFormat=stomp" daemon prio=1
tid=0xb0e01ab0 nid=0x2a85 runnable [0xb0973000..0xb0973f30]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x36417840> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:123)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport Status Monitor: default" daemon prio=1 tid=0xb0e00f48
nid=0x2a84 waiting on condition [0xb09f4000..0xb09f4fb0]
at java.lang.Thread.sleep(Native Method)
at
org.apache.activemq.broker.TransportStatusDetector.run(TransportStatusDetector.java:98)
at java.lang.Thread.run(Thread.java:595)
"ActiveMQ Transport Server: tcp://XXX:61616" daemon prio=1 tid=0xb0e01630
nid=0x2a83 runnable [0xb1061000..0xb1062030]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x36417a10> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
org.apache.activemq.transport.tcp.TcpTransportServer.run(TcpTransportServer.java:123)
at java.lang.Thread.run(Thread.java:595)
"RMI LeaseChecker" daemon prio=1 tid=0x08586318 nid=0x2a80 waiting on condition
[0xb0b78000..0xb0b781b0]
at java.lang.Thread.sleep(Native Method)
at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:310)
at java.lang.Thread.run(Thread.java:595)
"RMI RenewClean-[XXX:40355]" daemon prio=1 tid=0x0850cd08 nid=0x2a7e in
Object.wait() [0xb0c79000..0xb0c79eb0]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x36417bc0> (a java.lang.ref.ReferenceQueue$Lock)
at
sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:500)
at java.lang.Thread.run(Thread.java:595)
"GC Daemon" daemon prio=1 tid=0x0857f9d0 nid=0x2a7c in Object.wait()
[0xb0d7d000..0xb0d7dfb0]
at java.lang.Object.wait(Native Method)
- waiting on <0x364035b8> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:100)
- locked <0x364035b8> (a sun.misc.GC$LatencyLock)
"RMI Reaper" prio=1 tid=0x0857ea58 nid=0x2a7b in Object.wait()
[0xb0dfe000..0xb0dff030]
at java.lang.Object.wait(Native Method)
- waiting on <0x363ffae8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x363ffae8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:336)
at java.lang.Thread.run(Thread.java:595)
"RMI TCP Accept-0" daemon prio=1 tid=0x0857b1c8 nid=0x2a7a runnable
[0xb0fe0000..0xb0fe10b0]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x36403220> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
at java.lang.Thread.run(Thread.java:595)
"RMI TCP Accept-1099" daemon prio=1 tid=0x084fae08 nid=0x2a78 runnable
[0xb10e2000..0xb10e31b0]
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
- locked <0x36400498> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at sun.rmi.transport.tcp.TCPTransport.run(TCPTransport.java:334)
at java.lang.Thread.run(Thread.java:595)
"Timer-0" daemon prio=1 tid=0x084fa780 nid=0x2a77 in Object.wait()
[0xb1163000..0xb1163e30]
at java.lang.Object.wait(Native Method)
- waiting on <0x363ffc10> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:474)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x363ffc10> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=1 tid=0x080a3688 nid=0x2a75 runnable
[0x00000000..0x00000000]
"CompilerThread0" daemon prio=1 tid=0x080a20e0 nid=0x2a74 waiting on condition
[0x00000000..0xb1e52998]
"Signal Dispatcher" daemon prio=1 tid=0x080a1168 nid=0x2a73 runnable
[0x00000000..0x00000000]
"Finalizer" daemon prio=1 tid=0x0809a3d8 nid=0x2a72 in Object.wait()
[0xb2154000..0xb21550b0]
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0x36390738> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=1 tid=0x08099680 nid=0x2a71 in Object.wait()
[0xb21d6000..0xb21d6130]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:474)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x36390680> (a java.lang.ref.Reference$Lock)
"main" prio=1 tid=0x0805cac8 nid=0x2a68 in Object.wait()
[0xbfffc000..0xbfffd1a8]
at java.lang.Object.wait(Native Method)
- waiting on <0x36417cd0> (a [Z)
at java.lang.Object.wait(Object.java:474)
at
org.apache.activemq.console.command.StartCommand.waitForShutdown(StartCommand.java:106)
- locked <0x36417cd0> (a [Z)
at
org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:67)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:49)
at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:81)
at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:49)
at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.activemq.console.Main.runTaskClass(Main.java:135)
at org.apache.activemq.console.Main.main(Main.java:67)
"VM Thread" prio=1 tid=0x08096ae8 nid=0x2a70 runnable
"VM Periodic Task Thread" prio=1 tid=0x080a4b70 nid=0x2a76 waiting on condition
Found one Java-level deadlock:
=============================
"ActiveMQ Failover Worker: 5862378":
waiting to lock monitor 0x0809c22c (object 0x36579610, a java.lang.Object),
which is held by "Thread-7"
"Thread-7":
waiting to lock monitor 0x0809c2ac (object 0x365793b0, a java.lang.Object),
which is held by "ActiveMQ Failover Worker: 5862378"
Java stack information for the threads listed above:
===================================================
"ActiveMQ Failover Worker: 5862378":
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:43)
- waiting to lock <0x36579610> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$2.transportResumed(DemandForwardingBridgeSupport.java:164)
- locked <0x3654b290> (a
org.apache.activemq.network.DemandForwardingBridgeSupport$2)
at
org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102)
at
org.apache.activemq.transport.TransportFilter.transportResumed(TransportFilter.java:102)
at
org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:175)
- locked <0x365793b0> (a java.lang.Object)
at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:87)
at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:24)
at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:38)
"Thread-7":
at
org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:335)
- waiting to lock <0x365793b0> (a java.lang.Object)
at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
- locked <0x36579610> (a java.lang.Object)
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.startRemoteBridge(DemandForwardingBridgeSupport.java:235)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$4.run(DemandForwardingBridgeSupport.java:197)
Found 1 deadlock.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
