I'm still investigating the cause of the dead lock. Here's a piece of thread
dump I found from the producer broker. * Note producer broker configures the
NetworkConnector..
"ActiveMQ Transport: tcp://lod-uimlongda/10.130.156.161:61616@45650" #139
prio=5 os_prio=0 tid=0x00007fddc0007800 nid=0x1a1a waiting on condition
[0x00007fdd91ede000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000cf606228> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
*at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:66)*
at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at
org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1432)
at
org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:946)
at
org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:902)
at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:190)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:124)
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at
org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:162)
at
org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:154)
- locked <0x00000000cf6064d8> (a
java.util.concurrent.atomic.AtomicBoolean)
at
org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:131)
*at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)*
at
org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.addSubscription(DemandForwardingBridgeSupport.java:922)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.addConsumerInfo(DemandForwardingBridgeSupport.java:1197)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteConsumerAdvisory(DemandForwardingBridgeSupport.java:776)
- locked <0x00000000d5428cd8> (a java.net.URI)
at
org.apache.activemq.network.DemandForwardingBridgeSupport.serviceRemoteCommand(DemandForwardingBridgeSupport.java:623)
at
org.apache.activemq.network.DemandForwardingBridgeSupport$3.onCommand(DemandForwardingBridgeSupport.java:225)
at
org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:124)
at
org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:125)
at
org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:300)
at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:233)
at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
at java.lang.Thread.run(Thread.java:745)
I see the MutexTransport.oneway is called twice this is stacktrace, and it
finally got locked. Is this a self-lock? I checked other threads in the
thread dump, but didn't see other thread could potentially hold the
writeLock...
@Override
public void oneway(Object command) throws IOException {
*writeLock.lock();*
try {
next.oneway(command);
} finally {
writeLock.unlock();
}
}
Here's the full thread dump.
2016.jstack
<http://activemq.2283324.n4.nabble.com/file/n4709271/2016.jstack>
--
View this message in context:
http://activemq.2283324.n4.nabble.com/ActiveMQ-duplex-network-connector-dead-lock-5-13-1-5-11-1-tp4708952p4709271.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
