Keith Wall created QPID-5050:
--------------------------------
Summary: Clients calling Connection#stop() from within an
ExceptionListener have potential for deadlock
Key: QPID-5050
URL: https://issues.apache.org/jira/browse/QPID-5050
Project: Qpid
Issue Type: Bug
Components: Java Client
Affects Versions: 0.22, 0.20, 0.18, 0.16, 0.14, 0.12, 0.10, 0.8
Reporter: Keith Wall
A deadlock possibility exists for client applications calling Connection#stop
within an application owned ExceptionListener.
Unfortunately a common messaging framework (Spring), installs such a
ExceptionListener.
In a recent support call, such a deadlock had occurred between the dispatcher
thread (whose onMessage was in the process of creating a session) and a pooled
thread bouncing a message back to the application. (The bounced messages is
returned to the application via the exception listener).
The deadlock involves the Dispatcher._lock and AMQConnection._failoverMutex.
The deadlock was reproduced with a system test (attached to Jira) and deadlock
captured with jstack -l <pid> (output below).
{noformat}
Dispatcher-1-Conn-4 pool-8-thread-1
+ acquires c888 (Dispatcher#_lock) + acquires ca68
(AMQConnection#_failoverMutex)
+ tries to acquire ca68 + tries to acquire c888
{noformat}
{noformat}
Found one Java-level deadlock:
=============================
"pool-8-thread-1":
waiting to lock monitor 0x000000005b0d3560 (object 0x00000000f70bc888, a
java.lang.Object),
which is held by "Dispatcher-1-Conn-4"
"Dispatcher-1-Conn-4":
waiting to lock monitor 0x000000005b187308 (object 0x00000000f70bca68, a
java.lang.Object),
which is held by "pool-8-thread-1"
Java stack information for the threads listed above:
===================================================
"pool-8-thread-1":
at
org.apache.qpid.client.AMQSession$Dispatcher.setConnectionStopped(AMQSession.java:3276)
- waiting to lock <0x00000000f70bc888> (a java.lang.Object)
at org.apache.qpid.client.AMQSession.stop(AMQSession.java:2382)
at org.apache.qpid.client.AMQConnection.stop(AMQConnection.java:835)
at
org.apache.qpid.test.unit.client.connection.ExceptionListenerTest$4.onException(ExceptionListenerTest.java:206)
at
org.apache.qpid.client.AMQConnection.exceptionReceived(AMQConnection.java:1329)
- locked <0x00000000f70bca68> (a java.lang.Object)
at org.apache.qpid.client.AMQSession_0_8$4.run(AMQSession_0_8.java:600)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"Dispatcher-1-Conn-4":
at
org.apache.qpid.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:333)
- waiting to lock <0x00000000f70bca68> (a java.lang.Object)
at
org.apache.qpid.client.AMQConnection.executeRetrySupport(AMQConnection.java:624)
at
org.apache.qpid.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
at
org.apache.qpid.client.AMQSession.createProducerImpl(AMQSession.java:2600)
at
org.apache.qpid.client.AMQSession.createProducer(AMQSession.java:1176)
at org.apache.qpid.client.AMQSession.createProducer(AMQSession.java:98)
at
org.apache.qpid.test.unit.client.connection.ExceptionListenerTest$5.onMessage(ExceptionListenerTest.java:229)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:744)
at
org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:718)
at
org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3388)
at
org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3327)
- locked <0x00000000f71747e0> (a java.lang.Object)
- locked <0x00000000f70bc888> (a java.lang.Object)
at
org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3114)
at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3107)
at
org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
at
org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3250)
at java.lang.Thread.run(Thread.java:662)
Found 1 deadlock.
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
