[
https://issues.apache.org/jira/browse/QPID-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742128#comment-13742128
]
ASF subversion and git services commented on QPID-5050:
-------------------------------------------------------
Commit 1514664 from [~k-wall] in branch 'qpid/trunk'
[ https://svn.apache.org/r1514664 ]
QPID-5050: Move invocation of ExceptionListener to after the failoverMutex is
released avoiding deadlock possibility
Previously, the ExceptionListener was invoked whilst the failoverMutex was
held, between the
two potential state changes (connection state change and session state change).
This commit reorders the statements so that the ExceptionListner is fired after
the failoverMutex
is released. It also means that the ExceptionListener is fired *after* both
connection/session
have undergone any state changes. The exceptionListener member is also made
thread safe.
> Clients calling Connection#stop() from within an ExceptionListener have
> potential for deadlock
> ----------------------------------------------------------------------------------------------
>
> Key: QPID-5050
> URL: https://issues.apache.org/jira/browse/QPID-5050
> Project: Qpid
> Issue Type: Bug
> Components: Java Client
> Affects Versions: 0.8, 0.10, 0.12, 0.14, 0.16, 0.18, 0.20, 0.22
> Reporter: Keith Wall
> Assignee: Keith Wall
> Attachments:
> 0001-QPID-5050-Move-invocation-of-ExceptionListener-to-af.patch
>
>
> A deadlock possibility exists for client applications calling Connection#stop
> within an application owned ExceptionListener.
> Unfortunately a common messaging framework (Spring), installs such a
> ExceptionListener.
> In a recent support call, such a deadlock had occurred between the dispatcher
> thread (whose onMessage was in the process of creating a session) and a
> pooled thread bouncing a message back to the application. (The bounced
> messages is returned to the application via the exception listener).
> The deadlock involves the Dispatcher._lock and AMQConnection._failoverMutex.
> The deadlock was reproduced with a system test (attached to Jira) and
> deadlock captured with jstack -l <pid> (output below).
> {noformat}
> Dispatcher-1-Conn-4 pool-8-thread-1
> + acquires c888 (Dispatcher#_lock) + acquires ca68
> (AMQConnection#_failoverMutex)
> + tries to acquire ca68 + tries to acquire c888
> {noformat}
> {noformat}
> Found one Java-level deadlock:
> =============================
> "pool-8-thread-1":
> waiting to lock monitor 0x000000005b0d3560 (object 0x00000000f70bc888, a
> java.lang.Object),
> which is held by "Dispatcher-1-Conn-4"
> "Dispatcher-1-Conn-4":
> waiting to lock monitor 0x000000005b187308 (object 0x00000000f70bca68, a
> java.lang.Object),
> which is held by "pool-8-thread-1"
> Java stack information for the threads listed above:
> ===================================================
> "pool-8-thread-1":
> at
> org.apache.qpid.client.AMQSession$Dispatcher.setConnectionStopped(AMQSession.java:3276)
> - waiting to lock <0x00000000f70bc888> (a java.lang.Object)
> at org.apache.qpid.client.AMQSession.stop(AMQSession.java:2382)
> at org.apache.qpid.client.AMQConnection.stop(AMQConnection.java:835)
> at
> org.apache.qpid.test.unit.client.connection.ExceptionListenerTest$4.onException(ExceptionListenerTest.java:206)
> at
> org.apache.qpid.client.AMQConnection.exceptionReceived(AMQConnection.java:1329)
> - locked <0x00000000f70bca68> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQSession_0_8$4.run(AMQSession_0_8.java:600)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> "Dispatcher-1-Conn-4":
> at
> org.apache.qpid.client.AMQConnectionDelegate_8_0.executeRetrySupport(AMQConnectionDelegate_8_0.java:333)
> - waiting to lock <0x00000000f70bca68> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQConnection.executeRetrySupport(AMQConnection.java:624)
> at
> org.apache.qpid.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
> at
> org.apache.qpid.client.AMQSession.createProducerImpl(AMQSession.java:2600)
> at
> org.apache.qpid.client.AMQSession.createProducer(AMQSession.java:1176)
> at
> org.apache.qpid.client.AMQSession.createProducer(AMQSession.java:98)
> at
> org.apache.qpid.test.unit.client.connection.ExceptionListenerTest$5.onMessage(ExceptionListenerTest.java:229)
> at
> org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:744)
> at
> org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:718)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3388)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3327)
> - locked <0x00000000f71747e0> (a java.lang.Object)
> - locked <0x00000000f70bc888> (a java.lang.Object)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3114)
> at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3107)
> at
> org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:54)
> at
> org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3250)
> at java.lang.Thread.run(Thread.java:662)
> Found 1 deadlock.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
