This is definitely a bug, the question is whose bug it is (whose code should change): ActiveMQ, Mule, Spring, or some combination of the three.
I looked in http://grepcode.com/file/repo1.maven.org/maven2/org. springframework/spring-jms/3.0.0.RELEASE/org/springframework/jms/connection/ SingleConnectionFactory.java and I don't see the org.springframework.Single ConnectionFactory.getConnection() method you referenced. What version of Spring are you using? You referenced a toString() method; where is that code? My initial reaction based on what you've written is that although calling external code while holding a lock isn't great (and we might want to change the code to not do that), the most egregious problem is the fact that a toString() method is doing something that involves acquiring a lock, and that's my first thought for the best way to solve this immediate problem. Tim On Mar 10, 2017 4:11 PM, "Facundo Darío Velazquez Santillán" < facundovelazquezsantil...@gmail.com> wrote: Hi. In Mule ESB, I have detected a deadlock caused by the interaction between Spring and ActiveMQ. Basically, there are two execution chains that are causing the deadlock: First Chain: when a Message arrives to the JMS Queue: 1) ActiveMQMessageConsumer.dispatch() gets the lock of FifoMessageDispatchChannel.mutex. 2) ActiveMQMessageConsumer.dispatch() calls the onMessage() method without releasing the lock. 3) onMessage Mule Implementation calls org.springframework. SingleConnectionFactory$SharedConnectionInvocationHandler.invoke() because It needs a toString of the used resource (in this case the connection) to logging purposes. 3) in ToString() method, org.springframework.SingleConnectionFactory.getConnection() method is called and It takes the org.springframework.Single ConnectionFactory.connectionMonitor lock to access to connection (that's a shared resource). Lock ordering in this chain: FifoMessageDispatchChannel.mutex lock -> org.springframework.SingleC onnectionFactory.connectionMonitor lock Second Chain: when A reconnection-strategy is applied: 1) Mule stops the current connection calling org.springframework.Sin gleConnectionFactory.localStop() 2) LocalStop takes the org.springframework.SingleConnectionFactory .connectionMonitor lock. 3) After a chain of calls: FifoMessageDispatchChannel.stop() is called. 4) FifoMessageDispatchChannel.stop() takes the FifoMessageDispatchChannel.mutex lock. Lock Ordering in this case: org.springframework.SingleConnectionFactory.connectionMonitor lock -> FifoMessageDispatchChannel.mutex lock As you can see, The chains don't take care about lock ordering, and It's causing a deadlock. I think the root problem is the invocation of the external implemented method onEvent(). Calling the external implemented onEvent() method in the synchronized lock can cause an unpredictable behaviour in lock ordering causing deadlock issues. I think It could be resolved if the lock is released before invoking onEvent(). I would like to know their opinions about this case to report an issue if It is necessary. Thanks. Cheers.
