Yes, It's clearly a deadlock...
Notice that the first thread is waiting to lock the same object that the second thread has locked....meanwhile, the second thread is not waking up, it's "waiting for space".....And the application grinds to a halt.... In the first thread, see: waiting to lock <0x2704bef8> (a java.lang.Object) In the second thread, see: - locked <0x2704bef8> (a java.lang.Object) Further, in the second thread, at the top, see: org.apache.activemq.memory.UsageManager.waitForSpace(UsageManager.java:105) Since the consumer is locked out, it can't clear space in the queue, and the producer will never return from waiting for space, since it is locking out the only potential consumer... Jason James.Strachan wrote: > > That stack trace looks more like a wait than a deadlock? Are you sure > its a real deadlock? > > This FAQ entry should help > http://activemq.apache.org/my-producer-blocks.html > > On 5/15/07, Jason Rosenberg <[EMAIL PROTECTED]> wrote: >> >> Hi all, I'm just using ActiveMQ for the first time. I've encountered a >> deadlock condition, that appears rooted in the ActiveMQ internals. I'm >> not >> sure if this is a known problem or not. I've been able to work around >> the >> issue, but thought I'd forward this on, to see if it warrants filing a >> bug, >> or if it's a 'feature'.... >> >> I am using version 4.1.1, and am running it within tomcat 5.5.23, using >> the >> vm://localhost broker, and am using non-persistent messaging, with >> AUTO_ACKNOWLEDGE. I am using queues, and the consumers are triggered >> using >> the setMessageListener api. >> >> Essentially what seems to be happening is that the producer generates >> more >> messages than the consumer can keep up with, and eventually appears to >> cause >> the message queue to run out of space. What I would expect in this >> case, >> is that the consumer would still be able to process messages, at it's own >> slow pace....and as it processes messages, space would open up in the >> queue, >> and the producer would be able to slowly continue submitting more >> messages.... >> >> I realize normally this is not an ideal use of messaging, e.g. you don't >> want to generate more messages than you can possibly consume, >> etc....However, I would expect that it would still make progress, etc.. >> >> I am instead seeing a simple deadlock condition, the producer locks up >> while >> waiting for space, and locks a monitor that the consumer appears to be >> waiting on....I've included a partial thread dump below, with the 2 >> relevant >> threads: >> >> "ActiveMQ Session Task" daemon prio=7 tid=0x0055be30 nid=0x18ac000 >> waiting >> for monitor entry [0xb1820000..0xb1820d10] >> at >> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:46) >> - waiting to lock <0x2704bef8> (a java.lang.Object) >> at >> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) >> at >> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1155) >> at >> org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1651) >> at >> org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:679) >> at >> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:855) >> at >> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:99) >> at >> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:166) >> at >> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:117) >> at >> org.apache.activemq.thread.PooledTaskRunner.access$100(PooledTaskRunner.java:26) >> at >> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:44) >> at >> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) >> at >> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) >> at java.lang.Thread.run(Thread.java:613) >> >> >> >> "http-8080-Processor25" daemon prio=5 tid=0x00540ed0 nid=0x1902800 in >> Object.wait() [0xb27bf000..0xb27bfd10] >> at java.lang.Object.wait(Native Method) >> - waiting on <0x2702d7c8> (a java.lang.Object) >> at >> org.apache.activemq.memory.UsageManager.waitForSpace(UsageManager.java:105) >> - locked <0x2702d7c8> (a java.lang.Object) >> at >> org.apache.activemq.memory.UsageManager.waitForSpace(UsageManager.java:100) >> at org.apache.activemq.broker.region.Queue.send(Queue.java:318) >> at >> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:292) >> at >> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:382) >> at >> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:193) >> at >> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126) >> at >> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:98) >> at >> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:136) >> at >> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:498) >> at >> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:604) >> at >> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294) >> at >> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185) >> at >> org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:95) >> at >> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:65) >> at >> org.apache.activemq.transport.vm.VMTransport.syncOneWay(VMTransport.java:96) >> at >> org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:83) >> at >> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:47) >> - locked <0x2704bef8> (a java.lang.Object) >> at >> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) >> at >> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1155) >> at >> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1549) >> at >> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:465) >> at >> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:356) >> at >> com.shopzilla.utils.NonPersistentLocalMessaging.sendMessage(NonPersistentLocalMessaging.java:208) >> at >> com.shopzilla.its.ws.ITSSessionizationFileCollector._sendDataListMessage(ITSSessionizationFileCollector.java:140) >> at >> com.shopzilla.its.ws.ITSSessionizationFileCollector.doPost(ITSSessionizationFileCollector.java:101) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:710) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:803) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151) >> at >> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870) >> at >> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) >> at >> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) >> at >> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) >> at >> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) >> at java.lang.Thread.run(Thread.java:613) >> >> >> Thanks, >> >> Jason >> -- >> View this message in context: >> http://www.nabble.com/Deadlock-condition%2C-producer-in-waitForSpace%2C-consumer-slow%2C-etc.-tf3759923s2354.html#a10627274 >> Sent from the ActiveMQ - User mailing list archive at Nabble.com. >> >> > > > -- > James > ------- > http://macstrac.blogspot.com/ > > -- View this message in context: http://www.nabble.com/Deadlock-condition%2C-producer-in-waitForSpace%2C-consumer-slow%2C-etc.-tf3759923s2354.html#a10628442 Sent from the ActiveMQ - User mailing list archive at Nabble.com.
