no errors in the logs, but we see message flow resume after a server
restart...I'm starting to question our connection/server configuration a bit
now given this thread dump...
our application uses camel-activemq (2.10.1) and runs embedded in an AMQ
broker (5.7)....use the same connection factory/pool for all of our apps
camel routes that produce/consume from queues, etc...
here is my client code connection setup...
ActiveMQConnectionFactory queueConnectionFactory = new
ActiveMQConnectionFactory("vm://localhost?create=false");
queueConnectionFactory.setObjectMessageSerializationDefered(true);
queueConnectionFactory.setOptimizedMessageDispatch(true);
queueConnectionFactory.setDispatchAsync(true);
queueConnectionFactory.setUseAsyncSend(true);
queueConnectionFactory.setAlwaysSessionAsync(true);
PooledConnectionFactory pcf = new
PooledConnectionFactory(queueConnectionFactory);
pcf.setIdleTimeout(0);
ActiveMQComponent amq = ActiveMQComponent.activeMQComponent(brokerURL);
amq.setConnectionFactory(pcf);
amq.setMaxConcurrentConsumers(20);
amq.setCacheLevelName("CACHE_CONSUMER");
amq.setAcknowledgementModeName("AUTO_ACKNOWLEDGE");
this.getContext().addComponent("activemq", amq);
here is my activemq.xml...
<broker xmlns="http://activemq.apache.org/schema/core";
brokerName="localhost" dataDirectory="../data" useJmx="true">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue=">" producerFlowControl="false">
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<networkConnectors>
</networkConnectors>
<persistenceAdapter>
<kahaDB directory="../data/kaha"
enableJournalDiskSyncs="false"
indexWriteBatchSize="10000"
indexCacheSize="1000" />
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage><memoryUsage limit="4 gb"/></memoryUsage>
<storeUsage><storeUsage limit="100 gb"/></storeUsage>
<tempUsage><tempUsage limit="10 gb"/></tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61668" />
<transportConnector name="vm" uri="vm://localhost" />
</transportConnectors>
</broker>
<import resource="jndi.xml"/>
<import resource="camel.xml"/>
ceposta wrote
> Did you see any exceptions in the broker logs?
> Seems like the server-side VMTransport stopped draining its message queue
> and processing consumer acks... causing the client VMTransport to block on
> a put...and all other consumers using that connection to block...
>
>
> On Mon, Jul 22, 2013 at 3:46 PM, boday <
> ben.oday@
> >wrote:
>
>> seeing some thread blocking in our ActiveMQ 5.7 production application
>> (using
>> VM transport and AMQ connection pool)...anyone know of any
>> configuration/known bugs that could be contributing to this?
>>
>> seeing 20 thread like this one, WAITING for a single thread to release a
>> lock on 659ec0a4
>>
>> "Camel (camel-1) thread #29 - JmsConsumer[MyQueue]" - Thread t@130
>> java.lang.Thread.State: WAITING
>> at sun.misc.Unsafe.park(Native Method)
>> - waiting to lock <659ec0a4> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync) owned by "Camel
>> (camel-1) thread #13 - JmsConsumer[MyQueue]" t@94
>> at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
>> at
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842)
>> at
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178)
>> at
>>
>> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186)
>> at
>> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
>> at
>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:66)
>> at
>>
>> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
>> at
>>
>> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1290)
>> at
>>
>> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1284)
>> at
>>
>> org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1898)
>> at
>> org.apache.activemq.ActiveMQSession.sendAck(ActiveMQSession.java:2064)
>> at
>> org.apache.activemq.ActiveMQSession.sendAck(ActiveMQSession.java:2059)
>> at
>>
>> org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:921)
>> - locked <7212432> (a java.util.LinkedList)
>> at
>>
>> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:602)
>> at
>>
>> org.apache.activemq.pool.PooledMessageConsumer.receive(PooledMessageConsumer.java:67)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> at java.lang.Thread.run(Thread.java:662)
>>
>> Locked ownable synchronizers:
>> - locked <10cf5246> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>
>>
>> then thiis thread appears to be locking the resource in
>> question...659ec0a4
>>
>>
>> "Camel (camel-1) thread #13 - JmsConsumer[MyQueue]" - Thread t@94
>> java.lang.Thread.State: WAITING
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for <1ec8dc2f> (a
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>> at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
>> at
>>
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
>> at
>> java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:306)
>> at
>> org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:93)
>> at
>> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
>> at
>>
>> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
>> at
>>
>> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1290)
>> at
>>
>> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1284)
>> at
>>
>> org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1898)
>> at
>> org.apache.activemq.ActiveMQSession.sendAck(ActiveMQSession.java:2064)
>> at
>> org.apache.activemq.ActiveMQSession.sendAck(ActiveMQSession.java:2059)
>> at
>>
>> org.apache.activemq.ActiveMQMessageConsumer.afterMessageIsConsumed(ActiveMQMessageConsumer.java:921)
>> - locked <119a6cfe> (a java.util.LinkedList)
>> at
>>
>> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:602)
>> at
>>
>> org.apache.activemq.pool.PooledMessageConsumer.receive(PooledMessageConsumer.java:67)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
>> at
>>
>> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
>> at
>>
>> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>> at java.lang.Thread.run(Thread.java:662)
>>
>> Locked ownable synchronizers:
>> - locked <659ec0a4> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>
>> - locked <3169a0b9> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/consumer-threads-waiting-on-VMTransport-locks-tp4669580.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
> --
> *Christian Posta*
> http://www.christianposta.com/blog
> twitter: @christianposta
--
View this message in context:
http://activemq.2283324.n4.nabble.com/consumer-threads-waiting-on-VMTransport-locks-tp4669580p4669593.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
