And the all the messages shows this on thread dump..
"pool-flow.seda.servicemix-jms-thread-1433" - Thread t...@6141
java.lang.Thread.State: WAITING on
java.util.concurrent.locks.abstractqueuedsynchronizer$conditionobj...@1540b71
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:676)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- locked java.util.concurrent.locks.reentrantlock$nonfairs...@19353a3
.............
............
subh wrote:
>
> Continuing the discussion. I am seeing jms messages getting stuck in the
> queue and not getting published to the topic..On taking thread dump, I saw
> one instance where SMX client I am using was in waiting state..causing all
> the messages subsequent to that to be queued.
>
> Also, I think it happened while both jmsconsumers (i have another consumer
> and provider combination) was receiving messages simultaneously although
> the other jms provider has a higher priority than this one.
>
> Any idea/suggestions on what might be causing it will be great..
>
> Below is my jmsconsumer
>
> <jms:consumer service="orderservice:extOrderConsumerQueue"
> endpoint="jmsInQueueConsumer"
> targetService="orderservice:orderPublisherBean"
> targetEndpoint="endpoint"
> destinationName="queue/ORS/OrderIn"
> pubSubDomain="false"
> cacheLevel="1"
> connectionFactory="#connectionFactory" />
>
> and jmsprovider
>
> <jms:provider service="orderservice:orsDispatchedOrder"
> endpoint="orsProvider"
> destinationName="topic/ORS/OrderIn"
> connectionFactory="#connectionFactory"
> pubSubDomain="true"
> />
>
> and this the SMX bean code using SMX client to publish message to the
> topic
>
> private static final String ORS_LOE_JMS_PROVIDER_SERVICE =
> "service:http://services.fuelquest.com/orderservice/orsDispatchedOrder";;
> ......
> .....
> if (exchange.getStatus() == ExchangeStatus.ACTIVE) {
> smxclient = SMXClient.getInstance();
> Destination destination =
> smxclient.createDestination(ORS_LOE_JMS_PROVIDER_SERVICE);
> Message message =
> destination.createInOnlyMessage();
> message.setProperty(REPLY_HEADER,
> Boolean.TRUE);
> message.setBody(messageBody);
>
> smxclient.send(message);
> smxclient.done(exchange);
> }
>
> and the snippet from the thread dump....
>
> "pool-flow.seda.servicemix-bean-thread-1975" - Thread t...@6142
> java.lang.Thread.State: WAITING on
> java.util.concurrent.locks.abstractqueuedsynchronizer$conditionobj...@1540b71
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:676)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1738)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at
> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:53)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:183)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:163)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
> at
> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
> at
> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88)
> at
> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:903)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:396)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:432)
> at
> org.apache.servicemix.common.EndpointDeliveryChannel.send(EndpointDeliveryChannel.java:89)
> at
> org.apache.servicemix.common.endpoints.SimpleEndpoint.send(SimpleEndpoint.java:70)
> at
> org.apache.servicemix.common.endpoints.SimpleEndpoint.done(SimpleEndpoint.java:85)
> at
> org.apache.servicemix.jms.endpoints.JmsProviderEndpoint.process(JmsProviderEndpoint.java:500)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
> at
> org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:623)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1738)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at
> org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:53)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:183)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:163)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
> at
> org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
> at
> org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88)
> at
> org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:903)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:396)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:432)
> at
> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:148)
> at
> org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:152)
> at
> com.fuelquest.service.fqservices.orsesb.beansu.OrderServicePublishCallerBean.writeOrderPublishedMessageToSendOut(OrderServicePublishCallerBean.java:235)
> at
> com.fuelquest.service.fqservices.orsesb.beansu.OrderServicePublishCallerBean.onMessageExchange(OrderServicePublishCallerBean.java:141)
> at
> org.apache.servicemix.bean.BeanEndpoint.onProviderExchange(BeanEndpoint.java:230)
> - locked org.apache.servicemix.bean.support.requ...@17474ef
> at
> org.apache.servicemix.bean.BeanEndpoint.process(BeanEndpoint.java:217)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
> at
> org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
> at
> org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
> at
> org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:623)
> at
> org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
> at
> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> Locked ownable synchronizers:
> - locked java.util.concurrent.locks.reentrantlock$nonfairs...@a93013
>
>
>
> Eighty8 wrote:
>>
>> No probelem. I saw blocks i the Saxon API's too...which is used by
>> things other than eip...good luck.
>> -A
>>
>> subh wrote:
>>>
>>> Thanks for the input and suggestions.
>>>
>>> As of now, I have removed the EIP component from the flow as I had
>>> observed the exchange waiting for threads to finish.
>>>
>>> Also, I have defined separate connection pool for the jms components as
>>> suggested in a different thread. I am seeing much better performance,
>>> but will need to monitor for a few days to.
>>>
>>>
>>>
>>> Eighty8 wrote:
>>>>
>>>> I found I had to throttle my messages by sleeping ~250-500ms apart. I
>>>> recommend buying a YourKit license and attaching it to your server.
>>>> You'll see a lot more of what's going on under the hood. Look for
>>>> 'lots of read' in the threads indicating blocks that are waiting to
>>>> clear.
>>>>
>>>> I have not had success, yet, using component.properties to controls the
>>>> eip and saxon LifeCycles, but am continuing to try to get that working.
>>>>
>>>> You can inspect the configurable properties by enabling jetty on your
>>>> ESB and access jmx-console.
>>>> Good luck.
>>>>
>>>>
>>>> subh wrote:
>>>>>
>>>>> I have a simple flow as below
>>>>>
>>>>> External message producer --> SMX JMS queue --> SMX Bean --> Camel
>>>>> router --> SMX JMS Topic
>>>>>
>>>>> The servicemix bean has the business logic to generate the outgoing
>>>>> message.
>>>>> To forward the message onto the next service, I am using
>>>>> ServiceMixClient.
>>>>>
>>>>> I am using the ClientFactory to create a instance of the client.
>>>>>
>>>>> ClientFactory factory = new
>>>>> InitialContext().lookup(ClientFactory.DEFAULT_JNDI_NAME);
>>>>> ServiceMixClient client = factory.createClient();
>>>>>
>>>>> But after processing 1000 - 1500 messages, the Servicemix bean
>>>>> component becomes very slow to respond. On checking the log, I see
>>>>> that the bean thread count increases considerably and the expired
>>>>> threads are not garbage collected.
>>>>>
>>>>> Due to this, all incoming messages are getting blocked which forces us
>>>>> to restart the servicemix server. Once it is restarted, the processing
>>>>> becomes very fast and we come to the same situation again in a day or
>>>>> so.
>>>>>
>>>>> Am I doing something wrong by using the ServicemixClient inside the
>>>>> bean? If no, then what are the best practices for using the Servicemix
>>>>> client class.
>>>>>
>>>>> Are there alternative ways to specify the "target service" from the
>>>>> SMX bean endpoint?
>>>>>
>>>>> Any hint or suggestion would be helpful.
>>>>>
>>>>> I am using activemq5.3 and servicemix 3.3
>>>>>
>>>>> Regards
>>>>> Subh
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
--
View this message in context:
http://old.nabble.com/ServicemixBean-thread-issue..-tp27714350p27823959.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
