Continuing the discussion. I am seeing jms messages getting stuck in the
queue and not getting published to the topic..On taking thread dump, I saw
one instance where SMX client I am using was in waiting state..causing all
the messages subsequent to that to be queued.
Also, I think it happened while both jmsconsumers (i have another consumer
and provider combination) was receiving messages simultaneously although the
other jms provider has a higher priority than this one.
Any idea/suggestions on what might be causing it will be great..
Below is my jmsconsumer
<jms:consumer service="orderservice:extOrderConsumerQueue"
endpoint="jmsInQueueConsumer"
targetService="orderservice:orderPublisherBean"
targetEndpoint="endpoint"
destinationName="queue/ORS/OrderIn"
pubSubDomain="false"
cacheLevel="1"
connectionFactory="#connectionFactory" />
and jmsprovider
<jms:provider service="orderservice:orsDispatchedOrder"
endpoint="orsProvider"
destinationName="topic/ORS/OrderIn"
connectionFactory="#connectionFactory"
pubSubDomain="true"
/>
and this the SMX bean code using SMX client to publish message to the topic
private static final String ORS_LOE_JMS_PROVIDER_SERVICE =
"service:http://services.fuelquest.com/orderservice/orsDispatchedOrder";;
......
.....
if (exchange.getStatus() == ExchangeStatus.ACTIVE) {
smxclient = SMXClient.getInstance();
Destination destination =
smxclient.createDestination(ORS_LOE_JMS_PROVIDER_SERVICE);
Message message =
destination.createInOnlyMessage();
message.setProperty(REPLY_HEADER,
Boolean.TRUE);
message.setBody(messageBody);
smxclient.send(message);
smxclient.done(exchange);
}
and the snippet from the thread dump....
"pool-flow.seda.servicemix-bean-thread-1975" - Thread t...@6142
java.lang.Thread.State: WAITING on
java.util.concurrent.locks.abstractqueuedsynchronizer$conditionobj...@1540b71
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.ArrayBlockingQueue.put(ArrayBlockingQueue.java:252)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:676)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at
java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1738)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at
org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:53)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:183)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:163)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
at
org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
at
org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88)
at
org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:903)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:396)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:432)
at
org.apache.servicemix.common.EndpointDeliveryChannel.send(EndpointDeliveryChannel.java:89)
at
org.apache.servicemix.common.endpoints.SimpleEndpoint.send(SimpleEndpoint.java:70)
at
org.apache.servicemix.common.endpoints.SimpleEndpoint.done(SimpleEndpoint.java:85)
at
org.apache.servicemix.jms.endpoints.JmsProviderEndpoint.process(JmsProviderEndpoint.java:500)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:623)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at
java.util.concurrent.ThreadPoolExecutor$CallerRunsPolicy.rejectedExecution(ThreadPoolExecutor.java:1738)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at
org.apache.servicemix.executors.impl.ExecutorImpl.execute(ExecutorImpl.java:53)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue.enqueue(SedaQueue.java:128)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.enqueuePacket(SedaFlow.java:183)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doSend(SedaFlow.java:163)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.send(AbstractFlow.java:123)
at
org.apache.servicemix.jbi.nmr.DefaultBroker.sendExchangePacket(DefaultBroker.java:283)
at
org.apache.servicemix.jbi.security.SecuredBroker.sendExchangePacket(SecuredBroker.java:88)
at
org.apache.servicemix.jbi.container.JBIContainer.sendExchange(JBIContainer.java:903)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.doSend(DeliveryChannelImpl.java:396)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.send(DeliveryChannelImpl.java:432)
at
org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:148)
at
org.apache.servicemix.client.DefaultServiceMixClient.send(DefaultServiceMixClient.java:152)
at
com.fuelquest.service.fqservices.orsesb.beansu.OrderServicePublishCallerBean.writeOrderPublishedMessageToSendOut(OrderServicePublishCallerBean.java:235)
at
com.fuelquest.service.fqservices.orsesb.beansu.OrderServicePublishCallerBean.onMessageExchange(OrderServicePublishCallerBean.java:141)
at
org.apache.servicemix.bean.BeanEndpoint.onProviderExchange(BeanEndpoint.java:230)
- locked org.apache.servicemix.bean.support.requ...@17474ef
at
org.apache.servicemix.bean.BeanEndpoint.process(BeanEndpoint.java:217)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:627)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:581)
at
org.apache.servicemix.common.AsyncBaseLifeCycle.onMessageExchange(AsyncBaseLifeCycle.java:535)
at
org.apache.servicemix.common.SyncLifeCycleWrapper.onMessageExchange(SyncLifeCycleWrapper.java:60)
at
org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:623)
at
org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:172)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:168)
at
org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Locked ownable synchronizers:
- locked java.util.concurrent.locks.reentrantlock$nonfairs...@a93013
Eighty8 wrote:
>
> No probelem. I saw blocks i the Saxon API's too...which is used by things
> other than eip...good luck.
> -A
>
> subh wrote:
>>
>> Thanks for the input and suggestions.
>>
>> As of now, I have removed the EIP component from the flow as I had
>> observed the exchange waiting for threads to finish.
>>
>> Also, I have defined separate connection pool for the jms components as
>> suggested in a different thread. I am seeing much better performance, but
>> will need to monitor for a few days to.
>>
>>
>>
>> Eighty8 wrote:
>>>
>>> I found I had to throttle my messages by sleeping ~250-500ms apart. I
>>> recommend buying a YourKit license and attaching it to your server.
>>> You'll see a lot more of what's going on under the hood. Look for 'lots
>>> of read' in the threads indicating blocks that are waiting to clear.
>>>
>>> I have not had success, yet, using component.properties to controls the
>>> eip and saxon LifeCycles, but am continuing to try to get that working.
>>>
>>> You can inspect the configurable properties by enabling jetty on your
>>> ESB and access jmx-console.
>>> Good luck.
>>>
>>>
>>> subh wrote:
>>>>
>>>> I have a simple flow as below
>>>>
>>>> External message producer --> SMX JMS queue --> SMX Bean --> Camel
>>>> router --> SMX JMS Topic
>>>>
>>>> The servicemix bean has the business logic to generate the outgoing
>>>> message.
>>>> To forward the message onto the next service, I am using
>>>> ServiceMixClient.
>>>>
>>>> I am using the ClientFactory to create a instance of the client.
>>>>
>>>> ClientFactory factory = new
>>>> InitialContext().lookup(ClientFactory.DEFAULT_JNDI_NAME);
>>>> ServiceMixClient client = factory.createClient();
>>>>
>>>> But after processing 1000 - 1500 messages, the Servicemix bean
>>>> component becomes very slow to respond. On checking the log, I see that
>>>> the bean thread count increases considerably and the expired threads
>>>> are not garbage collected.
>>>>
>>>> Due to this, all incoming messages are getting blocked which forces us
>>>> to restart the servicemix server. Once it is restarted, the processing
>>>> becomes very fast and we come to the same situation again in a day or
>>>> so.
>>>>
>>>> Am I doing something wrong by using the ServicemixClient inside the
>>>> bean? If no, then what are the best practices for using the Servicemix
>>>> client class.
>>>>
>>>> Are there alternative ways to specify the "target service" from the SMX
>>>> bean endpoint?
>>>>
>>>> Any hint or suggestion would be helpful.
>>>>
>>>> I am using activemq5.3 and servicemix 3.3
>>>>
>>>> Regards
>>>> Subh
>>>>
>>>
>>>
>>
>>
>
>
--
View this message in context:
http://old.nabble.com/ServicemixBean-thread-issue..-tp27714350p27823844.html
Sent from the ServiceMix - User mailing list archive at Nabble.com.
