Le 15/01/10 11:50, Rob Davies a écrit : > David - can you upgrade 5.3 ? Not immediatly, but worth a try > > On 15 Jan 2010, at 10:40, David Delbecq wrote: > >> Le 15/01/10 11:25, Gary Tully a écrit : >>> A thread dump of the broker jvm, when the clients are stalled, would >>> help >>> diagnose what is going on. btw, what activemq version are you using? >>> >> BrokerService - ActiveMQ 5.2.0 JMS Message Broker >> (localhost) is starting >> >> I'll come back with broker dump when there is a problem again. This can >> be in several hours or within a few weeks. However i'll try to force a >> stalling next week... >> >> >>> 2010/1/15 David Delbecq <[email protected]> >>> >>> >>>> Le 15/01/10 10:24, Dejan Bosanac a écrit : >>>> >>>>> Hi David, >>>>> >>>>> you're probably blocked by "producer flow control". Take a look here >>>>> for more info: http://activemq.apache.org/producer-flow-control.html >>>>> >>>>> Cheers >>>>> >>>> I already took a look at it, read it three time to be sure i >>>> understand. >>>> It's quite technical but beyond that, i don't think this apply for me. >>>> My queues are all empty!!! (they stalled when there were 50 messages >>>> pending but where finally emptied), connections are not shared by >>>> several queues and am not sure how to implement the cursor part of >>>> documentation. There is no "slow consumer" that could influence it at >>>> that right moment (customers are running an waiting for new >>>> messages as >>>> you can see in capture). Moreover my messages are of about 1k payload >>>> (about 10 small text entries in a MapMessage), and my activeMQ >>>> config is >>>> like this >>>> <policyEntry queue=">" memoryLimit="500mb"/> >>>> <policyEntry topic=">" memoryLimit="500mb"> >>>> .... >>>> <systemUsage> >>>> <systemUsage> >>>> <memoryUsage> >>>> <memoryUsage limit="20 mb"/> >>>> </memoryUsage> >>>> <storeUsage> >>>> <storeUsage limit="1 gb" name="foo"/> >>>> </storeUsage> >>>> <tempUsage> >>>> <tempUsage limit="100 mb"/> >>>> </tempUsage> >>>> </systemUsage> >>>> </systemUsage> >>>> >>>> >>>> There is no way a bunch of 10 pending messages could fill a 1gb store, >>>> isn't it? What config part of activemq do i need to change to ensure >>>> that it can store at least about 10000 persistent messages (that makes >>>> only 10Mb of datas!!) before failing? Currently it fails and is never >>>> recovering without a restart of broker when there are only 50 small >>>> pending messages in the queue. As a workaround i have set >>>> "sendFailIfNoSpace" to true, so my client will fail instead of >>>> stalling, >>>> i'll see when this happen more quickly like that.... >>>> >>>> >>>> Here is broker part of my config file, can someone help me fix this? >>>> Thank you: >>>> >>>> >>>> <broker xmlns="http://activemq.apache.org/schema/core"; >>>> brokerName="localhost" dataDirectory="${activemq.base}/data" >>>> persistent="true"> >>>> <plugins> >>>> <simpleAuthenticationPlugin> >>>> <users> >>>> <authenticationUser username="admin" password="******" >>>> groups="users,admins"/> >>>> <authenticationUser username="sharkTool" >>>> password="********" >>>> groups="serverTools"/> >>>> <authenticationUser username="adminConsole" >>>> password="*********" >>>> groups="serverTools"/> >>>> </users> >>>> </simpleAuthenticationPlugin> >>>> <authorizationPlugin> >>>> <map> >>>> <authorizationMap> >>>> <authorizationEntries> >>>> <authorizationEntry queue=">" read="admins" write="admins" >>>> admin="admins" /> >>>> <authorizationEntry queue="WF.>" read="users,serverTools" >>>> write="serverTools" admin="admins,serverTools" /> >>>> >>>> <authorizationEntry topic=">" read="admins" write="admins" >>>> admin="admins" /> >>>> <authorizationEntry topic="WF.>" read="users,serverTools" >>>> write="serverTools" admin="admins,serverTools" /> >>>> >>>> <authorizationEntry topic="ActiveMQ.Advisory.>" >>>> read="guests,users,serverTools" write="guests,users,serverTools" >>>> admin="guests,users,serverTools"/> >>>> </authorizationEntries> >>>> >>>> <!-- let's assign roles to temporary destinations. comment >>>> this entry if we don't want any roles assigned to temp destinations >>>> <tempDestinationAuthorizationEntry> >>>> <tempDestinationAuthorizationEntry >>>> read="tempDestinationAdmins" write="tempDestinationAdmins" >>>> admin="tempDestinationAdmins"/> >>>> </tempDestinationAuthorizationEntry> --> >>>> </authorizationMap> >>>> </map> >>>> </authorizationPlugin> >>>> >>>> >>>> </plugins> >>>> <!-- Destination specific policies using destination names or >>>> wildcards --> >>>> <destinationPolicy> >>>> <policyMap> >>>> <policyEntries> >>>> <policyEntry queue=">" memoryLimit="500mb"/> >>>> <policyEntry topic=">" memoryLimit="500mb"> >>>> <!-- you can add other policies too such as these >>>> <dispatchPolicy> >>>> <strictOrderDispatchPolicy/> >>>> </dispatchPolicy> >>>> <subscriptionRecoveryPolicy> >>>> <lastImageSubscriptionRecoveryPolicy/> >>>> </subscriptionRecoveryPolicy> >>>> --> >>>> </policyEntry> >>>> </policyEntries> >>>> </policyMap> >>>> </destinationPolicy> >>>> >>>> <!-- Use the following to configure how ActiveMQ is exposed in >>>> JMX --> >>>> <managementContext> >>>> <managementContext createConnector="false"/> >>>> </managementContext> >>>> >>>> <!-- The store and forward broker networks ActiveMQ will listen >>>> to --> >>>> <networkConnectors> >>>> <!-- by default just auto discover the other brokers --> >>>> <!-- <networkConnector name="default-nc" >>>> uri="multicast://default"/>--> >>>> <!-- Example of a static configuration: >>>> <networkConnector name="host1 and host2" >>>> uri="static://(tcp://host1:61616,tcp://host2:61616)"/> >>>> --> >>>> </networkConnectors> >>>> >>>> <persistenceAdapter> >>>> <amqPersistenceAdapter syncOnWrite="false" >>>> directory="${activemq.base}/data" maxFileLength="20 mb"/> >>>> </persistenceAdapter> >>>> >>>> >>>> >>>> <sslContext> >>>> <sslContext keyStore="file:${activemq.base}/conf/broker.ks" >>>> keyStorePassword="password" >>>> trustStore="file:${activemq.base}/conf/broker.ts" >>>> trustStorePassword="password"/> >>>> </sslContext> >>>> >>>> <!-- The maximum about of space the broker will use before >>>> slowing down producers --> >>>> <systemUsage> >>>> <systemUsage> >>>> <memoryUsage> >>>> <memoryUsage limit="20 mb"/> >>>> </memoryUsage> >>>> <storeUsage> >>>> <storeUsage limit="1 gb" name="foo"/> >>>> </storeUsage> >>>> <tempUsage> >>>> <tempUsage limit="100 mb"/> >>>> </tempUsage> >>>> </systemUsage> >>>> </systemUsage> >>>> >>>> >>>> <!-- The transport connectors ActiveMQ will listen to --> >>>> <transportConnectors> >>>> <transportConnector name="openwire" >>>> uri="tcp://localhost:61616"/> >>>> <!-- <transportConnector name="ssl" >>>> uri="ssl://localhost:61617"/> >>>> <transportConnector name="stomp" >>>> uri="stomp://localhost:61613"/> >>>> <transportConnector name="xmpp" >>>> uri="xmpp://localhost:61222"/> --> >>>> </transportConnectors> >>>> <destinations> >>>> <queue physicalName="WF.Regraph" /> >>>> <queue physicalName="WF.Notify"/> >>>> <topic physicalName="WF.Events"/> >>>> </destinations> >>>> <destinationInterceptors> >>>> <virtualDestinationInterceptor> >>>> <virtualDestinations> >>>> <compositeQueue name="WF.IncomingEvents"> >>>> <forwardTo> >>>> <queue physicalName="WF.Regraph"/> >>>> <queue physicalName="WF.Notify"/> >>>> <topic physicalName="WF.Events" /> >>>> </forwardTo> >>>> </compositeQueue> >>>> </virtualDestinations> >>>> </virtualDestinationInterceptor> >>>> </destinationInterceptors> >>>> >>>> >>>> </broker> >>>> >>>> >>>> >>>>> -- >>>>> >>>>> >>>>> On Fri, Jan 15, 2010 at 10:24 AM, David Delbecq <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Hello, >>>>> >>>>> can someone explain me why my activeMQ client is stalled in such >>>>> situation (i have 10 threads blocked like this having as >>>>> consequence a complete blocking of my client application): >>>>> >>>>> "http-8081-Processor6" daemon prio=10 tid=00841c38 nid=512 >>>>> lwp_id=2426611 waiting on condition [236f3000..236f153c] >>>>> java.lang.Thread.State: WAITING (parking) >>>>> at sun.misc.Unsafe.park(Native Method) >>>>> - parking to wait for <40ae1828> (a >>>>> >>>>> >>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) >>>> >>>>> at >>>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) >>>>> at >>>>> >>>>> >>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889) >>>> >>>> >>>>> at >>>>> >>>>> >>>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317) >>>> >>>> >>>>> at >>>>> >>>>> >>>> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40) >>>> >>>> >>>>> at >>>>> >>>>> >>>> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80) >>>> >>>> >>>>> at >>>>> >>>>> >>>> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233) >>>> >>>> >>>>> at >>>>> >>>>> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659) >>>>> - locked <40ae01e8> (a java.lang.Object) >>>>> at >>>>> >>>>> >>>> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227) >>>> >>>> >>>>> at >>>>> >>>>> >>>> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241) >>>> >>>> >>>>> at >>>>> be.rmi.shark.JMSEventManager.sendEvent(JMSEventManager.java:146) >>>>> >>>>> >>>>> considering that the activeMQ lists size, as referenced by >>>>> activeMQ admin interface are as follow: >>>>> Name Number Of Pending Messages Number Of Consumers >>>>> Messages Sent Messages Received Views >>>>> Operations >>>>> WF.Regraph 0 1 1714 1720 Browse Send To >>>>> >>>> Purge Delete >>>> >>>>> WF.Notify 0 1 1696 1742 Browse Send To >>>>> >>>> Purge Delete >>>> >>>>> WF.IncomingEvents 0 0 0 0 Browse >>>>> >>>> Send To Purge Delete >>>> >>>>> Is this to be considered normal that i must restart activeMQ >>>>> producers on a regular basis because of those this stalling? Where >>>>> can i find clear documentation on proper configuration of activeMQ >>>>> to prevent this stalling? Thank you. >>>>> >>>>> -- >>>>> David Delbecq >>>>> ICT >>>>> Institut Royal Météorologique >>>>> Ext:557 >>>>> >>>>> >>>>> >>>>> >>>> -- >>>> David Delbecq >>>> ICT >>>> Institut Royal Météorologique >>>> Ext:557 >>>> >>>> >>>> >>> >>> >>> >> >> >> -- >> David Delbecq >> ICT >> Institut Royal Météorologique >> Ext:557 >> > > Rob Davies > http://twitter.com/rajdavies > I work here: http://fusesource.com > My Blog: http://rajdavies.blogspot.com/ > I'm writing this: http://www.manning.com/snyder/ > > > > >
-- David Delbecq ICT Institut Royal Météorologique Ext:557
