We are using ActiveMQ 5.2.0 the latest stable release. This is what i have
seen:
Client ID: ...
Subscription Name: ...
Connection ID: ...
Pending Queue Size: 9894
Dispatched Queue Size 0
Dispatched Counter: 0
Enqueue Counter: 9894
Dequeue Counter: 0
Within the same MQ server, some consumers work, some just got stuck into the
state that I have
mentioned above. The active MQ log shows:
2009-01-25 14:33:14,489 [168.20.36:53300] ERROR RecoveryListenerAdapter
- Message id ID:hefeed1-48611-1232757655348-4:140816:-1:1:1 could not be
recovered from the data store - already dispatched
2009-01-25 14:33:14,490 [168.20.36:53300] ERROR RecoveryListenerAdapter
- Message id ID:hefeed1-48611-1232757655348-4:140816:-1:1:1 could not be
recovered from the data store - already dispatched
...
2009-01-25 14:33:15,428 [168.20.36:53310] ERROR DataManagerImpl
- Looking for key 4 but not found in fileMap: {5=data-topic-data-5 number =
5 , length = 2097120 refCount = 2460, 6=data-topic-data-6 number = 6 ,
length = 2097135 refCount = 2942, 7=data-topic-data-7 number = 7 , length =
2097127 refCount = 2844, 8=data-topic-data-8 number = 8 , length = 2097111
refCount = 3308, 9=data-topic-data-9 number = 9 , length = 2097127 refCount
= 3196, 10=data-topic-data-10 number = 10 , length = 2097111 refCount =
3341, 11=data-topic-data-11 number = 11 , length = 2097127 refCount = 3625,
12=data-topic-data-12 number = 12 , length = 2097111 refCount = 3602,
13=data-topic-data-13 number = 13 , length = 2097127 refCount = 3544,
14=data-topic-data-14 number = 14 , length = 2097111 refCount = 3412,
15=data-topic-data-15 number = 15 , length = 2097127 refCount = 3698,
17=data-topic-data-17 number = 17 , length = 2097127 refCount = 3612,
16=data-topic-data-16 number = 16 , length = 2097111 refCount = 3570,
19=data-topic-data-19 number = 19 , length = 2097127 refCount = 3588,
18=data-topic-data-18 number = 18 , length = 2097111 refCount = 3468,
21=data-topic-data-21 number = 21 , length = 2097127 refCount = 3322,
20=data-topic-data-20 number = 20 , length = 2097111 refCount = 3284,
23=data-topic-data-23 number = 23 , length = 2097127 refCount = 3336,
22=data-topic-data-22 number = 22 , length = 2097111 refCount = 3188,
25=data-topic-data-25 number = 25 , length = 2097127 refCount = 3270,
24=data-topic-data-24 number = 24 , length = 2097111 refCount = 3322,
27=data-topic-data-27 number = 27 , length = 2097127 refCount = 3240,
26=data-topic-data-26 number = 26 , length = 2097111 refCount = 3214,
29=data-topic-data-29 number = 29 , length = 2097127 refCount = 3290,
28=data-topic-data-28 number = 28 , length = 2097111 refCount = 3126,
31=data-topic-data-31 number = 31 , length = 2097127 refCount = 3382,
30=data-topic-data-30 number = 30 , length = 2097111 refCount = 3200,
34=data-topic-data-34 number = 34 , length = 2097111 refCount = 2928,
35=data-topic-data-35 number = 35 , length = 2097127 refCount = 3094,
32=data-topic-data-32 number = 32 , length = 2097111 refCount = 2866,
33=data-topic-data-33 number = 33 , length = 2097127 refCount = 3096,
38=data-topic-data-38 number = 38 , length = 2097111 refCount = 3202,
39=data-topic-data-39 number = 39 , length = 2097127 refCount = 3158,
36=data-topic-data-36 number = 36 , length = 2097111 refCount = 3146,
37=data-topic-data-37 number = 37 , length = 2097127 refCount = 3082,
42=data-topic-data-42 number = 42 , length = 2097111 refCount = 9917,
43=data-topic-data-43 number = 43 , length = 2097127 refCount = 24897,
40=data-topic-data-40 number = 40 , length = 2097111 refCount = 3144,
41=data-topic-data-41 number = 41 , length = 2097127 refCount = 3344,
44=data-topic-data-44 number = 44 , length = 2097111 refCount = 24650,
45=data-topic-data-45 number = 45 , length = 1051587 refCount = 12466}
2009-01-25 14:33:15,429 [168.20.36:53310] ERROR MapContainerImpl
- Failed to get value for offset=17748, key=(4, 610628, 47), value=(4,
610680, 63), previousItem=102, nextItem=-1
java.io.IOException: Could not locate data file data-topic-data-4
at
org.apache.activemq.kaha.impl.data.DataManagerImpl.getDataFile(DataManagerImpl.java:129)
at
org.apache.activemq.kaha.impl.data.SyncDataFileReader.readItem(SyncDataFileReader.java:65)
at
org.apache.activemq.kaha.impl.data.DataManagerImpl.readItem(DataManagerImpl.java:141)
at
org.apache.activemq.kaha.impl.container.MapContainerImpl.getValue(MapContainerImpl.java:481)
at
org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.recoverNextMessages(KahaTopicReferenceStore.java:254)
at
org.apache.activemq.store.amq.AMQTopicMessageStore.recoverNextMessages(AMQTopicMessageStore.java:59)
at
org.apache.activemq.broker.region.cursors.TopicStorePrefetch.doFillBatch(TopicStorePrefetch.java:94)
..
2009-01-25 14:33:15,429 [168.20.36:53310] ERROR AbstractStoreCursor
- Failed to fill batch
org.apache.activemq.kaha.RuntimeStoreException: java.io.IOException: Could
not locate data file data-topic-data-4
at
org.apache.activemq.kaha.impl.container.MapContainerImpl.getValue(MapContainerImpl.java:484)
at
org.apache.activemq.store.kahadaptor.KahaTopicReferenceStore.recoverNextMessages(KahaTopicReferenceStore.java:254)
at
org.apache.activemq.store.amq.AMQTopicMessageStore.recoverNextMessages(AMQTopicMessageStore.java:59)
at
org.apache.activemq.broker.region.cursors.TopicStorePrefetch.doFillBatch(TopicStorePrefetch.java:94)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:227)
at
org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:100)
at
org.apache.activemq.broker.region.cursors.StoreDurableSubscriberCursor.reset(StoreDurableSubscriberCursor.java:233)
at
org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:566)
at
org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:150)
at
org.apache.activemq.broker.region.DurableTopicSubscription.add(DurableTopicSubscription.java:197)
at
org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:580)
at
org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:427)
at org.apache.activemq.broker.region.Topic.send(Topic.java:365)
at
org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:350)
at
org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:437)
at
org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
at
org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
at
org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
at
org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:133)
at
org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
at
org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:639)
at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
..
To fix this problem, we have to delete the data store and let MQ recreate
it. My question is
that is there a way to recover from this situation? More importantly, is
there a way to
prevent ourself to get into this situation.
thanks
thomas
--
View this message in context:
http://www.nabble.com/5.2.0-dequeue-counter%3D0-pending-queue-stalls-tp21658012p21658012.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
