nup - cleaned out the data dir and restarted the broker. First message in
vanished. Wasn't persisted. So something is fundamentally broken.
topicA inflightCount = dispatchCount = enqueueCount = 1
topicB is completely empty
so the message wasn't persisted, wasn't processed, wasn't routed and just
vanished from the broker.
Alistair
--------------
mov eax,1
mov ebx,0
int 80
On 29 Sep 2011, at 15:13, Alistair Young wrote:
> route goes from topicA -> topicB, transacted.
> topicA inflightCount = 96 and increases on each batch of incoming messages
> topicB dispatchCount = enqueueCount
>
> wondering if the missing messages are connected to topicA inflightCount. I
> noticed there are two consumers for topicB. The main consumer gets its
> messages fine. Wonder if the second consumer is a durable topic consumer and
> therefore activemq is persisting its messages but it hasn't connected in a
> very long time. Would that cause the topic to get too big? i.e. messages go
> into the topic until the limit is reached. Main consumer pulls messages off
> and messages are able to go onto topicB again. Before consumer pulls and
> after limit reached, messages can't get from topicA -> topicB, hence the
> topicA inflightCount not zero?
>
> Alistair
>
> --------------
> mov eax,1
> mov ebx,0
> int 80
>
> On 29 Sep 2011, at 12:17, Tim wrote:
>
>> Sorry you might have tried this since I haven't been following this thread.
>> But can you check your jmx console.
>> In particular check 2 things.. the route to see if the number of exchanges
>> match what you think and how if any exchanges failed.
>> Also check the JMX console on activemq for the queue or topic in question
>> and see how many were enqueued vs dispatched.
>> Check your deadletter queue from there too
>>
>> On Thu, Sep 29, 2011 at 12:52 PM, Alistair Young
>> <alistair.yo...@uhi.ac.uk>wrote:
>>
>>> dunno - nothing works. Random messages are just vanishing once they reach
>>> the broker. No trace, no logs, no dead letter queue. Just vanishing. I've
>>> removed <transacted /> and <process> but it doesn't help. The producer is a
>>> few secs behind the broker:
>>>
>>> sent : 11:25:26
>>> arrived : 11:24:57
>>> timstamp on message : 1317291897071 = 29 Sep 2011 10:24:57 GMT, presumably
>>> the timestampplugin doing this
>>> message vanishes
>>>
>>> but all messages display this clock behaviour and not all vanish.
>>>
>>> Alistair
>>>
>>> --------------
>>> mov eax,1
>>> mov ebx,0
>>> int 80
>>>
>>> On 29 Sep 2011, at 10:24, Alistair Young wrote:
>>>
>>>> just saw your info about transacted being before from - will change that
>>> and monitor again.
>>>>
>>>> thanks,
>>>>
>>>> Alistair
>>>>
>>>> --------------
>>>> mov eax,1
>>>> mov ebx,0
>>>> int 80
>>>>
>>>> On 29 Sep 2011, at 10:18, Alistair Young wrote:
>>>>
>>>>> just noticed a batch of identical 5 messages, three were missing and
>>> another single message vanished. tracer logged nothing. No errors, dead
>>> letter queue empty.
>>>>>
>>>>> One thing that happens is another machine polls the stats topic in
>>> activemq every 2mins. Would that cause a problem? It asks for stats on the
>>> matrix topic, which is part of the transacted route.
>>>>>
>>>>> 29 September 2011 10:05:07 - Adding destination :
>>> Topic:ActiveMQ.Advisory.Connection
>>>>> 29 September 2011 10:05:07 - Creating new transaction with name [null]:
>>> PROPAGATION_REQUIRED,ISOLATION_DEFAULT
>>>>> 29 September 2011 10:05:07 - Stopping connection:
>>> vm://matrixBroker#285916
>>>>> 29 September 2011 10:05:07 - Stopped transport: vm://matrixBroker#285916
>>>>> 29 September 2011 10:05:07 - Connection Stopped:
>>> vm://matrixBroker#285916
>>>>> 29 September 2011 10:05:07 - Setting up new connection id:
>>> ID:prodprovisioning-matrix-41707-1317215126074-4:142961, address:
>>> vm://matrixBroker#285920
>>>>> 29 September 2011 10:05:07 - Adding Connection : ConnectionInfo
>>> {commandId = 1, responseRequired = true, connectionId =
>>> ID:prodprovisioning-matrix-41707-1317215126074-4:142961, clientId =
>>> ID:prodprovisioning-matrix-41707-1317215126074-5:142961, userName = null,
>>> password = *****, brokerPath = null, brokerMasterConnector = false,
>>> manageable = true, clientMaster = true, faultTolerant = false}
>>>>>
>>>>> Alistair
>>>>>
>>>>> --------------
>>>>> mov eax,1
>>>>> mov ebx,0
>>>>> int 80
>>>>>
>>>>> On 29 Sep 2011, at 09:36, Alistair Young wrote:
>>>>>
>>>>>>> <transacted/> Should be after <from>
>>>>>> it is after from - do you mean it should be before?
>>>>>> <route id="eDirSuccessBroadcast">
>>>>>> <from uri="activemq:topic:edirectoryprocessed"/>
>>>>>> <transacted />
>>>>>> <process ref="groupwiseProcessor" />
>>>>>> <to uri="activemq:topic:blackboard"/>
>>>>>> </route>
>>>>>>
>>>>>> thanks for the dead letter tips, will apply them.
>>>>>>
>>>>>> Alistair
>>>>>>
>>>>>> --------------
>>>>>> mov eax,1
>>>>>> mov ebx,0
>>>>>> int 80
>>>>>>
>>>>>> On 29 Sep 2011, at 09:20, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> <transacted/> Should be after <from>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Sep 29, 2011 at 10:09 AM, Alistair Young
>>>>>>> <alistair.yo...@uhi.ac.uk> wrote:
>>>>>>>>> Do you use message expiry?
>>>>>>>> no
>>>>>>>>
>>>>>>>>> timestamp plugin
>>>>>>>>
>>>>>>>> using that
>>>>>>>>
>>>>>>>> activemq 5.5.0
>>>>>>>> camel 2.8.0
>>>>>>>> spring 3.0.5
>>>>>>>>
>>>>>>>> noticed sl4j errors on startup, fixed that and now the tracer is
>>> logging so hopefully I can see any errors.
>>>>>>>>
>>>>>>>> <route id="matrix" errorHandlerRef="matrixDeadLetterErrorHandler">
>>>>>>>> <from uri="activemq:topic:matrix"/>
>>>>>>>> <process ref="matrixProcessor" />
>>>>>>>> <transacted />
>>>>>>>> <to uri="activemq:topic:edirectory"/>
>>>>>>>> </route>
>>>>>>>>
>>>>>>>> <bean id="jmsConnectionFactory"
>>> class="org.apache.activemq.ActiveMQConnectionFactory"
>>> depends-on="matrixBrokerID">
>>>>>>>> <property name="brokerURL"
>>> value="vm://matrixBroker?create=false"/>
>>>>>>>> </bean>
>>>>>>>>
>>>>>>>> <bean id="jmsTransactionManager"
>>> class="org.springframework.jms.connection.JmsTransactionManager">
>>>>>>>> <property name="connectionFactory"
>>> ref="jmsConnectionFactory"/>
>>>>>>>> </bean>
>>>>>>>>
>>>>>>>> <bean id="activemq"
>>> class="org.apache.activemq.camel.component.ActiveMQComponent">
>>>>>>>> <property name="connectionFactory"
>>> ref="jmsConnectionFactory"/>
>>>>>>>> <property name="transacted" value="true"/>
>>>>>>>> <property name="transactionManager"
>>> ref="jmsTransactionManager"/>
>>>>>>>> </bean>
>>>>>>>>
>>>>>>>> <bean id="matrixDeadLetterErrorHandler"
>>> class="org.apache.camel.builder.DeadLetterChannelBuilder">
>>>>>>>> <property name="deadLetterUri" value="jms:queue:dead"/>
>>>>>>>> <property name="redeliveryPolicy"
>>> ref="matrixRedeliveryPolicyConfig"/>
>>>>>>>> </bean>
>>>>>>>>
>>>>>>>> <bean id="matrixRedeliveryPolicyConfig"
>>> class="org.apache.camel.processor.RedeliveryPolicy">
>>>>>>>> <property name="maximumRedeliveries" value="10"/>
>>>>>>>> <property name="redeliveryDelay" value="250"/>
>>>>>>>> </bean>
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> Alistair
>>>>>>>>
>>>>>>>>
>>>>>>>> --------------
>>>>>>>> mov eax,1
>>>>>>>> mov ebx,0
>>>>>>>> int 80
>>>>>>>>
>>>>>>>> On 29 Sep 2011, at 08:53, Claus Ibsen wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Do you use message expiry?
>>>>>>>>> Make sure clocks between server/clients is synced as much as
>>> possible.
>>>>>>>>>
>>>>>>>>> There is a timestamp plugin
>>>>>>>>> http://activemq.apache.org/timestampplugin.html
>>>>>>>>>
>>>>>>>>> And do you use queue or topic.
>>>>>>>>> What version of AMQ and Camel are you using?
>>>>>>>>> And how have you configured the AMQ broker, and the Camel context?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Sep 29, 2011 at 7:21 AM, Taariq Levack <taar...@gmail.com>
>>> wrote:
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> Where the logs go, if it's logged at all, still depends on your
>>> logger and
>>>>>>>>>> how you configured it.
>>>>>>>>>>
>>>>>>>>>> Here are links to how to enable logging[1] and camel logging FAQ[2]
>>>>>>>>>>
>>>>>>>>>> [1]http://camel.apache.org/how-do-i-enable-debug-logging.html
>>>>>>>>>> [2]http://camel.apache.org/logging-questions.html
>>>>>>>>>>
>>>>>>>>>> Taariq
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Sep 28, 2011 at 1:23 PM, Alistair Young <
>>> alistair.yo...@uhi.ac.uk>wrote:
>>>>>>>>>>
>>>>>>>>>>> which is the best trace method to use? trace="true", or
>>> camelTracer and
>>>>>>>>>>> traceFormatter beans? and where does the log end up? I've tried
>>> them all but
>>>>>>>>>>> no log appears.
>>>>>>>>>>>
>>>>>>>>>>> Alistair
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> mov eax,1
>>>>>>>>>>> mov ebx,0
>>>>>>>>>>> int 80h
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 28 Sep 2011, at 12:08, Marco Westermann wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I suggest enable tracing to see exactly what happens in your
>>> route.
>>>>>>>>>>>>
>>>>>>>>>>>> regards, Marco
>>>>>>>>>>>>
>>>>>>>>>>>> Am 28.09.2011 13:01, schrieb Alistair Young:
>>>>>>>>>>>>> I now have a dead letter channel which is empty after losing 9
>>> out of 10
>>>>>>>>>>> messages. I also added a logging handler which logged nothing.
>>> Verified the
>>>>>>>>>>> messages arrived at the broker, then they just vanished.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alistair
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> -----------------
>>>>>>>>> FuseSource
>>>>>>>>> Email: cib...@fusesource.com
>>>>>>>>> Web: http://fusesource.com
>>>>>>>>> Twitter: davsclaus, fusenews
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> -----------------
>>>>>>> FuseSource
>>>>>>> Email: cib...@fusesource.com
>>>>>>> Web: http://fusesource.com
>>>>>>> Twitter: davsclaus, fusenews
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>
>>>>>
>>>>
>>>
>>>
>
