Re: Messages being lost from route

2011-10-04 Thread Alistair Young
I think camel + tomcat is a non starter. The same config works perfectly in activemq 5.5.0 standalone with camel. Used embedded in a webapp in tomcat 6, only the first message makes it through the route. The rest vanish. Alistair -- mov eax,1 mov ebx,0 int 80h On 4 Oct 2011, at 08:43, Alis

Re: Messages being lost from route

2011-10-04 Thread Alistair Young
why would adding this: to this: org.apache.activemq.camel.component.ActiveMQComponent cause the broker to stop working? There are no transacted routes. The faster the messages come in, the more disappear. I have a very simple config that is guaranteed to lose 99/100 messages if the broker is

Re: Messages being lost from route

2011-10-02 Thread Alistair Young
eventually found the problem. if transactions are enabled nothing works. Only the first message gets through the route, the rest disappear. The problems start by adding transacted and transactionManager to the ActiveMQComponent as per: http://camel.apache.org/transactional-client.html so I'm n

Re: Messages being lost from route

2011-09-30 Thread Alistair Young
thanks Tim, that would be very helpful of you. I was about to dive into the camel source to see why the route wouldn't accept anything but if you have time to do a small test project I would be very grateful. I'm running camel 2.8.1, activemq 5.5.0, spring 3.0.5 under tomcat. I've attached my p

Re: Messages being lost from route

2011-09-30 Thread Tim
Alistair. I just tried the same with an embedded activemq instance and it works great. Maybe this has to do with broker settings? I can setup a tiny test project for you if you want to try that? On Fri, Sep 30, 2011 at 11:27 AM, Alistair Young wrote: > I can now reproduce this every time. Sending

Re: Messages being lost from route

2011-09-30 Thread Alistair Young
I can now reproduce this every time. Sending 100 messages in very quick succession to a camel route. JMX: EnqueueCount = 100 DispatchCount = 100 InFlightCount = 100 DequeueCount = 1 only 1 message ever gets through the route. That's 99 percent message loss, every time.

Re: Messages being lost from route

2011-09-30 Thread Alistair Young
getting somewhere. - producer clock is 2mins ahead of broker clock, timestampplugin enabled on broker - no delay between messages at the producer = 62 out of 100 get through the route, consistently - 3sec delay at producer, around 88 - 97 get through. That'

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
no connection pool. What was disturbing was the first message sent to the broker after a restart and clean out of the data dir, disappeared. There's a similar route on the broker that works fine. The only difference is the producer for the wonky route is on windows and is up to 1min ahead of the

Re: Messages being lost from route

2011-09-29 Thread Tim
Well with only 17 you definitely aren't hitting any prefetch limits or anything like that. Are you using a connection pool? On Thu, Sep 29, 2011 at 5:05 PM, Alistair Young wrote: > I think this way madness lies. > > 17 sent to topicA, dispatchCount = 15, dequeueCount = 12 > topicB enqueueCount =

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
I think this way madness lies. 17 sent to topicA, dispatchCount = 15, dequeueCount = 12 topicB enqueueCount = 12 so 17 came in, 12 made it through, of the 5 that went missing it claims to have sent 3 to topicB but they never arrived and the last 2 just simply vanished completely. What on earth

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
nup - cleaned out the data dir and restarted the broker. First message in vanished. Wasn't persisted. So something is fundamentally broken. topicA inflightCount = dispatchCount = enqueueCount = 1 topicB is completely empty so the message wasn't persisted, wasn't processed, wasn't routed and just

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
route goes from topicA -> topicB, transacted. topicA inflightCount = 96 and increases on each batch of incoming messages topicB dispatchCount = enqueueCount wondering if the missing messages are connected to topicA inflightCount. I noticed there are two consumers for topicB. The main consumer get

Re: Messages being lost from route

2011-09-29 Thread Tim
Sorry you might have tried this since I haven't been following this thread. But can you check your jmx console. In particular check 2 things.. the route to see if the number of exchanges match what you think and how if any exchanges failed. Also check the JMX console on activemq for the queue or to

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
dunno - nothing works. Random messages are just vanishing once they reach the broker. No trace, no logs, no dead letter queue. Just vanishing. I've removed and but it doesn't help. The producer is a few secs behind the broker: sent : 11:25:26 arrived : 11:24:57 timstamp on message : 131729189

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
just saw your info about transacted being before from - will change that and monitor again. thanks, Alistair -- mov eax,1 mov ebx,0 int 80 On 29 Sep 2011, at 10:18, Alistair Young wrote: > just noticed a batch of identical 5 messages, three were missing and another > single messa

Re: Messages being lost from route

2011-09-29 Thread Claus Ibsen
Hi For topics you ought to set an unique clientId and subscription name the consumer side. http://camel.apache.org/jms http://activemq.apache.org/how-do-durable-queues-and-topics-work.html On Thu, Sep 29, 2011 at 11:18 AM, Alistair Young wrote: > just noticed a batch of identical 5 messages, t

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
just noticed a batch of identical 5 messages, three were missing and another single message vanished. tracer logged nothing. No errors, dead letter queue empty. One thing that happens is another machine polls the stats topic in activemq every 2mins. Would that cause a problem? It asks for stats

Re: Messages being lost from route

2011-09-29 Thread Claus Ibsen
Yes should be before the . We may add a validation check in the future | Camel 3.0 for that so you cannot mis configure it. On Thu, Sep 29, 2011 at 10:36 AM, Alistair Young wrote: >> Should be after > it is after from - do you mean it should be before? >     >       >       >       >       >

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
> Should be after it is after from - do you mean it should be before? thanks for the dead letter tips, will apply them. Alistair -- mov eax,1 mov ebx,0 int 80 On 29 Sep 2011, at 09:20, Claus Ibsen wrote: > Hi > > Should be after > > > On

Re: Messages being lost from route

2011-09-29 Thread Claus Ibsen
And its better to use the native redelivery / dead letter queue, features of the AMQ message broker, than from Camel. http://activemq.apache.org/message-redelivery-and-dlq-handling.html http://activemq.apache.org/redelivery-policy.html So remove the matrixDeadLetterErrorHandler and use the DLQ fro

Re: Messages being lost from route

2011-09-29 Thread Claus Ibsen
Hi Should be after On Thu, Sep 29, 2011 at 10:09 AM, Alistair Young wrote: >> Do you use message expiry? > no > >> timestamp plugin > > using that > > activemq 5.5.0 > camel 2.8.0 > spring 3.0.5 > > noticed sl4j errors on startup, fixed that and now the tracer is logging so > hopefully I can

Re: Messages being lost from route

2011-09-29 Thread Alistair Young
> Do you use message expiry? no > timestamp plugin using that activemq 5.5.0 camel 2.8.0 spring 3.0.5 noticed sl4j errors on startup, fixed that and now the tracer is logging so hopefully I can see any errors.

Re: Messages being lost from route

2011-09-29 Thread Claus Ibsen
Hi Do you use message expiry? Make sure clocks between server/clients is synced as much as possible. There is a timestamp plugin http://activemq.apache.org/timestampplugin.html And do you use queue or topic. What version of AMQ and Camel are you using? And how have you configured the AMQ broker,

Re: Messages being lost from route

2011-09-28 Thread Taariq Levack
Hi Where the logs go, if it's logged at all, still depends on your logger and how you configured it. Here are links to how to enable logging[1] and camel logging FAQ[2] [1]http://camel.apache.org/how-do-i-enable-debug-logging.html [2]http://camel.apache.org/logging-questions.html Taariq On We

Re: Messages being lost from route

2011-09-28 Thread Alistair Young
which is the best trace method to use? trace="true", or camelTracer and traceFormatter beans? and where does the log end up? I've tried them all but no log appears. Alistair -- mov eax,1 mov ebx,0 int 80h On 28 Sep 2011, at 12:08, Marco Westermann wrote: > Hi, > > I suggest enable tracin

Re: Messages being lost from route

2011-09-28 Thread Marco Westermann
Hi, I suggest enable tracing to see exactly what happens in your route. regards, Marco Am 28.09.2011 13:01, schrieb Alistair Young: I now have a dead letter channel which is empty after losing 9 out of 10 messages. I also added a logging handler which logged nothing. Verified the messages ar

Re: Messages being lost from route

2011-09-28 Thread Alistair Young
I now have a dead letter channel which is empty after losing 9 out of 10 messages. I also added a logging handler which logged nothing. Verified the messages arrived at the broker, then they just vanished. Alistair -- mov eax,1 mov ebx,0 int 80h On 28 Sep 2011, at 11:42, Alistair Young wro

Re: Messages being lost from route

2011-09-28 Thread Alistair Young
just noticed this: Default Error Handler: "whenever an Exchange could not be processed the error is propagated back to the client" the client sends to topic://A and camel routes to topic://B. If the routing fails will the Default Error Handler tell the client, even though the client doesn't kn

Re: Messages being lost from route

2011-09-28 Thread Alistair Young
thanks for that. Have set one up now and will watch it. Alistair -- mov eax,1 mov ebx,0 int 80h On 28 Sep 2011, at 11:21, Mick Knutson wrote: > Do you have a deadLetterChannel setup in case of message exceptions? > > --- > Thank You… > > Mick Knutson, President > > BASE Logic, Inc. > Ent

Re: Messages being lost from route

2011-09-28 Thread Mick Knutson
Do you have a deadLetterChannel setup in case of message exceptions? --- Thank You… Mick Knutson, President BASE Logic, Inc. Enterprise Architecture, Design, Mentoring & Agile Consulting p. (855) BASE-LOGIC: (227-3564-42) p. (478) BASE-LOGIC (227-3564-42) f. (855) BASE-LOGIC: (227-3564-42) Webs

Messages being lost from route

2011-09-28 Thread Alistair Young
Hi there, is there anything that would cause camel to ignore messages in a route? I'm getting persistent messages simply disappearing from a transacted route. I've verified the messages arrive at the activemq broker (tcpdump) and also that on one occasion, 5 out of 6 of the messages just disapp