Hi I had a look at it. There are *no* issues in Camel. DeadLetterChannel does *not* loose messages.
The problem Christian has is that he has not properly configured error handling in Camel. He have mixed error handlers - default error handler - dead letter channel error handler And since when he sends to the SEDA queue from a producer that are handled by the default error handler, then his message is *not* moved to a dead letter queue, but returned back to the caller as an exception. (as it used the default error handler) Configuring Camel to use dead letter channel error handler for the entire routes remedy Christians problem. I have added an unit test to Camel at: http://svn.apache.org/viewvc?rev=811997&view=rev On Fri, Sep 4, 2009 at 9:04 AM, Bach Christian<[email protected]> wrote: > Heya Camel Riders > > I stumbled upon a problem on having DeadLetterChannel doing the error > handling in an staged architecture (all in-VM, non-distributed). Our > batch processing application (uses Camel 2.0 and) is losing messages. > > It seems to only happen under peak loads, when SEDA queues become full > and start to throw IllegalStateExceptions on performing the add()s to > their internal BlockingQueues. Our producer recovers those exchanges and > yield()s, before re-sending the Exchanges to the SEDA endpoint. > > I have put together two unit tests that reproducibly illustrate the > problem and the absence thereof. Both unit tests are similar expect for > the dead letter queue exception handling implementation used. Both have > the SEDA queues, facing stages, configured to size=1 in order to emulate > peak load semantics. > > The first test case (DeadLetterChannelTest) is configured to use Camel's > DeadLetterChannel and reproducibly loses messages, depending on the run > between 5 to 15 (out of 45) on my machine. > > The second test case (ExceptionBarrierTest) is configured to use a > custom ExceptionBarrier endpoint in place of DeadLetterChannel. The > ExceptionBarrierComponent is derived from a Camel DirectComponent with > some custom code to handle Exchanges suffering from exceptions and > forwarding the erroneous exchanges to the dead letter queue. With the > ExceptionBarrier, which takes in its simplistic implementation advantage > of our specific routes to some extent, no messages are lost. > > The unit test are attached and I'd be most interested in learning what > is going wrong. > > Cheers, Christian. > > > <<camel-lost-messages.zip>> > > This message is for the named person's use only. It may contain confidential, > proprietary or legally privileged information. If you receive this message in > error, please notify the sender urgently and then immediately delete the > message and any copies of it from your system. Please also immediately > destroy any hardcopies of the message. > The sender's company reserves the right to monitor all e-mail communications > through their networks. > -- Claus Ibsen Apache Camel Committer Open Source Integration: http://fusesource.com Blog: http://davsclaus.blogspot.com/ Twitter: http://twitter.com/davsclaus
