Hi Christian, thanks for your interest. As I said the size of the application is very large, we have around 150 route builders spread over 5 or so processes. I can't easily boil down our issue to a few configurations or route builders. I was just trying to see if issues like this have been encountered before. I am trying to make a case for upgrading ActiveMQ and Camel but would like to know if the camel community believes that will clear up this issue.
Thanks Sam -----Original Message----- From: Christian Posta [mailto:christian.po...@gmail.com] Sent: Thursday, September 26, 2013 11:07 AM To: users@camel.apache.org Subject: Re: Failover JMS and Camel DefaultTimeoutManager failures Post your route, your configs, and a description of what it does, who are its clients, etc. Also link to the bug in the current code. On Thu, Sep 26, 2013 at 8:02 AM, Hendley, Sam <sam.hend...@sensus.com>wrote: > Hello all. > > I am working with a large application written in camel. We have > recently started noticing (it may have been occurring for a while) > that in some cases a sort of death spiral occurs. > > Out setup: > Camel 2.10.1 (unfortunately we can't upgrade) ActiveMQ 5.6.0 > (unfortunately we can't upgrade) > > As far as I can tell this is what is occurring: > > · Something puts excess load on the system and slows down our > application > > · Some ReplyManagerTimeoutChecks start failing > > · It starts doing excessive amounts of work dealing with timeouts > which delays the processing of the work causing more timeouts > > o Using a sampler it appears most the time is spent in > DefaultTimeoutMap.purge() > > § There is a .size() call that is very expensive which is discarded > unless trace level logging is turned on (this bug still exists in the > current camel code) > > · The activemq connections start timing out and disconnecting from > the broker side > > · This appears to cause the camel routes to spawn a new set of > TimeoutChecker and ReplyManager threads for each of the routes which > are all > > Has anyone seen issue like this? Any advice on what else I should be > looking for? > > Thanks > > Sam Hendley > -- *Christian Posta* http://www.christianposta.com/blog twitter: @christianposta
