Hi Its bad practice to have so long redelvery intervals. Its is intended for short time processing. If you want so long then off load the messages to some persistent store and process from there.
Though we have a ticket in JIRA to allow Camel's error handler to discard those pending redeliveries in can of shutdown etc. But its not implemented yet. On Mon, Jul 13, 2015 at 11:18 AM, jnupponen <[email protected]> wrote: > Hi! > > I've run into a problem with camel-quartz2 (at least on 2.13.2 and 2.15.2) > cron scheduling and AsyncDelayedDelivery-option in the DefaultErrorHandler. > If I want to shutdown/restart routes during DefaultErrorHandlers > redeliveries quartz scheduling gets deadlocked and quietly fails all future > fire times. Without AsyncDelayedDelivery-option this problem does not occur. > I'm running my routes in bundles in Karaf 3.0.3 but the problem also occurs > when running the code on jvm (at least on 1.7) without Karaf. > > I have configured quite long redelivery times for my routes (e.g. 8 times in > 2 hour interval) but at times I would need to be able to restart bundles > running in Karaf even when there would be redeliveries still left (e.g. when > installing new version of the bundle). > > This seems like a bug to me but what's your take on this? I've attached few > lines of log and some example route code to the end of this message. > > Best regards, > - Jussi > > > > Logs on org.quartz debug level show every few seconds: > [artzScheduler-camel-1_Worker-2] SimpleThreadPool DEBUG > Waiting for thread DefaultQuartzScheduler-camel-1_Worker-2 to shut down > > and jstack shows for DefaultQuartzScheduler-camel-1_Worker-2: > "DefaultQuartzScheduler-camel-1_Worker-2" prio=10 tid=0x00007fd2f8201000 > nid=0x2a16 in Object.wait() [0x00007fd3039dc000] > java.lang.Thread.State: TIMED_WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00000007ae410420> (a java.lang.Object) > at > org.quartz.simpl.SimpleThreadPool.shutdown(SimpleThreadPool.java:371) > - locked <0x00000007ae410420> (a java.lang.Object) > at org.quartz.core.QuartzScheduler.shutdown(QuartzScheduler.java:740) > at org.quartz.impl.StdScheduler.shutdown(StdScheduler.java:206) > at > org.apache.camel.component.quartz2.QuartzComponent.doStop(QuartzComponent.java:352) > at > org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102) > at > org.apache.camel.util.ServiceHelper.stopService(ServiceHelper.java:141) > at > org.apache.camel.util.ServiceHelper.stopAndShutdownService(ServiceHelper.java:204) > at > org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2827) > at > org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2852) > at > org.apache.camel.impl.DefaultCamelContext.shutdownServices(DefaultCamelContext.java:2840) > at > org.apache.camel.impl.DefaultCamelContext.doStop(DefaultCamelContext.java:2704) > - locked <0x00000007ad1abd88> (a > org.apache.camel.impl.DefaultCamelContext) > at > org.apache.camel.support.ServiceSupport.stop(ServiceSupport.java:102) > at > com.example.route.MyAsyncDelayedDeliveryTesting$1.process(MyAsyncDelayedDeliveryTesting.java:43) > at > org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63) > at > org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) > at > org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191) > at > org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:44) > at > org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109) > at > org.apache.camel.processor.loadbalancer.LoadBalancerSupport.process(LoadBalancerSupport.java:87) > at > org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:56) > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > at > org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) > - locked <0x00000007ae539450> (a java.lang.Object) > > > Example of this situation looks like this: > > package com.example.route; > > import static org.apache.camel.LoggingLevel.ERROR; > import static org.apache.camel.LoggingLevel.WARN; > > import org.apache.camel.Exchange; > import org.apache.camel.Processor; > import org.apache.camel.builder.RouteBuilder; > import org.joda.time.DateTime; > > public class MyAsyncDelayedDeliveryTesting extends RouteBuilder { > > > @Override > public void configure() throws Exception { > > // Set lower timeout for convenience (default 300 seconds). > getContext().getShutdownStrategy().setTimeout(10); > > // Start some route1 cron scheduled. > DateTime now = DateTime.now().plusSeconds(2); > from("quartz2://my_route_quartz_timer?cron=" > +now.getSecondOfMinute()+"+" > +now.getMinuteOfHour()+"+" > +now.getHourOfDay()+"+*+*+?") > .id("scheduling1") > .to("direct://my-route"); > > > // Start other route that shutdowns the context thus shutting > down the > route1. > // Simulate the situation where you want to restart your > route when there > are still > // ongoing redelivery tasks. > now = DateTime.now().plusSeconds(4); > from("quartz2://my_shutdown_route_quartz_timer?cron=" > +now.getSecondOfMinute()+"+" > +now.getMinuteOfHour()+"+" > +now.getHourOfDay()+"+*+*+?") > .id("scheduling2") > .process(new Processor() { > > @Override > public void process(Exchange arg0) throws Exception { > getContext().stop(); > > } > }); > > // Do some logic that causes exception and redeliveries for 1 > minute. > from("direct://my-route") > .id("route1") > .errorHandler(defaultErrorHandler() > .maximumRedeliveries(10) > .redeliveryDelay(1000*6) > // asyncDelayedRedelivery causes the problem. > .asyncDelayedRedelivery() > .retryAttemptedLogLevel(WARN) > .retriesExhaustedLogLevel(ERROR)) > > .process(new Processor() { > > @Override > public void process(Exchange exchange) throws > Exception { > throw new RuntimeException("Error"); > } > }); > } > } > > Full example is visible here: > https://github.com/jnupponen/camel-quartz2-asyncDelayedDelivery-problem > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Quartz2-problem-with-AsyncDelayedDelivery-tp5769210.html > Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen ----------------- Red Hat, Inc. Email: [email protected] Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen hawtio: http://hawt.io/ fabric8: http://fabric8.io/
