Hi Camel users TLDR: - how to implement wait/retry loop? - messages "wait" in a queue - each message should be reprocessed only once per interval (let's say per day) - after reprocessing a message either returns to the waiting queue or continues (goes out of loop) - after [maxRetry] attempts it is moved away (goes out of loop)
Full story: As part of a typical processing workflow, I have a queue with "waiting" messages. These messages are waiting for data of another system and I have no idea when the data arrives. Therefore I want to reprocess the messages in an interval, let's say once a day. If the data is still missing, they go again to the waiting queue. To move messages away that would wait forever, I set and check a maxRetry. How would I best implement this? Some thoughts I made: 1. Just throw an exception if the data is missing. The message is redelivered. Problem: The message is instantly redelivered. So it does the retries within milliseconds. I cannot customize the interval. 2. Use the Camel delayer. Works good in general (my current solution). I can even calculate the delay based on the JMSTimestamp to take into account the time a message waits to be consumed. Problem: For an interval of 1 day, a message is permanently in processing state. The consumer is blocked. When I shutdown the container, Camel waits 5 minutes and after restarting the message goes to the DLQ. 3. Use a CronScheduledRoutePolicy to start/stop the route in the given interval. Problem: When I start the route once a day for 5 minutes, the messages with missing data are circulating until the route is stopped again. So they try to find the data multiple times per second and instantly reach the maxRetry. They must try only once per interval. 4. Use a JMS selector as from-Endpoint that selects "JMSTimestamp < (currentTime - intervalTime)" Problem: The from-Endpoint URI seems to be a constant, I can't make the selector dynamic. 5. Use the Camel Throttler Problem: The throttler sets a bandwith, that is not what we need. 6. Use a Timer from-Endpoint and hand over to a bean that consumes the messages (newest idea, not yet tried). I guess I could manage to consume all waiting messages just once (even if they already return before I am finished). Problems I can think of: It is not very nice to build another JMS consumer (beside Camel), but I could use Spring JMS Template. Numbers 1, 3, 4 and 5 simply do not work (or I don't use them the right way) for this case. Number 2 is my current solution, number 6 a new idea. Any recommendations or other suggestions how to build this wait/retry loop? Thanks a lot Stephan
