Re: Camel Quartz Query
Hi Can you use an idempotent consumer[1] before sending to the web service? http://camel.apache.org/idempotent-consumer.html If you did have a sort of 'busy' status, you'd have to handle the situation where records are left in that 'busy' state due to a JVM crash for example. Taariq On Wed, Sep 25, 2013 at 5:20 PM, gilboy josephoto...@gmail.com wrote: Hi I have a quartz job(stateful) which wakes up every 60 seconds, pulls unprocessed records from a DB and sends each of them to a Web Service Endpoint. The Web Service call introduces a latency of approx .5 sec. Each time the quartz job fires there can be 100's of unprocessed records. Hence, after retrieving the records I iterate over the records via the splitter and I load balance(sticky - need to maintain order across subsets of the records) to 10 different seda queues. I then have 10 different routes which takes from each of the queues, transmits the message to the web service endpoint and then updates the record to be in a processed state. However, with the above approach I could easily send duplicates to the web service endpoint because a record which is returned when the quartz job fires at time 0 could be sitting on one of the seda queues at time 60 when the quartz job fires again and hence it is still in an unprocessed state in the database as it only gets updated to processed when we get a response from the Web Service call. I could solve this by introducing a new interim status and have the DB update the unprocessed records to the new interim status but just wanted to check to see if there was any other way of solving this as I have less control over the DB Thanks Joe -- View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Query-tp5740206.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Camel Quartz Query
+1 idempotent consumer. that should work nicely. On Wed, Sep 25, 2013 at 10:59 PM, Taariq Levack taar...@gmail.com wrote: Hi Can you use an idempotent consumer[1] before sending to the web service? http://camel.apache.org/idempotent-consumer.html If you did have a sort of 'busy' status, you'd have to handle the situation where records are left in that 'busy' state due to a JVM crash for example. Taariq On Wed, Sep 25, 2013 at 5:20 PM, gilboy josephoto...@gmail.com wrote: Hi I have a quartz job(stateful) which wakes up every 60 seconds, pulls unprocessed records from a DB and sends each of them to a Web Service Endpoint. The Web Service call introduces a latency of approx .5 sec. Each time the quartz job fires there can be 100's of unprocessed records. Hence, after retrieving the records I iterate over the records via the splitter and I load balance(sticky - need to maintain order across subsets of the records) to 10 different seda queues. I then have 10 different routes which takes from each of the queues, transmits the message to the web service endpoint and then updates the record to be in a processed state. However, with the above approach I could easily send duplicates to the web service endpoint because a record which is returned when the quartz job fires at time 0 could be sitting on one of the seda queues at time 60 when the quartz job fires again and hence it is still in an unprocessed state in the database as it only gets updated to processed when we get a response from the Web Service call. I could solve this by introducing a new interim status and have the DB update the unprocessed records to the new interim status but just wanted to check to see if there was any other way of solving this as I have less control over the DB Thanks Joe -- View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Query-tp5740206.html Sent from the Camel - Users mailing list archive at Nabble.com. -- *Christian Posta* http://www.christianposta.com/blog twitter: @christianposta
Camel Quartz Query
Hi I have a quartz job(stateful) which wakes up every 60 seconds, pulls unprocessed records from a DB and sends each of them to a Web Service Endpoint. The Web Service call introduces a latency of approx .5 sec. Each time the quartz job fires there can be 100's of unprocessed records. Hence, after retrieving the records I iterate over the records via the splitter and I load balance(sticky - need to maintain order across subsets of the records) to 10 different seda queues. I then have 10 different routes which takes from each of the queues, transmits the message to the web service endpoint and then updates the record to be in a processed state. However, with the above approach I could easily send duplicates to the web service endpoint because a record which is returned when the quartz job fires at time 0 could be sitting on one of the seda queues at time 60 when the quartz job fires again and hence it is still in an unprocessed state in the database as it only gets updated to processed when we get a response from the Web Service call. I could solve this by introducing a new interim status and have the DB update the unprocessed records to the new interim status but just wanted to check to see if there was any other way of solving this as I have less control over the DB Thanks Joe -- View this message in context: http://camel.465427.n5.nabble.com/Camel-Quartz-Query-tp5740206.html Sent from the Camel - Users mailing list archive at Nabble.com.
