Re: Bounded concurrency on SQL consumer

2021-07-30 Thread Tomasz Melcer
On 7/20/21 11:29 AM, Tomasz Melcer wrote: Is it possible to have some kind of bounded concurrency on these consumers, so that we wouldn't have to manually manage the times? So, we ended up implementing a mutex as part of the SQL query itself using SQL Server's sp_getapplock. While it's not wha

Re: Bounded concurrency on SQL consumer

2021-07-27 Thread Tomasz Melcer
On 7/23/21 10:20 AM, mail4...@gmail.com wrote: But then we realized that the thread is used for the whole route, and not just for the consumer. Did you try to separate your consumer and the route with SEDA? Like: from(YourConsumerHere).to("seda://next-step"); from("seda://next-step").process(N

Re: Bounded concurrency on SQL consumer

2021-07-23 Thread mail4bvn
> But then we realized that the thread is used for the whole route, and > not just for the consumer. Did you try to separate your consumer and the route with SEDA? Like: from(YourConsumerHere) .to("seda://next-step"); from("seda://next-step") .process(NextYourStepsHere); It will use another thr

Re: Bounded concurrency on SQL consumer

2021-07-22 Thread Tomasz Melcer
On 7/21/21 3:23 PM, Tomasz Melcer wrote: On 7/20/21 2:59 PM, Zoran Regvart wrote: Perhaps you could try using a custom executor service (`scheduledExecutorService`) with a fixed number of threads? […] Well, so we've tested it and, as documentation states here [1] Notice: If using a custom sc

Re: Bounded concurrency on SQL consumer

2021-07-21 Thread Paul McCulloch
> > > The problem here is that each report might have a different > post-processing pipeline, which is—as far as I know, this decision was > taken 8 years ago—why they were created as separate routes. > > You could put each post processing pipeline in a separate route; set the name of that route i

Re: Bounded concurrency on SQL consumer

2021-07-21 Thread Tomasz Melcer
On 7/20/21 2:59 PM, Zoran Regvart wrote: Perhaps you could try using a custom executor service (`scheduledExecutorService`) with a fixed number of threads? Yes, that's roughly the direction I think we will be going towards. My colleague suggested it independently and it might be the simplest

Re: Bounded concurrency on SQL consumer

2021-07-21 Thread Tomasz Melcer
On 7/20/21 5:23 PM, Paul McCulloch wrote: You could use JMS. For each report define a route from timer/quartz to a JMS queue - perhaps setting the query or the report name as the body. You could have a single report execution route reading from the JMS queue, and then executing the report define

Re: Bounded concurrency on SQL consumer

2021-07-20 Thread Paul McCulloch
You could use JMS. For each report define a route from timer/quartz to a JMS queue - perhaps setting the query or the report name as the body. You could have a single report execution route reading from the JMS queue, and then executing the report defined in the message. Use concurrentConsumers to

Re: Bounded concurrency on SQL consumer

2021-07-20 Thread Zoran Regvart
Hi Tomasz, Perhaps you could try using a custom executor service (`scheduledExecutorService`) with a fixed number of threads? zoran On Tue, Jul 20, 2021 at 11:29 AM Tomasz Melcer wrote: > > Hi! > > We've got a Camel instance that handles, among other things, reporting > from SQL. We've got tens