Hi Jim, Beside Babak's reply, if you still want the competing behavior, you can try the asyncConsumer option on the consumer, be warned about the message's order, I have not tested that myself though. Regards, Ngoc Hai.
________________________________ From: Jim Newsham <[email protected]> To: [email protected] Sent: Wednesday, February 15, 2012 8:15 AM Subject: Re: jms concurrentConsumers is not competing consumers Well, I haven't gotten a satisfactory response to this inquiry. Really looks like a bug to me. Any Camel contributors care to comment? Thanks, Jim On 2/6/2012 2:22 PM, Jim Newsham wrote: > > The MEP is InOut ("request/reply"). There are two participants, a consumer > and a producer. We need concurrent handling on the consumer (receiver, > request processor) side. From my reading of the documentation, the > "concurrentConsumers" option should provide concurrency for the consumer > side, and the "threads()" dsl should provide concurrency on the producer side. > > Although my use case is InOut (not InOnly), I agree with your statement > "concurrentConsumers should do it". It's mostly doing it, but there is a > critical liveness issue as I have described. > > Regarding the route config, see my other email from a few hours ago. It has > full Java code for running a server and client demo of the problem, including > initialization of routes. > > Thanks, > Jim > > > On 2/6/2012 1:58 PM, David Karlsen wrote: >> In the original mail you stated "option on our jms consumer endpoint >> so that we can handle numerous InOut requests" which I read as >> Producer with InOut MEP - and from the docs I read that should use the >> threads options. >> If you only have incoming (e.g. InOnly) then I read the docs as >> concurrentConsumers should do it. >> >> Maybe you could show the route config? >> >> 2012/2/6 Jim Newsham<[email protected]>: >>> Thanks David for the suggestion, but the documentation says that this option >>> is for concurrently handling replies. I am looking to concurrently handle >>> requests. Regardless of the documentation, I just gave this a try and it >>> does not fix the issue. >>> >>> Thanks again, >>> Jim >>> >>> On 2/6/2012 10:01 AM, David Karlsen wrote: >>>> Read the section on "Request-reply over JMS" - you're looking for the >>>> threads option. >>>> >>>> 2012/2/6 Jim Newsham<[email protected]>: >>>>> Hi Ngoc Hai, >>>>> >>>>> I'm not talking at all about the producer side where replies are >>>>> processed, >>>>> but the consumer side where requests are being received and processed. >>>>> Incoming requests are not being dispatched to idle threads; instead, >>>>> they >>>>> are backing up behind a single (non-idle) thread. >>>>> >>>>> To me, this looks like a bug. I am interested in hearing the opinions of >>>>> the community on this point, and I'm willing to file a JIRA issue. >>>>> >>>>> Thanks, >>>>> Jim >>>>> >>>>> On 2/3/2012 6:32 PM, Pham Ngoc Hai wrote: >>>>>> Hi Jim, >>>>>> Since it's InOut queue, the reply is processed by a single thread as >>>>>> stated in here: http://camel.apache.org/jms.html no matter how many >>>>>> concurrentConsumer you set. >>>>>> Regards, >>>>>> Ngoc Hai. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ________________________________ >>>>>> From: Jim Newsham<[email protected]> >>>>>> To: [email protected] >>>>>> Sent: Saturday, February 4, 2012 10:55 AM >>>>>> Subject: jms concurrentConsumers is not competing consumers >>>>>> >>>>>> >>>>>> We are using Camel 2.9.0. We are using the "concurrentConsumers=n" >>>>>> option >>>>>> on our jms consumer endpoint so that we can handle numerous InOut >>>>>> requests >>>>>> concurrently. In Camel documentation, this option is described as >>>>>> supporting competing consumers, which I interpret to mean a pool of >>>>>> threads >>>>>> which are _competing_ to handle incoming requests. I expect that >>>>>> whenever a >>>>>> request comes in, that request will be processed by some idle thread >>>>>> from >>>>>> the thread pool (if there are any). However, this is not that behavior >>>>>> we >>>>>> are observing. Instead, I observe some requests not being processed >>>>>> immediately (or for quite some time), despite having numerous idle >>>>>> consumer >>>>>> threads, and this is causing a severe liveness issue for our >>>>>> application. >>>>>> >>>>>> Let me describe a simple scenario. Assume we have a single InOut jms >>>>>> queue with concurrentConsumers=3, and the following stream of messages >>>>>> (and >>>>>> required processing time): >>>>>> >>>>>> Message #0 - 1h >>>>>> Message #1 - 1s >>>>>> Message #2 - 1s >>>>>> Message #3 - 1s >>>>>> Message #4 - 1s >>>>>> Message #5 - 1s >>>>>> Message #6 - 1s >>>>>> Message #7 - 1s >>>>>> Message #8 - 1s >>>>>> Message #9 - 1s >>>>>> Message #10 - 1s >>>>>> >>>>>> Because Message #0 requires 1 hour to process, it will occupy one >>>>>> concurrent consumer thread for 1 hour. Nevertheless, we expect the >>>>>> remaining 2 threads should be able to handle the remaining 9 messages >>>>>> very >>>>>> quickly. Instead, messages #3, #6, #9 are not processed until Message >>>>>> #0 >>>>>> completes processing -- an hour later. Based on this observation, it >>>>>> appears that messages are dispatched to queues owned by individual >>>>>> threads, >>>>>> in round-robin fashion, instead of having threads compete to consume >>>>>> messages from a single queue. When some messages can take a long time >>>>>> to >>>>>> process, it can mean that messages can queue up and go unprocessed for a >>>>>> very long time, even when there are free threads. >>>>>> >>>>>> As you can see, this is not "competing consumers". Is the above >>>>>> behavior >>>>>> considered a bug? Are there any suggested workarounds? >>>>>> >>>>>> Thanks! >>>>>> Jim >>>>> >>>> >> >> >
