Hi Ngoc Hai,
Thanks for the suggestion, the asyncConsumer option looks like it might
work. I actually implemented another workaround which is already
working for me, so I won't be trying the asyncConsumer option for now.
I wish I had known about the asyncConsumer option earlier though, since
the workaround I used is somewhat more complicated and ugly.
The workaround I am using involves:
- A jms consumer configured with "concurrentConsumers=1" and
"disableReplyTo=true"
- jms consumer forwards messages to a seda queue, configured with a
thread pool using threads() dsl. This is where our concurrent
processing is now done.
- seda queue handles the message as before (in our case, forwarding it
to another queue for processing).
- because jms queue is no longer handling the reply, the seda queue must
include a processor for handling the reply dynamically (reading
JMSReplyTo and JMSCorrelationID headers, and forwarding to the
appropriate dynamic destination using CamelJmsDestinationName).
- because jms queue can't propagate the exception to the sender (using
"transferException=true"), we needed to add an error handler to the jms
consumer route. The error handler must send the exception to the caller
using the same dynamic reply strategy as above (that is, using
JMSReplyTo, JMSCorrelationID, and CamelJmsDestinationName).
- in order to get the exception to propagate to the sender no matter
which route it happened in, we needed to disable the error handler on
*every downstream route* -- using .errorHandler(noErrorHandler()).
Since this is so ugly, I just may try the asyncConsumer option when I
get some time. However since it's currently working as-is, I can't
justify spending more time on it for now, given all of the other high
priority tasks I have on my plate.
Thanks for the suggestion!
Jim
On 2/22/2012 12:19 AM, Pham Ngoc Hai wrote:
Hi Jim,
Beside Babak's reply, if you still want the competing behavior, you can try the
asyncConsumer option on the consumer, be warned about the message's order, I
have not tested that myself though.
Regards,
Ngoc Hai.
________________________________
From: Jim Newsham<jnews...@referentia.com>
To: users@camel.apache.org
Sent: Wednesday, February 15, 2012 8:15 AM
Subject: Re: jms concurrentConsumers is not competing consumers
Well, I haven't gotten a satisfactory response to this inquiry. Really looks
like a bug to me. Any Camel contributors care to comment?
Thanks,
Jim
On 2/6/2012 2:22 PM, Jim Newsham wrote:
The MEP is InOut ("request/reply"). There are two participants, a consumer and a producer. We
need concurrent handling on the consumer (receiver, request processor) side. From my reading of the
documentation, the "concurrentConsumers" option should provide concurrency for the consumer side,
and the "threads()" dsl should provide concurrency on the producer side.
Although my use case is InOut (not InOnly), I agree with your statement
"concurrentConsumers should do it". It's mostly doing it, but there is a
critical liveness issue as I have described.
Regarding the route config, see my other email from a few hours ago. It has
full Java code for running a server and client demo of the problem, including
initialization of routes.
Thanks,
Jim
On 2/6/2012 1:58 PM, David Karlsen wrote:
In the original mail you stated "option on our jms consumer endpoint
so that we can handle numerous InOut requests" which I read as
Producer with InOut MEP - and from the docs I read that should use the
threads options.
If you only have incoming (e.g. InOnly) then I read the docs as
concurrentConsumers should do it.
Maybe you could show the route config?
2012/2/6 Jim Newsham<jnews...@referentia.com>:
Thanks David for the suggestion, but the documentation says that this option
is for concurrently handling replies. I am looking to concurrently handle
requests. Regardless of the documentation, I just gave this a try and it
does not fix the issue.
Thanks again,
Jim
On 2/6/2012 10:01 AM, David Karlsen wrote:
Read the section on "Request-reply over JMS" - you're looking for the
threads option.
2012/2/6 Jim Newsham<jnews...@referentia.com>:
Hi Ngoc Hai,
I'm not talking at all about the producer side where replies are
processed,
but the consumer side where requests are being received and processed.
Incoming requests are not being dispatched to idle threads; instead,
they
are backing up behind a single (non-idle) thread.
To me, this looks like a bug. I am interested in hearing the opinions of
the community on this point, and I'm willing to file a JIRA issue.
Thanks,
Jim
On 2/3/2012 6:32 PM, Pham Ngoc Hai wrote:
Hi Jim,
Since it's InOut queue, the reply is processed by a single thread as
stated in here: http://camel.apache.org/jms.html no matter how many
concurrentConsumer you set.
Regards,
Ngoc Hai.
________________________________
From: Jim Newsham<jnews...@referentia.com>
To: users@camel.apache.org
Sent: Saturday, February 4, 2012 10:55 AM
Subject: jms concurrentConsumers is not competing consumers
We are using Camel 2.9.0. We are using the "concurrentConsumers=n"
option
on our jms consumer endpoint so that we can handle numerous InOut
requests
concurrently. In Camel documentation, this option is described as
supporting competing consumers, which I interpret to mean a pool of
threads
which are _competing_ to handle incoming requests. I expect that
whenever a
request comes in, that request will be processed by some idle thread
from
the thread pool (if there are any). However, this is not that behavior
we
are observing. Instead, I observe some requests not being processed
immediately (or for quite some time), despite having numerous idle
consumer
threads, and this is causing a severe liveness issue for our
application.
Let me describe a simple scenario. Assume we have a single InOut jms
queue with concurrentConsumers=3, and the following stream of messages
(and
required processing time):
Message #0 - 1h
Message #1 - 1s
Message #2 - 1s
Message #3 - 1s
Message #4 - 1s
Message #5 - 1s
Message #6 - 1s
Message #7 - 1s
Message #8 - 1s
Message #9 - 1s
Message #10 - 1s
Because Message #0 requires 1 hour to process, it will occupy one
concurrent consumer thread for 1 hour. Nevertheless, we expect the
remaining 2 threads should be able to handle the remaining 9 messages
very
quickly. Instead, messages #3, #6, #9 are not processed until Message
#0
completes processing -- an hour later. Based on this observation, it
appears that messages are dispatched to queues owned by individual
threads,
in round-robin fashion, instead of having threads compete to consume
messages from a single queue. When some messages can take a long time
to
process, it can mean that messages can queue up and go unprocessed for a
very long time, even when there are free threads.
As you can see, this is not "competing consumers". Is the above
behavior
considered a bug? Are there any suggested workarounds?
Thanks!
Jim
