The MEP is InOut ("request/reply"). There are two participants, a
consumer and a producer. We need concurrent handling on the consumer
(receiver, request processor) side. From my reading of the
documentation, the "concurrentConsumers" option should provide
concurrency for the consumer side, and the "threads()" dsl should
provide concurrency on the producer side.
Although my use case is InOut (not InOnly), I agree with your statement
"concurrentConsumers should do it". It's mostly doing it, but there is
a critical liveness issue as I have described.
Regarding the route config, see my other email from a few hours ago. It
has full Java code for running a server and client demo of the problem,
including initialization of routes.
Thanks,
Jim
On 2/6/2012 1:58 PM, David Karlsen wrote:
In the original mail you stated "option on our jms consumer endpoint
so that we can handle numerous InOut requests" which I read as
Producer with InOut MEP - and from the docs I read that should use the
threads options.
If you only have incoming (e.g. InOnly) then I read the docs as
concurrentConsumers should do it.
Maybe you could show the route config?
2012/2/6 Jim Newsham<jnews...@referentia.com>:
Thanks David for the suggestion, but the documentation says that this option
is for concurrently handling replies. I am looking to concurrently handle
requests. Regardless of the documentation, I just gave this a try and it
does not fix the issue.
Thanks again,
Jim
On 2/6/2012 10:01 AM, David Karlsen wrote:
Read the section on "Request-reply over JMS" - you're looking for the
threads option.
2012/2/6 Jim Newsham<jnews...@referentia.com>:
Hi Ngoc Hai,
I'm not talking at all about the producer side where replies are
processed,
but the consumer side where requests are being received and processed.
Incoming requests are not being dispatched to idle threads; instead,
they
are backing up behind a single (non-idle) thread.
To me, this looks like a bug. I am interested in hearing the opinions of
the community on this point, and I'm willing to file a JIRA issue.
Thanks,
Jim
On 2/3/2012 6:32 PM, Pham Ngoc Hai wrote:
Hi Jim,
Since it's InOut queue, the reply is processed by a single thread as
stated in here: http://camel.apache.org/jms.html no matter how many
concurrentConsumer you set.
Regards,
Ngoc Hai.
________________________________
From: Jim Newsham<jnews...@referentia.com>
To: users@camel.apache.org
Sent: Saturday, February 4, 2012 10:55 AM
Subject: jms concurrentConsumers is not competing consumers
We are using Camel 2.9.0. We are using the "concurrentConsumers=n"
option
on our jms consumer endpoint so that we can handle numerous InOut
requests
concurrently. In Camel documentation, this option is described as
supporting competing consumers, which I interpret to mean a pool of
threads
which are _competing_ to handle incoming requests. I expect that
whenever a
request comes in, that request will be processed by some idle thread
from
the thread pool (if there are any). However, this is not that behavior
we
are observing. Instead, I observe some requests not being processed
immediately (or for quite some time), despite having numerous idle
consumer
threads, and this is causing a severe liveness issue for our
application.
Let me describe a simple scenario. Assume we have a single InOut jms
queue with concurrentConsumers=3, and the following stream of messages
(and
required processing time):
Message #0 - 1h
Message #1 - 1s
Message #2 - 1s
Message #3 - 1s
Message #4 - 1s
Message #5 - 1s
Message #6 - 1s
Message #7 - 1s
Message #8 - 1s
Message #9 - 1s
Message #10 - 1s
Because Message #0 requires 1 hour to process, it will occupy one
concurrent consumer thread for 1 hour. Nevertheless, we expect the
remaining 2 threads should be able to handle the remaining 9 messages
very
quickly. Instead, messages #3, #6, #9 are not processed until Message
#0
completes processing -- an hour later. Based on this observation, it
appears that messages are dispatched to queues owned by individual
threads,
in round-robin fashion, instead of having threads compete to consume
messages from a single queue. When some messages can take a long time
to
process, it can mean that messages can queue up and go unprocessed for a
very long time, even when there are free threads.
As you can see, this is not "competing consumers". Is the above
behavior
considered a bug? Are there any suggested workarounds?
Thanks!
Jim
