We are using Camel 2.9.0. We are using the "concurrentConsumers=n"
option on our jms consumer endpoint so that we can handle numerous InOut
requests concurrently. In Camel documentation, this option is described
as supporting competing consumers, which I interpret to mean a pool of
threads which are _competing_ to handle incoming requests. I expect
that whenever a request comes in, that request will be processed by some
idle thread from the thread pool (if there are any). However, this is
not that behavior we are observing. Instead, I observe some requests
not being processed immediately (or for quite some time), despite having
numerous idle consumer threads, and this is causing a severe liveness
issue for our application.
Let me describe a simple scenario. Assume we have a single InOut jms
queue with concurrentConsumers=3, and the following stream of messages
(and required processing time):
Message #0 - 1h
Message #1 - 1s
Message #2 - 1s
Message #3 - 1s
Message #4 - 1s
Message #5 - 1s
Message #6 - 1s
Message #7 - 1s
Message #8 - 1s
Message #9 - 1s
Message #10 - 1s
Because Message #0 requires 1 hour to process, it will occupy one
concurrent consumer thread for 1 hour. Nevertheless, we expect the
remaining 2 threads should be able to handle the remaining 9 messages
very quickly. Instead, messages #3, #6, #9 are not processed until
Message #0 completes processing -- an hour later. Based on this
observation, it appears that messages are dispatched to queues owned by
individual threads, in round-robin fashion, instead of having threads
compete to consume messages from a single queue. When some messages can
take a long time to process, it can mean that messages can queue up and
go unprocessed for a very long time, even when there are free threads.
As you can see, this is not "competing consumers". Is the above
behavior considered a bug? Are there any suggested workarounds?
Thanks!
Jim
