Camel riders,
I'm seeing an issue caused by the MulticastProcessor not handling async
routing end-to-end. It does use its own threadpool to process its routing
tasks, as well as to perform the aggregation (on-the-fly).
However, during all that time, the calling thread is blocked waiting for
the multicast to finish, even if the previous processors passed along an
AsyncCallback. This behaviour basically nullifies the async routing engine.
* How tough do you think it'll be to make the Multicast processor
async-aware end-to-end? My idea is to implement a technique similar to the
TemporaryReplyQueueManager in camel-jms, where we do bookkeeping in an
in-memory data structure, and trigger the AsyncCallback when we've
collected all aggregation steps, or if a timeout triggers.
* Do you think it's worthwhile to dedicate a full thread per request to
process its aggregations? In I/O bound cases, such as waiting for a WS
reply, file download, etc. it feels wasteful to keep a thread parked for
aggregations.
* What do you think about calling the aggregation strategy from the
thread(s) where the response(s) arrive(s)? (which could be from the
multicast threadpool or a component thread e.g. TempReplyQueueManager in
the case of JMS, depending on whether the component is sync or async).
In essence, I'm considering a multicast v2 implementation (which would also
affect RecipientList, Splitter, etc.) for 2.18, allowing the user to choose
the implementation via DSL, e.g.
.multicast().v2()
It it's more efficient and feedback from users is good, we could make it
the default multicast processor in Camel 3.0.
Thoughts?
Cheers,
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk>
