I'm not yet convinced we should remove something, without another thing to put in its place. e.g. why don't we figure out a cleaner model and just switch to it (say at 2.1).
For sure not that many things use async dispatch and it does cause confusion debugging; but there's definitely been folks with HTTP / WS / JBI async requirements over the last couple of years and AsyncProcessor is our only solution so far; so I'd rather leave it there - even if its sub optimal - until it can be replaced by a simple thing. Since its a purely internal API, I don't see that we have to definitely fix this for 2.0 2009/4/24 Claus Ibsen <claus.ib...@gmail.com>: > On Fri, Apr 24, 2009 at 10:17 AM, Willem Jiang <willem.ji...@gmail.com> wrote: >> >> +1 for introducing Channel into Camel world. >> >> For the AsyncProcessor part, it's is really difficult to understand. >> I think it is more than Call back, it will not block the calling thread >> which will be used in the camel-jhc component[1]. >> And I also did an enhancement[2] on the ErrorHandler of >> DeadLetterChannel by leveraging the AsyncProcessor to avoid the blocking >> the calling thread when the DeadLetterChannel waits for another retry. >> >> If the UnitOfWork can address these, I'm happy to give my +1 for >> removing the AsyncProcessor. If not, we need to find out a replacement >> before removing it. >> >> [1] http://cwiki.apache.org/CAMEL/asynchronous-processing.html >> [2] https://issues.apache.org/activemq/browse/CAMEL-1129 > > Yeah the AsyncProcessor and the callback notion currently implemented > in Camel is flawed, brittle, broken, not used at all from end user > perspective > and it started to spread out into the code base where it should not have. > > I really think it should be removed, and if we need it, a new Async > API introduced that resemble the JDK async support (for instance the > Future). > > To fix it I really think we need to > 1) Remove the existing AsyncProcessor > 2) At a later stage add a new API that leverages the JDK API. > > > About redelivery using different threads > ============================= > Note the using another thread for redeliver is only feasable for non > transacted routes. > Transacted routes tend to depend on ThreadLocal and reuse session and > other stuff. > > So the redeliver using different thread is not always desirable. > Note: But the redelivery is only supported by DeadLetterChannel that > does not support transacted routes anyway. > So you can say we are safe here. > But the end user does not have any choice to configure what they want. > > I do think that end users should have the choice which model they > would like Camel to use for redelivery handling. > > And with the Channel we would have another possiblity for redelivery > as we could just "route the message" back to the previous channel > and let it have some "delay time" before its "visible" on the channel > queue for re-processing. > > So Willam, it would be much easier to impl. the use different thread > for redelivery when the Channel is more enhanced in Camel. > For instance for InOnly routes where there are no caller waiting we > can safely use others threads for redeliver (if not transacted) > But for InOut we need to add our own barrier that waits until the > UnitOfWork is complete before we can return the response. > > Again I cannot stress too much that the camel-core API needs this > cleanup. You will get to this conclusion if you have been working 8h > around the clock in the camel-core code for as long as I have. > > > >> >> >> Willem >> >> >> Claus Ibsen wrote: >>> Hi Camel riders >>> >>> As you know we are working on Camel 2.0 and we decided to do a 2nd >>> milestone release. >>> This gives us more time to start on some of the ideas and internal >>> refactorings I wanted >>> to do for Camel 2.1. >>> >>> After having dived really deed in the camel core codebase for the last >>> 6+ months or longer, I >>> feel that we need to cease the moment and do a more extensive house >>> cleaning before the Camel 2.0 >>> release. The house cleaning is only internal and will not affect end >>> users of Camel. >>> >>> Here are 2 issues I would like to address in the foreseeable future. >>> >>> >>> 1) Channel >>> ========== >>> To introduce a Channel between each node in the route path. The >>> channel is a composite processor >>> that is responsible for routing the exchange to the next node in the path. >>> >>> We already do this but there is no visible notion of a Channel. The >>> "Channel" exists today as >>> a series of interceptors and an error handler. Today this is wrapped >>> at route build time >>> and then we have a static list of processors the exchanges is routed >>> through a runtime. >>> >>> So what I am working on is to composite this series of interceptors >>> and error handler into a >>> DefaultChannel class that has the logic to do needed work before an >>> Exchange is routed to the next >>> node. >>> >>> I have already discussed this a bit with Gert and James. James agreed >>> that it a was time for introducing >>> the Channel into Camel. >>> >>> So I have started experimenting with this and got it working locally. >>> The Channel is in this first >>> stage just wrapping the existing series of interceptors and error >>> handler and thus the route logic >>> is not affect. >>> >>> In the future the Channel will benefit in many ways as we can at >>> runtime add routing logic for instance >>> - enabling/disabling tracing (and other interceptors) at runtime >>> without reloading routes >>> - adding or removing interceptors at runtime without reloading routes >>> - adding or removing JMX performance metrics at runtime without reloading >>> routes >>> - we could use it to blocking routing >>> - enhanced tooling support >>> - potential you could persist exchanges at the channel >>> - later add true async processing with the help of the channel >>> - easier to traverse the runtime route graph as we can traverse the >>> Channel, where we can have next/prev or the like methods >>> - and much more we can imagine >>> >>> >>> >>> 2) AsyncProcessor >>> ================= >>> I propose to remove the AsyncProcessor all together. I had a chat with >>> James about it and it was an experiment by Hiram back in early 2007. >>> Basically he had not worked on the code since, and its not really put >>> into good use. But sadly over time the AsyncProcessor have spread >>> itself into other core parts of Camel. >>> >>> The basic idea was to attach a callback to a route so you could do >>> some "commit" work after the exchange is finished. >>> This idea is actually superseded by the UnitOfWork (introduced by >>> James) that is better for this kind of work. And we have schduled an >>> overhaul for UnitOfWork >>> in Camel 2.1 to allow more of our components to take advantage of this >>> and also expose DSLs for end users to attach their custom processors >>> or route. >>> >>> The code in Camel core that is affected by the AsyncProcessor is much >>> more complex than regular Processor. In fact there are some areas >>> where its not >>> used correctly and causes unforseen side effects that only surfaces in >>> some complex or rare unit tests. This issues is more apparent lately >>> as more >>> and more camel processors supports the async processor directly. >>> >>> The original code by Hiram only leverages the async callback in the >>> file component as it was part of his initial code. No other areas >>> benefits from this. >>> The code is basically making it bloat and complex inside Camel itself. >>> >>> End users do not use this. I do not recall a single question on it in >>> the user forum. >>> >>> So I am in a big +1 to get it out of the codebase. >>> >>> And before you say well how do we do async routing then? Well the seda >>> component is still there to truely spawn a new thread to route an >>> exchange. >>> And we have the UnitOfWork where you are supposed to register your >>> callback. We might make the UnitOfWork a bit easier to register >>> callbacks for >>> only commit or failure depending on what you want. If its a single >>> methods its also easier for dynamic languages to have nice DSL support >>> for it. >>> Well that is just me rambling now..... And the UnitOfWork was James >>> envision for doing async work after the exchange is ended. >>> >>> >>> Summary >>> ======= >>> >>> 1) Channel >>> The first cut of this is already done on my local laptop and it will >>> just work as is today. But its bases the foundation for more exotic >>> and advanced >>> stuff in Camel 2.x series. >>> >>> Already in progress. >>> >>> Any thoughts? >>> >>> >>> 2) AsyncProcessor >>> To reduce the complexity for maintaining the Camel codebase. Just as >>> Gert if he was a bit puzzled how to do his StreamCache that uses the >>> Async stuff. >>> When we get it out of there the codebase is much easier to maintain >>> and we have a greater chance to attract potential new committers. >>> >>> I vote +1 for this. >>> Any thoughts? >>> >>> >>> >>> >> >> > > > > -- > Claus Ibsen > Apache Camel Committer > > Open Source Integration: http://fusesource.com > Blog: http://davsclaus.blogspot.com/ > Twitter: http://twitter.com/davsclaus > Apache Camel Reference Card: > http://refcardz.dzone.com/refcardz/enterprise-integration > -- James ------- http://macstrac.blogspot.com/ Open Source Integration http://fusesource.com/
