On Mon, Jan 26, 2009 at 11:34 PM, Hadrian Zbarcea <hzbar...@gmail.com> wrote: > I think having dots ('.') in property names is not a good idea as they don't > get along iirc with some technologies. >
I'm not worry about syntax right now. > Secondly, my point was that at this time, I am against the idea of > separating the headers into two maps. I don't see a compelling reason to do Wait. I am not a proponent of two maps at all. Here is my position: * one single header map in Camel message like it is today. * User non-protocol header as such as operationName for CXF should NOT be set in header map. It should be set in exchange properties. This is what Roman and Hadrian suggested. * I suggest to come up with some naming convention for properties. Something like org.apache.camel.<component> * I suggest to come up with some naming convention for component internal properties that can be easily removed or ignored. Some like tmp.org.apache.camel.<component> * I suggest user defined protocol headers should be included in header map not in exchange properties. (This is really the unsettled point.) > so and we should properly use properties as Roman and I think Claus > suggested. I'm confident that we'll find a good way to do it. OK. enough said. > > Hadrian > > > On Jan 26, 2009, at 11:09 PM, William Tam wrote: > >> On Mon, Jan 26, 2009 at 5:34 PM, Hadrian Zbarcea <hzbar...@gmail.com> >> wrote: >>> >>> On Jan 26, 2009, at 4:17 PM, William Tam wrote: >>> >>>> On Mon, Jan 26, 2009 at 1:37 PM, Hadrian Zbarcea <hzbar...@gmail.com> >>>> wrote: >>>>> >>>>> I don't disagree, I was just suggesting that they should then travel as >>>>> properties. >>>> >>>> Why shouldn't they (protocol headers) travel as headers (to avoids >>>> unnecessary copying between exchange properties and protocol headers)? >>> >>> There should be no unnecessary copying, I agree. If a header (such as >>> username/password) has broadly known and accepted semantics, the endpoint >>> or >>> associated policy could set it as a property from start. Similarly, an >>> endpoint should look at both properties and headers when sending a >>> message. >> >> I think I finally get it. Your example of username/password are >> really properties in my view. E.g, we can have a property >> org.apache.camel.headers.username. It is broadly known in Camel. >> Sure, we can stick it in exchange properties and let endpoint read it >> and create a protocol header with it (like, USER=foo). I am fine with >> that idea. That does not make org.apache.camel.headers.username=foo a >> protocol header, though. It really isn't. >> >> I am concerned about true custom protocol header that is totally >> unknown to Camel (say, foo=bar). I want my app to be able to define >> and insert some header in protocol message header. I argue that >> foo=bar should be set in the header not in exchange properties. >> >>> >>>> >>>> >>>>> Whatever we name them, and whatever mechanism we decide to use, >>>>> as pointed out before, we need to distinguish between headers that are >>>>> endpoint/protocol specific and have no semantics outside the endpoint >>>>> and >>>>> headers (which we called properties and didn't use consistently) that >>>>> must >>>>> be carried over the lifetime of the Exchange. >>>> >>>> Custom protocol header fits into "no semantics outside the endpoint" >>>> (or else it won't get propagated) and so it goes to the exchange >>>> properties. How does the next component (in the pipeline) know what >>>> one of the properties are intended to be sent in protocol header? >>>> Wouldn't it be better to let protocol headers travel as headers so >>>> only the "header candidates" will be potentially be sent? >>> >>> Not sure I understand your question, but I think we should use policies >>> for >>> handling custom headers. >> >> Sorry about the typos and grammars. All I am saying is if we put user >> defined header in message headers, filter strategy (or policy if you >> will) won't have to iterate through the exchange properties for it. >> It is already in headers where it belongs. All we need to do is to >> apply filters to headers. It is much cleaner. In your example, if >> endpoint wants to read a specific property (e.g. >> org.apache.camel.headers.username) and make a header out of a >> property, it can certainly do so. >> >> (However, I am not sure there are really that many Camel well known >> properties at least in the presence.) >> >> >>> >>>> >>>> >>>>> >>>>> Hadrian >>>>> >>>>> >>>>> On Jan 26, 2009, at 1:07 PM, William Tam wrote: >>>>> >>>>>> On Mon, Jan 26, 2009 at 10:35 AM, Hadrian Zbarcea <hzbar...@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> This headers business is a bit of a tricky one. I hit it last year >>>>>>> in >>>>>>> the >>>>>>> context of security. >>>>>>> >>>>>>> I agree with the view that headers should only exist in the context >>>>>>> of >>>>>>> an >>>>>>> endpoint. I think outside of that there is no guarantee that the >>>>>>> semantics >>>>>>> of a header is preserved. I am not sure if headers should be >>>>>>> propagated >>>>>>> from one endpoint to another at all. >>>>>> >>>>>> There are certainly use cases that protocol headers DO need to be >>>>>> propagated between endpoints. If users want to integrate with some >>>>>> management and/or security tools like Actional, users are required to >>>>>> include custom headers in protocol headers. These custom headers >>>>>> travel with messages to allow trust zone enforcement and message >>>>>> correlation. They need to be preserved and propagated across hops >>>>>> which are potentially over different transport protocols. >>>>>> >>>>>>> Properties should be used instead. >>>>>>> Coming back to security, if http is used for instance there are >>>>>>> several >>>>>>> ways >>>>>>> of handling that. If basic auth is used for instance one gets a >>>>>>> user/pass, >>>>>>> but that may need to be translated to something else at endpoint >>>>>>> boundaries. >>>>>>> I don't think that the "Authorization" header should exist outside >>>>>>> the >>>>>>> http >>>>>>> endpoint for instance. >>>>>>> >>>>>>> Yes, we do propagate properties today, no issue there. But then some >>>>>>> policies need to be defined per endpoint to deal with known >>>>>>> headers/properties, and camel specific properties should be defined >>>>>>> to >>>>>>> deal >>>>>>> with know headers. Even better, endpoints should set protocol >>>>>>> specific >>>>>>> headers that are known as required to propagate (such as the auth >>>>>>> stuff) >>>>>>> as >>>>>>> properties from start. >>>>>>> >>>>>>> My $0.02 >>>>>>> Hadrian >>>>>>> >>>>>>> >>>>>>> On Jan 26, 2009, at 9:44 AM, Claus Ibsen wrote: >>>>>>> >>>>>>>> On Mon, Jan 26, 2009 at 3:37 PM, Roman Kalukiewicz >>>>>>>> <roman.kalukiew...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> Why don't we talk about exchange properties here? My feeling here >>>>>>>>> is >>>>>>>>> that properties should be used as user-headers, while headers are >>>>>>>>> always protocol headers. In fact it works this way right now: If I >>>>>>>>> want to keep some value through the whole flow I put it into >>>>>>>>> properties. >>>>>>>>> >>>>>>>>> By current convention if I put something on a header it is sent as >>>>>>>>> protocol-specific header (JMS property, HTTP header), and out >>>>>>>>> headers >>>>>>>>> are filled also with protocol headers (JSM properties of out >>>>>>>>> emssage, >>>>>>>>> HTTP response headers). In this case headers shouldn't be >>>>>>>>> propagated, >>>>>>>>> as there is no way to distinguish things propagated, from things >>>>>>>>> retrieved. And out headers ARE different than in headers. >>>>>>>>> >>>>>>>>> It is a matter of naming, but currently headers are (what you call) >>>>>>>>> protocol/system headers, while properties are user-headers (work as >>>>>>>>> variables). Do we really need to extend it further? If someone mix >>>>>>>>> those two concepts then it is problem of documentation, but not >>>>>>>>> lack >>>>>>>>> of functionality. I would just extend DSL a little to be able to >>>>>>>>> retrieve a property (instead using header()). >>>>>>>>> >>>>>>>>> What do you think, guys? Maybe we should clearly communicate what >>>>>>>>> things are for and what are the consequences of using one or >>>>>>>>> another. >>>>>>>>> >>>>>>>>> Roman >>>>>>>>> >>>>>>>>> PS. Pipeline should propagate all headers of course, but I believe >>>>>>>>> an >>>>>>>>> endpoint is a place where we shouldn't guarantee that headers will >>>>>>>>> be >>>>>>>>> propagated by stating it clearly. >>>>>>>> >>>>>>>> Properties have just lived in the dark and end users does not really >>>>>>>> know they exists. We have some builder methods to set/get >>>>>>>> properties. >>>>>>>> I guess we need to document and maybe make sure the Spring DSL also >>>>>>>> has support for accessing properties as well. >>>>>>>> >>>>>>>> To my knowledge properties is always preserved so I doubt we have an >>>>>>>> issue there. >>>>>>>> >>>>>>>> So users should just start learn using properties as well :) >>>>>>>> However then we have the ProducerTemplate that has one liners for >>>>>>>> sending an Exchange. We dont have a sendBodyAndProperties method. >>>>>>>> But >>>>>>>> yet again it has too many methods already. They can just use >>>>>>>> send("xxx", Exchange) and have exchange populated with the >>>>>>>> properties >>>>>>>> of choice. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> 2009/1/26 Claus Ibsen <claus.ib...@gmail.com>: >>>>>>>>>> >>>>>>>>>> On Sat, Jan 24, 2009 at 9:08 PM, William Tam >>>>>>>>>> <email.w...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> What we have stored in Headers today in Camel is both: >>>>>>>>>>>> - user headers >>>>>>>>>>>> - and system headers (added by Camel itself). >>>>>>>>>>>> >>>>>>>>>>>> I am starting to be more and more convinced that we should >>>>>>>>>>>> separate >>>>>>>>>>>> the two. >>>>>>>>>>>> So any headers that a users has enforced to be set should be >>>>>>>>>>>> kept >>>>>>>>>>>> in >>>>>>>>>>>> one Map and the others that the components set internally (such >>>>>>>>>>>> as >>>>>>>>>>>> SQL >>>>>>>>>>>> number of rows returned, or whatnot we have, there are many) in >>>>>>>>>>>> another Map. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> It means that a component would have to look for header in more >>>>>>>>>>> than >>>>>>>>>>> one place. Besides, the distinction of user vs system header is >>>>>>>>>>> not >>>>>>>>>>> always clear. For example, the operation name header for cxf >>>>>>>>>>> endpoint >>>>>>>>>>> can be set by user but it is also created by cxf component. I >>>>>>>>>>> am >>>>>>>>>>> sure there are many more examples. There is another header >>>>>>>>>>> category: >>>>>>>>>>> protocol headers. A protocol header is not really a user or >>>>>>>>>>> system >>>>>>>>>>> header. Protocol headers are header propagated from protocol >>>>>>>>>>> like >>>>>>>>>>> HTTP, which we do want to preserve in message header. >>>>>>>>>>> >>>>>>>>>>>> The user headers is always preserved and copied along in the >>>>>>>>>>>> routing. >>>>>>>>>>>> User can always clear/remove unwanted headers. >>>>>>>>>>>> The system headers should be short lived as they are not really >>>>>>>>>>>> useable. So they are "alive" in the next step (process) in the >>>>>>>>>>>> route, >>>>>>>>>>>> and when the pipeline invokes next route thereafter these >>>>>>>>>>>> information >>>>>>>>>>>> is cleared. >>>>>>>>>>>> >>>>>>>>>>>> Separating these will also make the routing/tracing a bit easier >>>>>>>>>>>> as >>>>>>>>>>>> Users can recognize their own headers instead its mixed with all >>>>>>>>>>>> the >>>>>>>>>>>> noise the Camel components add. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I wonder we can leverage/extend the HeaderFilterStrategy >>>>>>>>>>> mechanism. >>>>>>>>>>> Currently, it is only used for filtering unwanted headers (in >>>>>>>>>>> both >>>>>>>>>>> request and response direction) when we propagate headers between >>>>>>>>>>> Camel and external messages (like HTTP). HeaderFilterStrategy >>>>>>>>>>> is >>>>>>>>>>> (or >>>>>>>>>>> will be) associated with an endpoint. We could make >>>>>>>>>>> HeaderFilterStrategy available to the exchange object. So, when >>>>>>>>>>> an >>>>>>>>>>> endpoint creates an exchange, the exchange gets a header filter >>>>>>>>>>> strategy. Then, pipeline can do something like this to filter >>>>>>>>>>> unwanted header: message.filterHeaders(). The header filter >>>>>>>>>>> strategy >>>>>>>>>>> is highly customizable for each endpoint (can have a component >>>>>>>>>>> wide >>>>>>>>>>> default) and it can be looked up from registry. >>>>>>>>>>> >>>>>>>>>> Good pointers William. >>>>>>>>>> >>>>>>>>>> Yeah we can revist it after you have moved the header filters to >>>>>>>>>> the >>>>>>>>>> endpoint. >>>>>>>>>> >>>>>>>>>> Then we can check up upon how to leverage it as you suggest. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Claus Ibsen >>>>>>>>>> Apache Camel Committer >>>>>>>>>> >>>>>>>>>> Open Source Integration: http://fusesource.com >>>>>>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Claus Ibsen >>>>>>>> Apache Camel Committer >>>>>>>> >>>>>>>> Open Source Integration: http://fusesource.com >>>>>>>> Blog: http://davsclaus.blogspot.com/ >>>>>>> >>>>>>> >>>>> >>>>> >>> >>> > >