Handling complex, multi-record, single-line pipe-separated text?
First up, I'm currently building a proxy, effectively, for some interfaces that I have to work with. On one side is a set of web services that my application will be calling, so that I can standardise on *something*. On the other side is a set of IBM MQ queues that - mostly - require fixed-length records. So what I am doing is sending a SOAP message to ServiceMix, transforming that into a POJO, then transforming that POJO into a flat file via BeanIO, which in turn gets sent out to MQ. That might seem a little inefficient, but it beats writing thousands of lines of XSL to transform the XML into a flat file. One format in particular, however, doesn't seem to be achievable with any of the data formats available in Camel, as far as I can tell, but I would appreciate some advice on this front. Bindy cannot - it isn't nearly comprehensive enough. BeanIO *almost* can, if only the records were on separate lines. But they're not - they're all on the same line. Neither Flatpack nor Smooks seem to handle this format either, from what I've read. The basic format is like a CSV file except with pipes " | " instead of commas. However, despite being only a single line, there are multiple records in that line, and several of the records have a variable number of repetitions. Now, given that I only need to *generate* this format, not parse it, I could probably generate it with BeanIO and do some sort of post-processing on it to strip out the newlines or something similar (the response is significantly simpler and contains no repeating records - parsing that is not a problem). However, I would like to know if there is anything out there which would support this format properly, should I find it necessary to parse it in the future, and to avoid hacking in something now which I will later regret. For example, if we take the following sample (trimmed down significantly for brevity): COM|US|CORP|FIELD1|DATE1|TIME1|DATE2|DATE3|TYPE|1|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|S|1|STUFF||CODE||ESTATUS||G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|I|...|R|...|R|...|R|...|Y|...|Y|...|Y|... And break that down, the first set of columns, COM|US|CORP|FIELD1|DATE1|TIME1|DATE2|DATE3|TYPE|1, is effectively a header. This appears exactly once and is not an issue. The next set of columns, A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5, represents a single record that can appear 1 to 99 times in the line. This where I start seeing problems. In my POJO, I would like to represent this as a list of "A" records, and then have the data format generate one record for each list item, but without adding a line-break afterwards. If I were to parse this, it would need to know that after the first record, the "COM" record, it should look at the first character to see what the type of the record is - in this case it is an "A" record, and there must be at least one record, and there may be up to 99 records. In the example, I have repeated it three times. Note that while BeanIO could, in theory, handle this as a repeating segment, I have other repeating segments following this one. The next set of columns, S|1|STUFF||CODE||ESTATUS||, is repeated exactly once, and the type of record is "S", identified by the first character. The next set, G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3, is similar to the "A" record, but can appear 0 to 99 times. I have included it three times in this example. The next set, I|..., is similar to "S" in that it only appears once. The next set, R|..., can appear 0 to 99 times. The next set, Y|..., can appear 0 to 99 times. There are other repeating segments too - that's just a small part of the whole record. I hope this makes sense - it seems like this is a particularly unusual record format to have to deal with, so it is perhaps unsurprising that I can't find a tool that will handle it.
Re: Handling complex, multi-record, single-line pipe-separated text?
I have, though it appears they don't support it out of the box. Unfortunate, but unsurprising. Thanks, - Andrew On Thu, Nov 14, 2013 at 11:28 PM, Claus Ibsen wrote: > Hi > > Have you been in touch with beanio project? Maybe there is something > they would like to see support out of the box in beanio. > > Otherwise you may need to write your own code to format the date accordingly. > > > > On Thu, Nov 14, 2013 at 3:05 AM, Andrew Thorburn wrote: >> First up, I'm currently building a proxy, effectively, for some >> interfaces that I have to work with. On one side is a set of web >> services that my application will be calling, so that I can >> standardise on *something*. On the other side is a set of IBM MQ >> queues that - mostly - require fixed-length records. So what I am >> doing is sending a SOAP message to ServiceMix, transforming that into >> a POJO, then transforming that POJO into a flat file via BeanIO, which >> in turn gets sent out to MQ. That might seem a little inefficient, but >> it beats writing thousands of lines of XSL to transform the XML into a >> flat file. >> >> One format in particular, however, doesn't seem to be achievable with >> any of the data formats available in Camel, as far as I can tell, but >> I would appreciate some advice on this front. Bindy cannot - it isn't >> nearly comprehensive enough. BeanIO *almost* can, if only the records >> were on separate lines. But they're not - they're all on the same >> line. Neither Flatpack nor Smooks seem to handle this format either, >> from what I've read. >> >> The basic format is like a CSV file except with pipes " | " instead of >> commas. However, despite being only a single line, there are multiple >> records in that line, and several of the records have a variable >> number of repetitions. >> >> Now, given that I only need to *generate* this format, not parse it, I >> could probably generate it with BeanIO and do some sort of >> post-processing on it to strip out the newlines or something similar >> (the response is significantly simpler and contains no repeating >> records - parsing that is not a problem). However, I would like to >> know if there is anything out there which would support this format >> properly, should I find it necessary to parse it in the future, and to >> avoid hacking in something now which I will later regret. >> >> For example, if we take the following sample (trimmed down >> significantly for brevity): >> >> COM|US|CORP|FIELD1|DATE1|TIME1|DATE2|DATE3|TYPE|1|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5|S|1|STUFF||CODE||ESTATUS||G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|I|...|R|...|R|...|R|...|Y|...|Y|...|Y|... >> >> And break that down, the first set of columns, >> COM|US|CORP|FIELD1|DATE1|TIME1|DATE2|DATE3|TYPE|1, is effectively a >> header. This appears exactly once and is not an issue. >> >> The next set of columns, >> A|ABC123|DEF456||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3|ADDRESS4|ADDRESS5, >> represents a single record that can appear 1 to 99 times in the line. >> This where I start seeing problems. In my POJO, I would like to >> represent this as a list of "A" records, and then have the data format >> generate one record for each list item, but without adding a >> line-break afterwards. If I were to parse this, it would need to know >> that after the first record, the "COM" record, it should look at the >> first character to see what the type of the record is - in this case >> it is an "A" record, and there must be at least one record, and there >> may be up to 99 records. In the example, I have repeated it three >> times. Note that while BeanIO could, in theory, handle this as a >> repeating segment, I have other repeating segments following this one. >> >> The next set of columns, S|1|STUFF||CODE||ESTATUS||, is repeated >> exactly once, and the type of record is "S", identified by the first >> character. >> >> The next set, >> G|ID|||SURNAME|FIRSTNAME|MIDDLENAME|GENDER|DOB|ADDRESS1|ADDRESS2|ADDRESS3, >> is similar to the "A" record,
Access Camel Context or OSGi Context from XSL Extension method?
I am currently in the process of building a "simple" proof of concept
using Camel and ServiceMix, and mostly it's been going fine - it so
far has done exactly what I want, if not necessarily always how I
want.
However, I'm coming up a bit short on how to achieve what I want right now.
The scenario is fairly simple:
SMX is acting as a layer between my Application, A, and a remote
application, R. To try and centralise a bunch of the logic, we are
planning to have A transform a (non-trivial) graph of domain objects
to XML and send them over to SMX. SMX will then take this set of XML,
transform it into some different XML via XSL, transform that into a
POJO, then transform the POJO into the flat file required by R.
The problem I'm running into is that I need to combine some of the
data in non-trivial ways. e.g. The domain objects I'm sending have
address data like unitNo, streetNo, streetName, streetType, suburb,
country, etc. Whereas R requires address line 1, 2, 3, 4, etc. So I
need to turn our separate address fields into combined fields, based
on various rules around shortening them if they're too long, etc.
On top of that, some values also need to be mapped, as the value of
the field in the domain object may be different to the value that R
expects (e.g. A may have a street type of ST, but R may expect
STREET). This occurs in the addresses, and also in other fields as
well.
This one ESB will handle multiple different interfaces on R, and I
imagine that while most will share the same requirements as far as
mapping and address formatting goes, I worry that some might not, so I
would like to come up with a solution that makes it easy to have one
generic implementation that can be overridden on a case-by-case basis,
or be able to be exposed as a web service, etc.
It's possible I'm overthinking this, but I'm not happy with my current solution:
Camel Route is Web Service -> XSL -> XML -> ..., where the XSL calls
out to a static Java method for transforming the address and
translating codes. Note that there could be 5 - 10 different addresses
in a single request, and 30+ elements that need a mapping, out of a
much larger set of elements.
My problem is that, while this seems fairly simple, it means I lose
access to everything Camel-, Spring- and OSGi-related. Mostly, it
means I'd have strong coupling between all my dependencies, no way to
access any properties apart from System properties, no way to call
Camel routes, etc, etc, which makes me very nervous, and just doesn't
*feel* right (I know, I know - that's a terribly subjective
qualification!).
Does anyone see a better solution, or have any advice about things I
should have a look at which could help? Can I, for example, gain
access to the current CamelContext so I can call out to a camel route
for address formatting or other such things?
Or am I really just stuck going with this:
public static NodeList formatAddress(NodeList address) { String unitNo
= address.item(0).getChildNodes().item(0).getTextContent(); ... }
public static String translate(String type, String val) { InputStream
is = XSLUtil.class.getResourceAsStream(...); ... }
Thanks,
- Andrew Thorburn
Re: Access Camel Context or OSGi Context from XSL Extension method?
I also found an infinite loop if a component loads an XSL stylesheet which has an invalid XPath function (e.g. - there should be a comma between $a and b: "($a, 'b')"), which didn't seem to stop when I shutdown the route (I don't recall seeing anything in the log suggesting it was being shut down, but there were a lot of errors scrolling by very fast). I had to head into the cache and nuke the bundle after force-quitting SMX to get it to work properly, which was quite awkward. I will raise a full bug report for that when I can. - Andrew On Fri, Nov 22, 2013 at 7:31 PM, Andrew Thorburn wrote: > Possibly, though I may have to go back to A for some strange cases, which > would rule that out. I also wasn't aware that you could override XSL > templates / imports / etc, so that does make a case for doing it in pure > XSL. That said, how do I reference a template that's in another OSGi bundle? > Do I just add the requisite Import-Package statement for the package the > resource is in? > > The other possibility, which *almost* worked, was creating a bean in Camel, > then passing that in as an XSL parameter and calling instance methods on it, > which would have given me full access to Camel/OSGi/etc, which would've been > perfect. Unfortunately, Xalan isn't OSGi friendly, and since it wants to > lookup the Class I'm calling methods on (frankly, it doesn't need to), it > explodes because it can't find it. I can probably work around that, but it's > less than ideal. > > So yeah, either injecting a bean into my template (with some effort to hack > around class loader limitations), or see if I can do it all in XSL. > > Thanks, > > - Andrew > > On 21/11/2013 9:58 PM, "Walzer, Thomas" > wrote: >> >> Not sure if I got your requirements right. >> >> How about going a plain XSL way (no java mapping)? Doing everything in XSL >> and using a generic template for address mapping. You could import it as >> needed and overide as needed. >> >> Cheers, Thomas. >> >> Am 21.11.2013 um 03:13 schrieb Andrew Thorburn : >> >> > I am currently in the process of building a "simple" proof of concept >> > using Camel and ServiceMix, and mostly it's been going fine - it so >> > far has done exactly what I want, if not necessarily always how I >> > want. >> > >> > However, I'm coming up a bit short on how to achieve what I want right >> > now. >> > >> > The scenario is fairly simple: >> > >> > SMX is acting as a layer between my Application, A, and a remote >> > application, R. To try and centralise a bunch of the logic, we are >> > planning to have A transform a (non-trivial) graph of domain objects >> > to XML and send them over to SMX. SMX will then take this set of XML, >> > transform it into some different XML via XSL, transform that into a >> > POJO, then transform the POJO into the flat file required by R. >> > >> > The problem I'm running into is that I need to combine some of the >> > data in non-trivial ways. e.g. The domain objects I'm sending have >> > address data like unitNo, streetNo, streetName, streetType, suburb, >> > country, etc. Whereas R requires address line 1, 2, 3, 4, etc. So I >> > need to turn our separate address fields into combined fields, based >> > on various rules around shortening them if they're too long, etc. >> > >> > On top of that, some values also need to be mapped, as the value of >> > the field in the domain object may be different to the value that R >> > expects (e.g. A may have a street type of ST, but R may expect >> > STREET). This occurs in the addresses, and also in other fields as >> > well. >> > >> > This one ESB will handle multiple different interfaces on R, and I >> > imagine that while most will share the same requirements as far as >> > mapping and address formatting goes, I worry that some might not, so I >> > would like to come up with a solution that makes it easy to have one >> > generic implementation that can be overridden on a case-by-case basis, >> > or be able to be exposed as a web service, etc. >> > >> > It's possible I'm overthinking this, but I'm not happy with my current >> > solution: >> > >> > Camel Route is Web Service -> XSL -> XML -> ..., where the XSL calls >> > out to a static Java method for transforming the address and >> > translating codes. Note that there could be 5 - 10 different addresses >> >
Re: Access Camel Context or OSGi Context from XSL Extension method?
Possibly, though I may have to go back to A for some strange cases, which
would rule that out. I also wasn't aware that you could override XSL
templates / imports / etc, so that does make a case for doing it in pure
XSL. That said, how do I reference a template that's in another OSGi
bundle? Do I just add the requisite Import-Package statement for the
package the resource is in?
The other possibility, which *almost* worked, was creating a bean in Camel,
then passing that in as an XSL parameter and calling instance methods on
it, which would have given me full access to Camel/OSGi/etc, which would've
been perfect. Unfortunately, Xalan isn't OSGi friendly, and since it wants
to lookup the Class I'm calling methods on (frankly, it doesn't need to),
it explodes because it can't find it. I can probably work around that, but
it's less than ideal.
So yeah, either injecting a bean into my template (with some effort to hack
around class loader limitations), or see if I can do it all in XSL.
Thanks,
- Andrew
On 21/11/2013 9:58 PM, "Walzer, Thomas"
wrote:
> Not sure if I got your requirements right.
>
> How about going a plain XSL way (no java mapping)? Doing everything in XSL
> and using a generic template for address mapping. You could import it as
> needed and overide as needed.
>
> Cheers, Thomas.
>
> Am 21.11.2013 um 03:13 schrieb Andrew Thorburn :
>
> > I am currently in the process of building a "simple" proof of concept
> > using Camel and ServiceMix, and mostly it's been going fine - it so
> > far has done exactly what I want, if not necessarily always how I
> > want.
> >
> > However, I'm coming up a bit short on how to achieve what I want right
> now.
> >
> > The scenario is fairly simple:
> >
> > SMX is acting as a layer between my Application, A, and a remote
> > application, R. To try and centralise a bunch of the logic, we are
> > planning to have A transform a (non-trivial) graph of domain objects
> > to XML and send them over to SMX. SMX will then take this set of XML,
> > transform it into some different XML via XSL, transform that into a
> > POJO, then transform the POJO into the flat file required by R.
> >
> > The problem I'm running into is that I need to combine some of the
> > data in non-trivial ways. e.g. The domain objects I'm sending have
> > address data like unitNo, streetNo, streetName, streetType, suburb,
> > country, etc. Whereas R requires address line 1, 2, 3, 4, etc. So I
> > need to turn our separate address fields into combined fields, based
> > on various rules around shortening them if they're too long, etc.
> >
> > On top of that, some values also need to be mapped, as the value of
> > the field in the domain object may be different to the value that R
> > expects (e.g. A may have a street type of ST, but R may expect
> > STREET). This occurs in the addresses, and also in other fields as
> > well.
> >
> > This one ESB will handle multiple different interfaces on R, and I
> > imagine that while most will share the same requirements as far as
> > mapping and address formatting goes, I worry that some might not, so I
> > would like to come up with a solution that makes it easy to have one
> > generic implementation that can be overridden on a case-by-case basis,
> > or be able to be exposed as a web service, etc.
> >
> > It's possible I'm overthinking this, but I'm not happy with my current
> solution:
> >
> > Camel Route is Web Service -> XSL -> XML -> ..., where the XSL calls
> > out to a static Java method for transforming the address and
> > translating codes. Note that there could be 5 - 10 different addresses
> > in a single request, and 30+ elements that need a mapping, out of a
> > much larger set of elements.
> >
> > My problem is that, while this seems fairly simple, it means I lose
> > access to everything Camel-, Spring- and OSGi-related. Mostly, it
> > means I'd have strong coupling between all my dependencies, no way to
> > access any properties apart from System properties, no way to call
> > Camel routes, etc, etc, which makes me very nervous, and just doesn't
> > *feel* right (I know, I know - that's a terribly subjective
> > qualification!).
> >
> > Does anyone see a better solution, or have any advice about things I
> > should have a look at which could help? Can I, for example, gain
> > access to the current CamelContext so I can call out to a camel route
> > for address formatting or other such things?
> >
> > Or am I really just stuck going with this:
> >
> > public static NodeList formatAddress(NodeList address) { String unitNo
> > = address.item(0).getChildNodes().item(0).getTextContent(); ... }
> >
> > public static String translate(String type, String val) { InputStream
> > is = XSLUtil.class.getResourceAsStream(...); ... }
> >
> > Thanks,
> >
> > - Andrew Thorburn
>
>
Accessing property on bean in header?
I've been having a bunch of trouble trying to do something that seems
like it should be really simple... Using Camel 2.10.7 on ServiceMix
4.5.2.
Basically, I'm sending a SOAP message, which contains a SOAP header
that looks like this:
?
?
?
Camel takes the bean from a CXF Endpoint in Payload format, and then
adds the SOAP headers to the Exchange Headers. In this case, the SOAP
header is being marshalled via JAXB before being added to the Exchange
Headers (so it's put into the headers as an object).
I now have a com.test.GlobalRequestHeader object in my exchange
headers, and want to access the property region, say.
How do I do this?
I've tried ${headerAs(soap.header.globalRequestHeader,
com.test.GlobalRequestHeader).region} and that doesn't work. I've
tried a bunch of other ways of doing in via Simple, and it just does
not work. Checking TRACE logs, I see that it's always being converted
to a String, instead of having methods called on it, which is a bit
strange. Is it possible that the fact that it's a JAXB-annotated class
is interfering with the Simple language somehow?
I can definitely access the header, because I see the XML being
written out to the log, but I don't want the goddamn XML, I want the
value of the property. But no matter how I try, all I see in the log
is the header objected converted to XML, not the value of the
property.
I have tried the following expressions:
[${headerAs(soap.header.globalRequestHeader,
com.test.GlobalRequestHeader).getRegion()}]
[${headerAs(soap.header.globalRequestHeader,
com.test.GlobalRequestHeader).region}]
[${in.headers[soap.header.globalRequestHeader].region}]
[${in.headers[soap.header.globalRequestHeader].getRegion()}]
The first two put out XML in the log. The second two put out null /
the empty string.
I really hope I'm missing something, because it doesn't seem like it
should be this hard to access a single property on a header object.
The alternative, I guess, would be to pipe the object through another
processor and set all the properties as individual headers, which
seems a bit absurd, frankly.
Thanks,
- Andrew
Re: Accessing property on bean in header?
I should add here that I want to log the result, so I have something like:
But even with the following:
${headerAs(soap.header.globalRequestHeader,
com.test.GlobalRequestHeader).getRegion()}
I still get the raw XML instead of the value of the region property.
It basically seems that what I want isn't possible, and that even
thinking about using SOAP headers here was a terrible idea, given the
amount of pain it's caused me.
Thanks,
- Andrew
On Wed, Dec 4, 2013 at 7:50 PM, Andrew Thorburn wrote:
> I've been having a bunch of trouble trying to do something that seems
> like it should be really simple... Using Camel 2.10.7 on ServiceMix
> 4.5.2.
>
> Basically, I'm sending a SOAP message, which contains a SOAP header
> that looks like this:
>
>
> ?
> ?
> ?
>
>
> Camel takes the bean from a CXF Endpoint in Payload format, and then
> adds the SOAP headers to the Exchange Headers. In this case, the SOAP
> header is being marshalled via JAXB before being added to the Exchange
> Headers (so it's put into the headers as an object).
>
> I now have a com.test.GlobalRequestHeader object in my exchange
> headers, and want to access the property region, say.
>
> How do I do this?
>
> I've tried ${headerAs(soap.header.globalRequestHeader,
> com.test.GlobalRequestHeader).region} and that doesn't work. I've
> tried a bunch of other ways of doing in via Simple, and it just does
> not work. Checking TRACE logs, I see that it's always being converted
> to a String, instead of having methods called on it, which is a bit
> strange. Is it possible that the fact that it's a JAXB-annotated class
> is interfering with the Simple language somehow?
>
> I can definitely access the header, because I see the XML being
> written out to the log, but I don't want the goddamn XML, I want the
> value of the property. But no matter how I try, all I see in the log
> is the header objected converted to XML, not the value of the
> property.
>
> I have tried the following expressions:
>
> [${headerAs(soap.header.globalRequestHeader,
> com.test.GlobalRequestHeader).getRegion()}]
> [${headerAs(soap.header.globalRequestHeader,
> com.test.GlobalRequestHeader).region}]
> [${in.headers[soap.header.globalRequestHeader].region}]
> [${in.headers[soap.header.globalRequestHeader].getRegion()}]
>
> The first two put out XML in the log. The second two put out null /
> the empty string.
>
> I really hope I'm missing something, because it doesn't seem like it
> should be this hard to access a single property on a header object.
> The alternative, I guess, would be to pipe the object through another
> processor and set all the properties as individual headers, which
> seems a bit absurd, frankly.
>
> Thanks,
>
> - Andrew
Re: Accessing property on bean in header?
Apologies for the spam, but this should be the last post, and I
believe there's a bug somewhere. According the documentation
(http://camel.apache.org/simple.html), I should be able to do
something like this:
${in.header.type} is 'java.lang.String'
And I will get back true or false. However, the following
${in.headers[soap.header.globalRequestHeader]}
is 'com.test.GlobalRequestHeader'
Sets the header to the following value:
?
?
is 'com.proaxiom.hsbc.xsd.request.GlobalRequestHeader'
OK, possibly that's due to setting it to java.lang.Object.
I tried again, but with resultType="java.lang.Boolean" and got back
"false", which is pretty clearly not correct. Tried a few variations
on that, and the value on the left is always converted to a String
before any further processing is conducted, making the "is" operator a
bit pointless (in one case, using ${bean:camelContext}, I got a nasty
exception, which I can share if needed). I assume this is a bug? Has
it been fixed in a later version of Camel?
- Andrew
On Wed, Dec 4, 2013 at 8:03 PM, Andrew Thorburn wrote:
> I should add here that I want to log the result, so I have something like:
>
> loggingLevel="INFO" />
>
> But even with the following:
>
>
> resultType="java.lang.Object">${headerAs(soap.header.globalRequestHeader,
> com.test.GlobalRequestHeader).getRegion()}
>
>
> I still get the raw XML instead of the value of the region property.
> It basically seems that what I want isn't possible, and that even
> thinking about using SOAP headers here was a terrible idea, given the
> amount of pain it's caused me.
>
> Thanks,
>
> - Andrew
>
> On Wed, Dec 4, 2013 at 7:50 PM, Andrew Thorburn wrote:
>> I've been having a bunch of trouble trying to do something that seems
>> like it should be really simple... Using Camel 2.10.7 on ServiceMix
>> 4.5.2.
>>
>> Basically, I'm sending a SOAP message, which contains a SOAP header
>> that looks like this:
>>
>>
>> ?
>> ?
>> ?
>>
>>
>> Camel takes the bean from a CXF Endpoint in Payload format, and then
>> adds the SOAP headers to the Exchange Headers. In this case, the SOAP
>> header is being marshalled via JAXB before being added to the Exchange
>> Headers (so it's put into the headers as an object).
>>
>> I now have a com.test.GlobalRequestHeader object in my exchange
>> headers, and want to access the property region, say.
>>
>> How do I do this?
>>
>> I've tried ${headerAs(soap.header.globalRequestHeader,
>> com.test.GlobalRequestHeader).region} and that doesn't work. I've
>> tried a bunch of other ways of doing in via Simple, and it just does
>> not work. Checking TRACE logs, I see that it's always being converted
>> to a String, instead of having methods called on it, which is a bit
>> strange. Is it possible that the fact that it's a JAXB-annotated class
>> is interfering with the Simple language somehow?
>>
>> I can definitely access the header, because I see the XML being
>> written out to the log, but I don't want the goddamn XML, I want the
>> value of the property. But no matter how I try, all I see in the log
>> is the header objected converted to XML, not the value of the
>> property.
>>
>> I have tried the following expressions:
>>
>> [${headerAs(soap.header.globalRequestHeader,
>> com.test.GlobalRequestHeader).getRegion()}]
>> [${headerAs(soap.header.globalRequestHeader,
>> com.test.GlobalRequestHeader).region}]
>> [${in.headers[soap.header.globalRequestHeader].region}]
>> [${in.headers[soap.header.globalRequestHeader].getRegion()}]
>>
>> The first two put out XML in the log. The second two put out null /
>> the empty string.
>>
>> I really hope I'm missing something, because it doesn't seem like it
>> should be this hard to access a single property on a header object.
>> The alternative, I guess, would be to pipe the object through another
>> processor and set all the properties as individual headers, which
>> seems a bit absurd, frankly.
>>
>> Thanks,
>>
>> - Andrew
Re: Accessing property on bean in header?
Yes, I did eventually figure that out, and when I did, the predicate
returns false due to the left-hand side being converted to a String before
the comparison takes place. Given that it's probably a bug, is it likely
fixed in a later version of Camel? I'll try to check it myself, probably in
a day or so.
- Andrew
On 4/12/2013 8:28 PM, "Claus Ibsen" wrote:
> On Wed, Dec 4, 2013 at 8:22 AM, Andrew Thorburn wrote:
> > Apologies for the spam, but this should be the last post, and I
> > believe there's a bug somewhere. According the documentation
> > (http://camel.apache.org/simple.html), I should be able to do
> > something like this:
> >
> > ${in.header.type} is 'java.lang.String'
> >
> > And I will get back true or false. However, the following
> >
> >
> > resultType="java.lang.Object">${in.headers[soap.header.globalRequestHeader]}
> > is 'com.test.GlobalRequestHeader'
> >
> >
>
> This is an expression and not a predicate.
>
> You would need to set the resultType to boolean so Camel would know
> its a predicate.
> And not sure if Camel 2.10.x is too old to support that way.
> Newer releases sure does.
>
>
>
>
>
>
> > Sets the header to the following value:
> >
> >
> > xmlns:ns2="uri:com:test">
> > ?
> > ?
> >
> > is 'com.proaxiom.hsbc.xsd.request.GlobalRequestHeader'
> >
> > OK, possibly that's due to setting it to java.lang.Object.
> >
> > I tried again, but with resultType="java.lang.Boolean" and got back
> > "false", which is pretty clearly not correct. Tried a few variations
> > on that, and the value on the left is always converted to a String
> > before any further processing is conducted, making the "is" operator a
> > bit pointless (in one case, using ${bean:camelContext}, I got a nasty
> > exception, which I can share if needed). I assume this is a bug? Has
> > it been fixed in a later version of Camel?
> >
> > - Andrew
> >
> > On Wed, Dec 4, 2013 at 8:03 PM, Andrew Thorburn
> wrote:
> >> I should add here that I want to log the result, so I have something
> like:
> >>
> >> >> loggingLevel="INFO" />
> >>
> >> But even with the following:
> >>
> >>
> >> resultType="java.lang.Object">${headerAs(soap.header.globalRequestHeader,
> >> com.test.GlobalRequestHeader).getRegion()}
> >>
> >>
> >> I still get the raw XML instead of the value of the region property.
> >> It basically seems that what I want isn't possible, and that even
> >> thinking about using SOAP headers here was a terrible idea, given the
> >> amount of pain it's caused me.
> >>
> >> Thanks,
> >>
> >> - Andrew
> >>
> >> On Wed, Dec 4, 2013 at 7:50 PM, Andrew Thorburn
> wrote:
> >>> I've been having a bunch of trouble trying to do something that seems
> >>> like it should be really simple... Using Camel 2.10.7 on ServiceMix
> >>> 4.5.2.
> >>>
> >>> Basically, I'm sending a SOAP message, which contains a SOAP header
> >>> that looks like this:
> >>>
> >>>
> >>> ?
> >>> ?
> >>> ?
> >>>
> >>>
> >>> Camel takes the bean from a CXF Endpoint in Payload format, and then
> >>> adds the SOAP headers to the Exchange Headers. In this case, the SOAP
> >>> header is being marshalled via JAXB before being added to the Exchange
> >>> Headers (so it's put into the headers as an object).
> >>>
> >>> I now have a com.test.GlobalRequestHeader object in my exchange
> >>> headers, and want to access the property region, say.
> >>>
> >>> How do I do this?
> >>>
> >>> I've tried ${headerAs(soap.header.globalRequestHeader,
> >>> com.test.GlobalRequestHeader).region} and that doesn't work. I've
> >>> tried a bunch of other ways of doing in via Simple, and it just does
> >>> not work. Checking TRACE logs, I see that it's always being converted
> >>> to a String, instead of having methods called on it, which is a bit
> >>> strange. Is it possible that the fact that it's a JAXB-annotated class
> >>> is interfering with the Simple language somehow?
> >>>
> >>> I can definitely access
Re: Accessing property on bean in header?
Something is broken here - not sure if it's me or Camel, but I can't
say that I expected that,given the following:
setHeader("test").simple("${in.header.simpleBean} == null",
Boolean.class).setHeader("test2").simple("${in.headers.simpleBean} !=
null", Boolean.class)
*Both* headers are false! This is using Camel 2.12.2, so the latest
version. The full output and code is here:
http://pastebin.com/xdSsatEP, and it's pretty simple, so if someone
could explain what I'm doing wrong that would be awesome.
The header, simpleBean, is definitely not null - that's confirmed in a
log message and by actually checking the header in another processor,
so I think something has gone horribly wrong in Camel... I also still
can't extract properties from header objects, so I'm assuming that's
not supported.
Thanks,
- Andrew
On Wed, Dec 4, 2013 at 8:36 PM, Andrew Thorburn wrote:
> Yes, I did eventually figure that out, and when I did, the predicate returns
> false due to the left-hand side being converted to a String before the
> comparison takes place. Given that it's probably a bug, is it likely fixed
> in a later version of Camel? I'll try to check it myself, probably in a day
> or so.
>
> - Andrew
>
> On 4/12/2013 8:28 PM, "Claus Ibsen" wrote:
>>
>> On Wed, Dec 4, 2013 at 8:22 AM, Andrew Thorburn wrote:
>> > Apologies for the spam, but this should be the last post, and I
>> > believe there's a bug somewhere. According the documentation
>> > (http://camel.apache.org/simple.html), I should be able to do
>> > something like this:
>> >
>> > ${in.header.type} is 'java.lang.String'
>> >
>> > And I will get back true or false. However, the following
>> >
>> >
>> > > > resultType="java.lang.Object">${in.headers[soap.header.globalRequestHeader]}
>> > is 'com.test.GlobalRequestHeader'
>> >
>> >
>>
>> This is an expression and not a predicate.
>>
>> You would need to set the resultType to boolean so Camel would know
>> its a predicate.
>> And not sure if Camel 2.10.x is too old to support that way.
>> Newer releases sure does.
>>
>>
>>
>>
>>
>>
>> > Sets the header to the following value:
>> >
>> >
>> > > > xmlns:ns2="uri:com:test">
>> > ?
>> > ?
>> >
>> > is 'com.proaxiom.hsbc.xsd.request.GlobalRequestHeader'
>> >
>> > OK, possibly that's due to setting it to java.lang.Object.
>> >
>> > I tried again, but with resultType="java.lang.Boolean" and got back
>> > "false", which is pretty clearly not correct. Tried a few variations
>> > on that, and the value on the left is always converted to a String
>> > before any further processing is conducted, making the "is" operator a
>> > bit pointless (in one case, using ${bean:camelContext}, I got a nasty
>> > exception, which I can share if needed). I assume this is a bug? Has
>> > it been fixed in a later version of Camel?
>> >
>> > - Andrew
>> >
>> > On Wed, Dec 4, 2013 at 8:03 PM, Andrew Thorburn
>> > wrote:
>> >> I should add here that I want to log the result, so I have something
>> >> like:
>> >>
>> >> > >> loggingLevel="INFO" />
>> >>
>> >> But even with the following:
>> >>
>> >>
>> >> > >> resultType="java.lang.Object">${headerAs(soap.header.globalRequestHeader,
>> >> com.test.GlobalRequestHeader).getRegion()}
>> >>
>> >>
>> >> I still get the raw XML instead of the value of the region property.
>> >> It basically seems that what I want isn't possible, and that even
>> >> thinking about using SOAP headers here was a terrible idea, given the
>> >> amount of pain it's caused me.
>> >>
>> >> Thanks,
>> >>
>> >> - Andrew
>> >>
>> >> On Wed, Dec 4, 2013 at 7:50 PM, Andrew Thorburn
>> >> wrote:
>> >>> I've been having a bunch of trouble trying to do something that seems
>> >>> like it should be really simple... Using Camel 2.10.7 on ServiceMix
>> >>> 4.5.2.
>> >>>
>> >>> Basically, I'm sending a SOAP message, which contains a SOAP header
>> >>> that looks like this:
>> >>>
Re: Accessing property on bean in header?
Had another crack at this, and my ultimate problem was that I didn't
read the documentation carefully enough - I can do what I want, just
not how I wanted to do it.
What I wanted to have was a property called "a.b.c", and then be able
to call an OGNL expression on it. I had thought I could do it by using
the following fashion to get a header: in.headers[a.b.c].OGNL - that
does not work, and the documentation does not list that as a
possibility - OK, that's my bad, and I'll raise a feature request if I
want to see supported.
The other way I had initially tried, which had not worked, was
in.headers.a.b.c.OGNL - if I want to use an OGNL expression to access
a header property, I cannot have a '.' in the name of the header.
That's something I feel the documentation could be clearer on, though
it never said my way would work, and after a bit of though I'm not
surprised that it doesn't work.
Anyway, I can do what I want, I just have to name my header
soap_header_* instead of soap.header.* (or what have you - just so
long as the header name does not contain a period). Case closed.
Thanks,
- Andrew
On Thu, Dec 5, 2013 at 2:15 PM, Andrew Thorburn wrote:
> Something is broken here - not sure if it's me or Camel, but I can't
> say that I expected that,given the following:
>
> setHeader("test").simple("${in.header.simpleBean} == null",
> Boolean.class).setHeader("test2").simple("${in.headers.simpleBean} !=
> null", Boolean.class)
>
> *Both* headers are false! This is using Camel 2.12.2, so the latest
> version. The full output and code is here:
> http://pastebin.com/xdSsatEP, and it's pretty simple, so if someone
> could explain what I'm doing wrong that would be awesome.
>
> The header, simpleBean, is definitely not null - that's confirmed in a
> log message and by actually checking the header in another processor,
> so I think something has gone horribly wrong in Camel... I also still
> can't extract properties from header objects, so I'm assuming that's
> not supported.
>
> Thanks,
>
> - Andrew
>
> On Wed, Dec 4, 2013 at 8:36 PM, Andrew Thorburn wrote:
>> Yes, I did eventually figure that out, and when I did, the predicate returns
>> false due to the left-hand side being converted to a String before the
>> comparison takes place. Given that it's probably a bug, is it likely fixed
>> in a later version of Camel? I'll try to check it myself, probably in a day
>> or so.
>>
>> - Andrew
>>
>> On 4/12/2013 8:28 PM, "Claus Ibsen" wrote:
>>>
>>> On Wed, Dec 4, 2013 at 8:22 AM, Andrew Thorburn wrote:
>>> > Apologies for the spam, but this should be the last post, and I
>>> > believe there's a bug somewhere. According the documentation
>>> > (http://camel.apache.org/simple.html), I should be able to do
>>> > something like this:
>>> >
>>> > ${in.header.type} is 'java.lang.String'
>>> >
>>> > And I will get back true or false. However, the following
>>> >
>>> >
>>> > >> > resultType="java.lang.Object">${in.headers[soap.header.globalRequestHeader]}
>>> > is 'com.test.GlobalRequestHeader'
>>> >
>>> >
>>>
>>> This is an expression and not a predicate.
>>>
>>> You would need to set the resultType to boolean so Camel would know
>>> its a predicate.
>>> And not sure if Camel 2.10.x is too old to support that way.
>>> Newer releases sure does.
>>>
>>>
>>>
>>>
>>>
>>>
>>> > Sets the header to the following value:
>>> >
>>> >
>>> > >> > xmlns:ns2="uri:com:test">
>>> > ?
>>> > ?
>>> >
>>> > is 'com.proaxiom.hsbc.xsd.request.GlobalRequestHeader'
>>> >
>>> > OK, possibly that's due to setting it to java.lang.Object.
>>> >
>>> > I tried again, but with resultType="java.lang.Boolean" and got back
>>> > "false", which is pretty clearly not correct. Tried a few variations
>>> > on that, and the value on the left is always converted to a String
>>> > before any further processing is conducted, making the "is" operator a
>>> > bit pointless (in one case, using ${bean:camelContext}, I got a nasty
>>> > exception, which I can share if needed). I assume this is a bug? Has
>>> > it been fixed in a later version of Camel?
>>&g
JMS failed because destination does not exist, but keeps trying to refresh?
Using SMX 4.5.3 and Camel 2.10.7 (the version included with SMX). I've got my application set up to use the IBM MQ JMS APIs to transfer messages to a WebSphere MQ server. That works fine (which surprised me, frankly!), but if the destination doesn't exist, I see the following in the log: 19:14:11,034 | WARN | ] | entQueueMessageListenerContainer | 104 - org.springframework.jms - 3.0.7.RELEASE | Setup of JMS message listener invoker failed for destination '' - trying to recover. Cause: MQJMS2008: failed to open MQ queue ; nested exception is com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2085 19:14:11,458 | INFO | ] | entQueueMessageListenerContainer | 104 - org.springframework.jms - 3.0.7.RELEASE | Successfully refreshed JMS Connection In this case, I'm using a Request/Reply pattern, with the request going out on Queue A, and the reply coming in on Queue B. If neither A nor B exist, I get the above error in the logs, every 10 seconds or so, as it tries to reconnect to the reply queue (which fails, because it does not and will never exist). It also spits out an error connecting to the request queue, but that's not a surprise, and only occurs once. Does anyone know how I can deal with this? Do I need to set a particular property on the JMS Config or something? I see there is a receiveTimeout, but since it's not even connecting, I'm not sure that's going to be much use. Thanks, - Andrew Thorburn
Re: JMS failed because destination does not exist, but keeps trying to refresh?
Bean & Context Settings (Blueprint file):
http://www.osgi.org/xmlns/blueprint/v1.0.0";
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0";
xmlns:camel="http://camel.apache.org/schema/blueprint";
xmlns:camelcxf="http://camel.apache.org/schema/blueprint/cxf";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint
http://camel.apache.org/schema/blueprint/camel-blueprint-2.10.7.xsd
http://camel.apache.org/schema/blueprint/cxf
http://camel.apache.org/schema/blueprint/cxf/camel-cxf-2.9.0.xsd
http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0
http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.1.0.xsd
">
http://ws.client.org.com/";>
http://ws.client.org.com/}clientPort"; />
nmr:requestTransform
imq:queue:REQUEST_QUEUE?replyTo=REPLY_QUEUE&replyToType=Shared
nmr:responseTransform
${in.header.requestTransformUri}
${in.header.mqUri}
${in.header.responseTransformUri}
Thanks,
- Andrew Thorburn
On Tue, Jan 28, 2014 at 1:21 PM, Andrew Thorburn wrote:
> Using SMX 4.5.3 and Camel 2.10.7 (the version included with SMX).
>
> I've got my application set up to use the IBM MQ JMS APIs to transfer
> messages to a WebSphere MQ server. That works fine (which surprised
> me, frankly!), but if the destination doesn't exist, I see the
> following in the log:
>
> 19:14:11,034 | WARN | ] |
> entQueueMessageListenerContainer | 104 - org.springframework.jms -
> 3.0.7.RELEASE | Setup of JMS message listener invoker failed for
> destination '' - trying to recover. Cause: MQJMS2008:
> failed to open MQ queue ; nested exception is com.ibm.mq.MQException:
> MQJE001: Completion Code 2, Reason 2085
> 19:14:11,458 | INFO | ] |
> entQueueMessageListenerContainer | 104 - org.springframework.jms -
> 3.0.7.RELEASE | Successfully refreshed JMS Connection
>
> In this case, I'm using a Request/Reply pattern, with the request
> going out on Queue A, and the reply coming in on Queue B. If neither A
> nor B exist, I get the above error in the logs, every 10 seconds or
> so, as it tries to reconnect to the reply queue (which fails, because
> it does not and will never exist). It also spits out an error
> connecting to the request queue, but that's not a surprise, and only
> occurs once.
>
> Does anyone know how I can deal with this? Do I need to set a
> particular property on the JMS Config or something? I see there is a
> receiveTimeout, but since it's not even connecting, I'm not sure
> that's going to be much use.
>
> Thanks,
>
> - Andrew Thorburn
>
JMS Request/Reply but without setting JMSReplyTo header on outgoing message?
I've got a slightly awkward situation here. I've set up ServiceMix 4.5.3
with Camel 2.10.7, which is acting as a transformer and routing tool, and
has the following flow:
Receive message from CXF SOAP Web Service.
Transform message
Put message onto one of several WebSphere MQ Queues (using the MQ JMS
libraries)
Wait for a reply on one of several shared reply queues
Transform the reply
Send the reply.
The problem that I'm now having is that, for *one* of the request queues, I
need to ensure that the R2Q field in the MQMD is blank. That is, I should
not be passing the replyToQueue with the rest of the message. Since I'm
working several layers up from the actual MQ libraries, I'm not sure
exactly how I can do this.
The JMS endpoint is defined as
imq:queue:DUMMY?replyTo=${in.header.replyQueueName}&replyToType=Shared&useMessageIDAsCorrelationID=true
(The request queue is set with the header CamelJmsDestinationName).
So is there any way that I can listen for a reply on the queue defined in
the header ${in.header.replyQueueName} but without setting the JMSReplyTo
header on the outgoing message?
The actual reason for this is that the message is routed through several MQ
servers, and the reply queue that I'm using is *not* the same as the queue
that the other side will be putting the message on.
It goes something like:
Camel -> MQA
Camel listening for a reply on QueueA
MQA -> MQB
MQB -> MQC
MQC -> MQD
MQD puts reply on QueueB.
MQD -> MQC
MQC -> MQB
MQB -> MQA
MQA -> Camel
And at some point there, the message is moved from QueueB to QueueA. But if
the replyToQueue is set to QueueA, the other side is unable to deal with
that (they try and put it on that queue, which doesn't exist on MQD), so I
need to avoid setting it. Note that the replyToQueue is passed unchanged
from MQA -> MQB -> MQC -> MQD, which I suspect isn't correct, but is also
not something I can change.
If this seems a bit confusing, it's probably because I'm a bit confused.
Also, I *only* have control over the Camel side of things. I have zero
control over any of the MQ servers.
The decision about which queue to put the request or reply on is done on a
per-environment level - basically we just have a properties file saying
which queue to use (I believe it's roughly the same on the other side).
Thanks,
- Andrew Thorburn
Re: sql component and delay
It should be "&consumer.delay=5000". Since you're using XML, you'll
need to escape the & character.
- Andrew
On Tue, Mar 25, 2014 at 12:08 PM, Lydie wrote:
> When I do that:
>
>
> uri="sql:{{sql.oracle.selectPull2}}?consumer.onConsume={{sql.oracle.markProcessed}}&consumer.delay=5000"
> />
>
> I get the following error:
>
> org.springframework.beans.factory.xml.XmlBeanDefinitionStoreException:
> Line 78 in XML document from class path resource
> [META-INF/spring/camel-context.xml] is invalid; nested exception is
> org.xml.sax.SAXParseException; lineNumber: 78; columnNumber: 103; The
> reference to entity "consumer.delay" must end with the ';' delimiter.
>at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.doLoadBeanDefinitions(XmlBeanDefinitionReader.java:396)
>at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:334)
>at
> org.springframework.beans.factory.xml.XmlBeanDefinitionReader.loadBeanDefinitions(XmlBeanDefinitionReader.java:302)
>at
> org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:174)
>at
> org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:209)
>at
> org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:180)
>at
> org.springframework.beans.factory.support.AbstractBeanDefinitionReader.loadBeanDefinitions(AbstractBeanDefinitionReader.java:243)
>at
> org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:127)
>at
> org.springframework.context.support.AbstractXmlApplicationContext.loadBeanDefinitions(AbstractXmlApplicationContext.java:93)
>at
> org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:131)
>at
> org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:530)
>at
> org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:444)
>at
> org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:139)
>at
> org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:83)
>at
> com.cgi.horizon.integration.TestHorizonIntegration.createApplicationContext(TestHorizonIntegration.java:40)
>at
> org.apache.camel.test.spring.CamelSpringTestSupport.doCreateApplicationContext(CamelSpringTestSupport.java:89)
>at
> org.apache.camel.test.spring.CamelSpringTestSupport.doPreSetup(CamelSpringTestSupport.java:79)
>at
> org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:216)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
>at java.lang.reflect.Method.invoke(Unknown Source)
>at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
>at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>at
> org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>at
> org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>
Re: CXF Schema Validation with a Custom Response
Assuming that your schema is not embedded in the WSDL, but in separate XSD
files, you should look at the Validator Component:
http://camel.apache.org/validation.html
As in the example there, you can just catch a
org.apache.camel.ValidationException
(or, more specifically, a org.apache.camel.SchemaValidationException), and
use that to get what failed. It's not perfect, but it may work for your
needs.
- Andrew
On Thu, Jul 17, 2014 at 10:33 PM, Rob wrote:
> Hi
> I'm trying to employ schema validation on a camel route which uses
> cxf:cxfEndpoint:
>
> address="/{{mainPath}}/v2"
> serviceClass="com.myCompany.v2.ServicesSoap">
>
> value="${mainSchemaValidation}" />
>
>
>
> What I need to do is create a custom response using the error message as
> part of the reply but I need to be able to set the Exchange body after
> getting the error message. ie I have a ResultSet in my xsd which would have
> the description set using the validation error message.
>
> I have tried using only this doesn't catch it as the error
> seems to be thrown during the cxf phase and is not within the route. I
> have
> investigated the use of interceptors but it looks rather complex for my
> needs and experience.
>
> If there are any alternatives or explanations on how I might use
> interceptors for this that would be appreciated. I am using camel version
> 2.12.2 and apache cxf version 2.7.8.
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/CXF-Schema-Validation-with-a-Custom-Response-tp5753958.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
JMS - asyncConsumer &
Hey guys, Just want to clarify that I understand how this works correctly: Setting the "asyncConsumer" option to "true" on a JMS Consumer does not, in and of itself, mean that messages are guaranteed to be processed asynchronously, but it does mean that if an endpoint supports asynchronous behaviour, the Camel JMS consumer will allow that. In other words, if you set "asyncConsumer" to "true", but none of your endpoints in your pipeline allow asynchronous processing, it will all get processed synchronously anyway. In that case, you would need to wrap everything you wanted processed asynchronously in a tag (or a threads() call), which would allow you to control the number of threads, the queue size, etc. Conversely, even if you provide a wrapper, if asyncConsumer is set to "false", it will get processed synchronously. Is that correct? It appears to be, based on what I can see in the source code (looking at Threads, JMS Configuration, JMS Consumer, EndpointMessageListener, etc). If that's correct, wouldn't it make sense to rename the option asyncConsumer to asyncAllowed (or maybe asyncConsumerAllowed)? asyncConsumer implies, at least to me, that the consumer will somehow magically run everything asynchronously without the need for further configuration, when that does not appear to be true. asyncAllowed is closer to what actually happens - in that setting the option does not magically make everything asynchronous, but if you *want* to make it asynchronous yourself, you can. Also, this implies that the example at the bottom of http://camel.apache.org/async.html is, if not wrong, then at least missing information, as it does not show that the JMS consumer has had the "asyncConsumer" option set on it anywhere. FWIW, as I'm integrating with WebSphere MQ, and the queue I'm consuming from is set to EXCLUSIVE_OPEN (or something along those lines), I don't believe that I can make any use of the concurrentConsumers/maxConcurrentConsumers options on the JMS endpoint. Thanks, - Andrew Thorburn
Re: JMS - asyncConsumer &
I'd be happy to update the docs, though I've haven't signed the agreement yet. Can I take it that I understand what's going on correctly, regarding the asyncConsumer option? Thanks, - Andrew On Thu, Sep 25, 2014 at 7:35 PM, Claus Ibsen wrote: > Hi > > You are welcome to improve the docs, eg add some details about this to > the asyncConsumer option. > http://camel.apache.org/jms > > Though maybe its best placed be on the async routing engine page, as > it applies to how it works and under which circumstances. > http://camel.apache.org/asynchronous-routing-engine.html > > And then you can refer to this page from the jms page, for more details. > > And lets keep the option name as is, as this is the naming convention > used by Camel and in the source code as well. > > We love contributions > http://camel.apache.org/contributing.html > > And here is how to help edit the docs > http://camel.apache.org/how-do-i-edit-the-website.html > > On Thu, Sep 25, 2014 at 4:07 AM, Andrew Thorburn wrote: > > Hey guys, > > > > Just want to clarify that I understand how this works correctly: > > > > Setting the "asyncConsumer" option to "true" on a JMS Consumer does not, > in > > and of itself, mean that messages are guaranteed to be processed > > asynchronously, but it does mean that if an endpoint supports > asynchronous > > behaviour, the Camel JMS consumer will allow that. > > > > In other words, if you set "asyncConsumer" to "true", but none of your > > endpoints in your pipeline allow asynchronous processing, it will all get > > processed synchronously anyway. > > > > In that case, you would need to wrap everything you wanted processed > > asynchronously in a tag (or a threads() call), which would > allow > > you to control the number of threads, the queue size, etc. > > > > Conversely, even if you provide a wrapper, if asyncConsumer is > > set to "false", it will get processed synchronously. > > > > Is that correct? It appears to be, based on what I can see in the source > > code (looking at Threads, JMS Configuration, JMS Consumer, > > EndpointMessageListener, etc). > > > > If that's correct, wouldn't it make sense to rename the option > > asyncConsumer to asyncAllowed (or maybe asyncConsumerAllowed)? > > asyncConsumer implies, at least to me, that the consumer will somehow > > magically run everything asynchronously without the need for further > > configuration, when that does not appear to be true. asyncAllowed is > closer > > to what actually happens - in that setting the option does not magically > > make everything asynchronous, but if you *want* to make it asynchronous > > yourself, you can. > > > > Also, this implies that the example at the bottom of > > http://camel.apache.org/async.html is, if not wrong, then at least > missing > > information, as it does not show that the JMS consumer has had the > > "asyncConsumer" option set on it anywhere. > > > > FWIW, as I'm integrating with WebSphere MQ, and the queue I'm consuming > > from is set to EXCLUSIVE_OPEN (or something along those lines), I don't > > believe that I can make any use of the > > concurrentConsumers/maxConcurrentConsumers options on the JMS endpoint. > > > > Thanks, > > > > - Andrew Thorburn > > > > -- > Claus Ibsen > - > Red Hat, Inc. > Email: cib...@redhat.com > Twitter: davsclaus > Blog: http://davsclaus.com > Author of Camel in Action: http://www.manning.com/ibsen > hawtio: http://hawt.io/ > fabric8: http://fabric8.io/ >
Re: Not all processors are listed in JMX, preventing detailed route statistics/profiling
I've seen issues like this in HawtIO, and it's actually a Jolokia issue. Basically, Jolokia will only fetch the first N elements (unsorted, I believe) from the server, and if your MBean isn't one of those N, then... bugger? The solution is to go into HawtIO / Preferences / Jolokia, and change Max Collection Size from 500 (the default) up to a larger number. I went to 5000, and all of a sudden all my routes appeared in the Camel console, and in the JMX tab. This will have an impact on browser performance, but I'm not sure there's much that can be done, apart from raising a feature request with Jolokia / HawtIO-Camel to fetch only a subset of available MBeans, rather than fetching them all and filtering on the client-side. Now, if you're looking at the JMX info through a different source (not HawtIO/Jolokia) and still not seeing them all, then this probably won't help much. - Andrew On Fri, Oct 17, 2014 at 7:41 AM, BlackTie wrote: > I have a webapp using Camel (v2.13.2) about to go into production, at the > point of trying to tune things for performance. I'm having an issue with > getting detail statistics for some routes, though. I'm using Hawtio 1.2, > and for some routes, Hawtio is displaying the statistics for each processor > in the route, and others it only shows the statistics at the route level. > I > checked JMX with jconsole and noticed that, for the routes that don't show > specifics for the processors, the processors in those routes are not listed > under the processors section in JMX. Some routes with this problem are > complex, others are fairly simple. > > Any ideas what would cause some to/choice/multicast/etc processors from not > being listed in JMX at all? Without the statistics, it makes it very > difficult to figure out where things are slowing down when the system is > under heavy load. > > The jmxAgent is enabled for the camel context, with statisticsLevel set to > All. > > Thanks for any help you can provide on this problem! > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Not-all-processors-are-listed-in-JMX-preventing-detailed-route-statistics-profiling-tp5757634.html > Sent from the Camel - Users mailing list archive at Nabble.com. >
HTTP4 Component and encoding message body
When sending a message to an HTTP4 Endpoint, will the content of the message body be automatically encoded? e.g. The spaces replaced with %20 and whatever else is necessary to ensure that the POST request isn't mangled on the other side. I can't see anything in the documentation to tell me whether this is handled automatically or whether I need to convert it myself. At this stage it looks like it is handled automatically, since I don't see anything telling me otherwise, but I'm not sure. More to the point, how do I pass multiple parameters as the body? e.g. I'd like to have a Web Service call, transform whatever was passed to the web service to parameters for an HTTP POST request, and pass them to the HTTP4 endpoint. e.g. Given a Web Service Request, value1 value2&value2.1 have the body of the POST request be: "p1=value1&p2=value2%26value2.1". My request is a little more complex than that, but not by much. Hopefully what I want makes sense? Thanks, - Andrew
Re: HTTP4 Component and encoding message body
@Willem: Fair enough, then. I think it would be good if it was stated explicitly that the body is used as-is with no encoding, as I obviously found it confusing. @Scott: Undecided. Happy to use either POJO or PAYLOAD for CXF, whichever ends up being easier (probably PAYLOAD). And yes, there is a web service response that is required. The response from the HTTP call is very simple (and is not HTML), but I do need to extract some bits of it. Sadly, I don't think it's XML or JSON or anything, just a string. If it makes a difference, this will be deployed in SMX 4.4.1 (or later). Thanks, - Andrew On Fri, Jun 1, 2012 at 2:51 AM, Scott England-Sullivan wrote: > Hi Andrew, > > Regarding the second part of your question, do you use CXF to deserialize > the request Object? > > Also, is there a web service response expected after the HTTP4 POST is > processed? > > On Thu, May 31, 2012 at 4:54 AM, Willem Jiang >wrote: > > > Camel-HTTP4 components just takes the message body as an input stream, if > > you want to do the encoding you can add a processor or dataformat to do > > this kind of work. > > > > Willem > > > > On Thu, May 31, 2012 at 4:34 PM, Andrew Thorburn > wrote: > > > > > When sending a message to an HTTP4 Endpoint, will the content of the > > > message body be automatically encoded? e.g. The spaces replaced with > %20 > > > and whatever else is necessary to ensure that the POST request isn't > > > mangled on the other side. > > > > > > I can't see anything in the documentation to tell me whether this is > > > handled automatically or whether I need to convert it myself. At this > > stage > > > it looks like it is handled automatically, since I don't see anything > > > telling me otherwise, but I'm not sure. > > > > > > More to the point, how do I pass multiple parameters as the body? e.g. > > I'd > > > like to have a Web Service call, transform whatever was passed to the > web > > > service to parameters for an HTTP POST request, and pass them to the > > HTTP4 > > > endpoint. > > > > > > e.g. > > > > > > Given a Web Service Request, > > > > > > > > > > > >value1 > > >value2&value2.1 > > > > > > > > > > > > have the body of the POST request be: "p1=value1&p2=value2%26value2.1". > > My > > > request is a little more complex than that, but not by much. Hopefully > > what > > > I want makes sense? > > > > > > Thanks, > > > > > > - Andrew > > > > > > > > -- > Scott England-Sullivan > -- > FuseSource > Web: http://www.fusesource.com > Blog: http://sully6768.blogspot.com > Twitter: sully6768 >
