Good catch :) Jan
> -----Ursprüngliche Nachricht----- > Von: Aki Yoshida [mailto:elak...@gmail.com] > Gesendet: Donnerstag, 18. Dezember 2014 18:29 > An: users@camel.apache.org > Betreff: Re: AW: camel - xsd > > Jan and I had a couple of email exchange on this. > It turns out that the compliance check at the beginning of the > processing was not working when there is no xml-declaration in the > input. > Consequently, when your input message has no xml-declaration and you > have sjxp parser, you will get incorrectly extracted tokens. > I created CAMEL-8164 to fix this. > > 2014-12-17 19:35 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > > @Jan, > > the failing test itself is expected if you don't have woodstox. The > > question is how it is failing, whether it stops at the beginning and > > throws the exception like in the quoted console output in my previous > > reply or after extracting the bad token and when comparing the token? > > You mentioned that you observed this second behavior and I would like > > to know where I can reproduce it. > > > > thanks. > > > > > > 2014-12-17 16:40 GMT+01:00 Jan Matèrne (jhm) <apa...@materne.de>: > >> Java(TM) SE Runtime Environment (build 1.7.0_21-b11) Java > HotSpot(TM) > >> 64-Bit Server VM (build 23.21-b01, mixed mode) by Oracle (according > >> to the copyright file) > >> - without woodstox: fail > >> - with woodstox: pass > >> > >> > >> Same for > >> Java(TM) SE Runtime Environment (build 1.6.0_43-b01) Java > HotSpot(TM) > >> 64-Bit Server VM (build 20.14-b01, mixed mode) (also Oracle) > >> > >> > >> Same for > >> Java(TM) SE Runtime Environment (build 1.8.0-ea-b87) Java > HotSpot(TM) > >> 64-Bit Server VM (build 25.0-b28, mixed mode) (also Oracle) > >> > >> > >> Same for > >> Java(TM) SE Runtime Environment (build 1.9.0-ea-b06) Java > HotSpot(TM) > >> 64-Bit Server VM (build 25.0-b62, mixed mode) (also Oracle) > >> > >> > >> All on Win7 64bit. > >> > >> > >> Jan > >> > >> > >> > >>> -----Ursprüngliche Nachricht----- > >>> Von: Aki Yoshida [mailto:elak...@gmail.com] > >>> Gesendet: Mittwoch, 17. Dezember 2014 15:49 > >>> An: users@camel.apache.org > >>> Cc: Jan Matèrne (jhm) > >>> Betreff: Re: AW: camel - xsd > >>> > >>> @Jan > >>> I just ran the camel-core's unit test > XMLTokenExpressionIteratorTest > >>> on > >>> jdk8 without woodstox (use profile deactivation -P!woodstox to > >>> disable woodstox), the tokenizer correctly reports the error at the > >>> beginning without proceeding to the tokenizing step. (on both OSX > >>> and Ubuntu with Oracle JDK8). > >>> > >>> > testExtractSomeUnqualifiedChild(org.apache.camel.support.XMLTokenExp > >>> res > >>> sionIteratorTest) > >>> Time elapsed: 0.001 sec <<< ERROR! > >>> javax.xml.stream.XMLStreamException: reader not supporting Location > >>> at > >>> > org.apache.camel.support.XMLTokenExpressionIterator$XMLTokenIterator > >>> .<i > >>> nit>(XMLTokenExpressionIterator.java:219) > >>> > >>> Can you tell me which JDK you had to get the incorrectly extracted > >>> tokens? > >>> > >>> thanks > >>> > >>> 2014-12-16 14:27 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > >>> > @Jan, > >>> > I just missed your previous replies. > >>> > I just saw you also have provided the JDK info already. > >>> > thanks. > >>> > > >>> > 2014-12-16 14:25 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > >>> >> It's good to hear it is working in your environment. > >>> >> > >>> >> by the way, I saw your camael doc update regarding the concrete > >>> error > >>> >> example you saw. > >>> >> We need to formulate that differently. > >>> >> There is a small sanity check before starting the tokenization > >>> >> (not comparing the impl name itself but trying to infer its > >>> >> conformance > >>> by > >>> >> its initial behavior). That check was working for Oracle JDK 7 > on > >>> OSX. > >>> >> I would like to know which JDK that you use. In any case, we can > >>> >> either make the initial sanity check to detect this incompatible > >>> >> behavior and detect it later if it is not detected at the > beginning. > >>> >> In either way, we can throw an exception at some point. > >>> >> > >>> >> regards, aki > >>> >> > >>> >> 2014-12-16 10:18 GMT+01:00 Jan Matèrne (jhm) > <apa...@materne.de>: > >>> >>> Sorry for my late answer, I was ill :( > >>> >>> > >>> >>> Thanks for your sample project, I investigated into it now. > >>> >>> All tests passed. (Camel 2.14.0) > >>> >>> > >>> >>> But your test used the XMLTokenExpressionIterator directly I > >>> >>> migrated my own test class (using the RouteBuilder) into your > >>> project. > >>> >>> - added the test class > >>> >>> - added the xsd > >>> >>> - added the dependency to commons-io + camel-test-spring > >>> >>> - changed the paths in my test class to fit your directory > >>> >>> layout Now this also passed. > >>> >>> > >>> >>> It's good to see that there is no bug in Camel. ;) > >>> >>> > >>> >>> > >>> >>> Because my example still fails in my other environment I check > >>> >>> that > >>> ... > >>> >>> > >>> >>> > >>> >>> Jan > >>> >>> > >>> >>> > >>> >>>> -----Ursprüngliche Nachricht----- > >>> >>>> Von: Aki Yoshida [mailto:elak...@gmail.com] > >>> >>>> Gesendet: Mittwoch, 10. Dezember 2014 23:57 > >>> >>>> An: users@camel.apache.org > >>> >>>> Betreff: Re: AW: camel - xsd > >>> >>>> > >>> >>>> i meant woodstox-core-asl (e.g., woodstox-core-asl-4.4.1.jar). > >>> >>>> its maven coordinate is > >>> >>>> > >>> >>>> mvn:org.codehaus.woodstox/woodstox-core-asl/4.4.1 > >>> >>>> > >>> >>>> you need to have woodstox or some other parser that reliably > >>> >>>> reports the offset location at each parse event. > >>> >>>> Sjsxp (sun/oracle implementation included in JDK) doesn't do > >>> >>>> that, so you can't use it. > >>> >>>> > >>> >>>> jmtest.tar.gz contains a maven project that can be executed > >>> >>>> directly by typing mvn test at the console or can be imported > >>> >>>> into your eclipse IDE. > >>> >>>> > >>> >>>> 2014-12-10 13:13 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > >>> >>>> > i just created a test that uses your data and verified that > >>> >>>> > it > >>> is > >>> >>>> working fine. > >>> >>>> > please take a look at this file at my dropbox. > >>> >>>> > https://www.dropbox.com/s/pfgvs9si9bvujxo/jmtest.tar.gz?dl=0 > >>> >>>> > just extract the files into camel-core and run JMNewsTest. > >>> >>>> > and see if that works in your environment, (i believe it > does). > >>> >>>> > > >>> >>>> > in that case, we need to find out the difference to your > case. > >>> >>>> > do you have woodstox-api in your class path? > >>> >>>> > > >>> >>>> > > >>> >>>> > 2014-12-10 12:15 GMT+01:00 Aki Yoshida <elak...@gmail.com>: > >>> >>>> >> okay. > >>> >>>> >> let me take a look. > >>> >>>> >> > >>> >>>> >> > >>> >>>> >> 2014-12-10 12:02 GMT+01:00 Jan Matèrne (jhm) > >>> <apa...@materne.de>: > >>> >>>> >>> No, input is valid xml. > >>> >>>> >>> After invoking xtokenize() the xml is invalid. > >>> >>>> >>> > >>> >>>> >>> CAMEL-8106 is very .... short. I can't see whether it is > >>> related. > >>> >>>> >>> > >>> >>>> >>> > >>> >>>> >>> I posted an example on the user list > >>> >>>> >>> http://mail-archives.apache.org/mod_mbox/camel- > >>> >>>> users/201412.mbox/%3C > >>> >>>> >>> 002b01d01452%24e8ee27a0%24baca76e0%24%40de%3E > >>> >>>> >>> > >>> >>>> >>> I had done a 'workaround' for this special example > >>> >>>> >>> // Workaround of a bug?? in XMLTokenizerExpression > >>> >>>> >>> > .setBody(simple("${body.replace('</</news:Newsletter>', > >>> >>>> '</news:Newsletter>')}")) > >>> >>>> >>> .setBody(simple("${body.replace('<</news:Newsletter>', > >>> >>>> >>> '</news:Newsletter>')}")) > >>> >>>> >>> > >>> >>>> >>> Without that the split messages have wrong end tags. > >>> >>>> >>> > >>> >>>> >>> > >>> >>>> >>> Jan > >>> >>>> >>> > >>> >>>> >>> > >>> >>>> >>>> -----Ursprüngliche Nachricht----- > >>> >>>> >>>> Von: Aki Yoshida [mailto:elak...@gmail.com] > >>> >>>> >>>> Gesendet: Mittwoch, 10. Dezember 2014 10:34 > >>> >>>> >>>> An: users@camel.apache.org > >>> >>>> >>>> Betreff: Re: AW: camel - xsd > >>> >>>> >>>> > >>> >>>> >>>> you are talking about the invalid-xml parsing bug in > >>> xtokenzier. > >>> >>>> >>>> that has been fixe with CAMEL-8106. It should be part of > >>> >>>> >>>> the new 2.14.1, which is to be released shortly. > >>> >>>> >>>> > >>> >>>> >>>> regards, aki > >>> >>>> >>>> > >>> >>>> >>>> 2014-12-10 9:25 GMT+01:00 Jan Matèrne (jhm) > >>> <apa...@materne.de>: > >>> >>>> >>>> > I played a little bit. > >>> >>>> >>>> > > >>> >>>> >>>> > Jan > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > public class XmlTest extends CamelTestSupport { > >>> >>>> >>>> > > >>> >>>> >>>> > @EndpointInject(uri="mock:valid") > >>> >>>> >>>> > MockEndpoint valid; > >>> >>>> >>>> > > >>> >>>> >>>> > @EndpointInject(uri="mock:validationError") > >>> >>>> >>>> > MockEndpoint validationError; > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > @Test > >>> >>>> >>>> > public void validNewsfeed() throws Exception { > >>> >>>> >>>> > // Newsfeed contains 2 news > >>> >>>> >>>> > valid.expectedMessageCount(2); > >>> >>>> >>>> > valid.expectedMessagesMatches( > >>> >>>> >>>> > // Use helper methods from the static > >>> >>>> >>>> > imported > >>> >>>> >>>> PredicateBuilder > >>> >>>> >>>> > and( > >>> >>>> >>>> > > >>> >>>> >>>> > header("newsfeed.date").isEqualTo("2014.12.09 > >>> >>>> >>>> 14:15"), > >>> >>>> >>>> > header("news.author").isEqualTo("Jan"), > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > >>> >>>> > >>> body().contains("xmlns:news=\"http://www.materne.de/camel/test/xml/ > >>> >>>> >>>> com > >>> >>>> >>>> > plex/\ > >>> >>>> >>>> > "") > >>> >>>> >>>> > ) > >>> >>>> >>>> > ); > >>> >>>> >>>> > // no error expected > >>> >>>> >>>> > validationError.expectedMessageCount(0); > >>> >>>> >>>> > > >>> >>>> >>>> > // Read xml from classpath and send to Camel > route > >>> >>>> >>>> > String xml = > >>> >>>> >>>> > > >>> >>>> >>>> > >>> >>>> > >>> IOUtils.toString(getClass().getResourceAsStream("/de/materne/camel/ > >>> >>>> >>>> tes > >>> >>>> >>>> > t/xml/ > >>> >>>> >>>> > complex/validNewsfeed.xml")); > >>> >>>> >>>> > sendBody("direct:in", xml); > >>> >>>> >>>> > > >>> >>>> >>>> > // 'execute' all tests > >>> >>>> >>>> > assertMockEndpointsSatisfied(); > >>> >>>> >>>> > } > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > @Test > >>> >>>> >>>> > public void invalidXml() throws Exception { > >>> >>>> >>>> > valid.expectedMessageCount(0); > >>> >>>> >>>> > validationError.expectedMessageCount(1); > >>> >>>> >>>> > > >>> >>>> >>>> > String xml = "<xml/>"; > >>> >>>> >>>> > sendBody("direct:in", xml); > >>> >>>> >>>> > > >>> >>>> >>>> > assertMockEndpointsSatisfied(); > >>> >>>> >>>> > } > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > @Override > >>> >>>> >>>> > protected RouteBuilder createRouteBuilder() throws > >>> >>>> >>>> > Exception > >>> >>>> { > >>> >>>> >>>> > return new RouteBuilder() { > >>> >>>> >>>> > @Override > >>> >>>> >>>> > public void configure() throws Exception { > >>> >>>> >>>> > // Our XML uses namespaces, so we have > >>> >>>> >>>> > to deal with > >>> >>>> >>>> that. > >>> >>>> >>>> > Namespaces ns = new Namespaces("news", > >>> >>>> >>>> > "http://www.materne.de/camel/test/xml/complex/";); > >>> >>>> >>>> > > >>> >>>> >>>> > // XSD-invalid data goes to this > endpoint > >>> >>>> >>>> > onException(ValidationException.class) > >>> >>>> >>>> > .to("mock:validationError"); > >>> >>>> >>>> > > >>> >>>> >>>> > from("direct:in") > >>> >>>> >>>> > // XSD-validation > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> .to("validator:de/materne/camel/test/xml/complex/newsfeed.xs > >>> >>>> >>>> > d") > >>> >>>> >>>> > > >>> >>>> >>>> > // Store newsfeed data in the > header > >>> >>>> >>>> > before > >>> >>>> >>>> split, > >>> >>>> >>>> > so we haven't to do that on each > >>> >>>> >>>> > // splittet news-message. > >>> >>>> >>>> > .setHeader("newsfeed.date", > >>> >>>> >>>> > ns.xpath("/news:Newsletter/@date";, String.class)) > >>> >>>> >>>> > > >>> >>>> >>>> > // > >>> http://camel.apache.org/splitter.html > >>> >>>> >>>> > // xtokenize() is available since > >>> >>>> >>>> > Camel > >>> >>>> 2.14. > >>> >>>> >>>> > // Use the 'wrap'-mode so we keep > >>> >>>> >>>> > the > >>> >>>> >>>> > Newsletter- > >>> >>>> >>>> Header > >>> >>>> >>>> > > >>> >>>> >>>> > .split().xtokenize("/news:Newsletter/News";, > >>> >>>> >>>> > 'w', > >>> >>>> >>>> > ns) > >>> >>>> >>>> > > >>> >>>> >>>> > // Workaround of a bug?? in > >>> >>>> >>>> XMLTokenizerExpression > >>> >>>> >>>> > > >>> >>>> >>>> > .setBody(simple("${body.replace('</</news:Newsletter>', > >>> >>>> >>>> > '</news:Newsletter>')}")) > >>> >>>> >>>> > > >>> >>>> >>>> > .setBody(simple("${body.replace('<</news:Newsletter>', > >>> >>>> >>>> > '</news:Newsletter>')}")) > >>> >>>> >>>> > > >>> >>>> >>>> > // Get some data from the splittet > news > >>> >>>> >>>> > .setHeader("news.date", > >>> >>>> >>>> > ns.xpath("/news:Newsletter/News/@date";, String.class)) > >>> >>>> >>>> > .setHeader("news.author", > >>> >>>> >>>> > ns.xpath("/news:Newsletter//News/@author";, > >>> >>>> >>>> > String.class)) > >>> >>>> >>>> > > >>> >>>> >>>> > .to("mock:valid"); > >>> >>>> >>>> > } > >>> >>>> >>>> > }; > >>> >>>> >>>> > } > >>> >>>> >>>> > > >>> >>>> >>>> > } > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > <?xml version="1.0" encoding="UTF-8"?> <schema > >>> >>>> >>>> > xmlns="http://www.w3.org/2001/XMLSchema"; > >>> >>>> >>>> > > >>> >>>> xmlns:news="http://www.materne.de/camel/test/xml/complex/"; > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> targetNamespace="http://www.materne.de/camel/test/xml/comple > >>> >>>> >>>> > x/"> > >>> >>>> >>>> > > >>> >>>> >>>> > <element name="Newsletter"> > >>> >>>> >>>> > <complexType> > >>> >>>> >>>> > <sequence> > >>> >>>> >>>> > <element name="News" > minOccurs="1" > >>> >>>> >>>> > maxOccurs="unbounded"> > >>> >>>> >>>> > <complexType> > >>> >>>> >>>> > > >>> >>>> >>>> > <simpleContent> > >>> >>>> >>>> > > >>> >>>> >>>> > <extension base="string"> > >>> >>>> >>>> > > >>> >>>> >>>> > <attribute name="date" type="string"/> > >>> >>>> >>>> > > >>> >>>> >>>> > <attribute name="author" type="string"/> > >>> >>>> >>>> > > >>> >>>> </extension> > >>> >>>> >>>> > > >>> </simpleContent> > >>> >>>> >>>> > </complexType> > >>> >>>> >>>> > </element> > >>> >>>> >>>> > </sequence> > >>> >>>> >>>> > <attribute name="date" type="string"/> > >>> >>>> >>>> > </complexType> > >>> >>>> >>>> > </element> > >>> >>>> >>>> > > >>> >>>> >>>> > </schema> > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > <news:Newsletter > >>> >>>> >>>> > > >>> >>>> >>>> > xmlns:news="http://www.materne.de/camel/test/xml/complex/"; > >>> >>>> >>>> > > >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema- > >>> >>>> instance" > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> > xsi:schemaLocation="http://www.materne.de/camel/test/xml/comple > >>> >>>> x/ > >>> >>>> >>>> > newsfeed.xsd " > >>> >>>> >>>> > date="2014.12.09 14:15"> > >>> >>>> >>>> > <News date="2014.12.09 14:15" author="Jan"> > >>> >>>> >>>> > Here is a demo news. > >>> >>>> >>>> > </News> > >>> >>>> >>>> > <News date="2014.12.09 14:13" author="Jan"> > >>> >>>> >>>> > Second news paragraph. > >>> >>>> >>>> > </News> > >>> >>>> >>>> > </news:Newsletter> > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> > > >>> >>>> >>>> >> -----Ursprüngliche Nachricht----- > >>> >>>> >>>> >> Von: smilevasu6 [mailto:srinivas.thu...@gmail.com] > >>> >>>> >>>> >> Gesendet: Dienstag, 9. Dezember 2014 15:12 > >>> >>>> >>>> >> An: users@camel.apache.org > >>> >>>> >>>> >> Betreff: Re: AW: camel - xsd > >>> >>>> >>>> >> > >>> >>>> >>>> >> Don't mind can you give me sample code instead of link > >>> >>>> >>>> >> > >>> >>>> >>>> >> > >>> >>>> >>>> >> > >>> >>>> >>>> >> -- > >>> >>>> >>>> >> View this message in context: > >>> >>>> >>>> >> http://camel.465427.n5.nabble.com/camel- > >>> >>>> >>>> >> xsd-tp5760376p5760396.html Sent from the Camel - Users > >>> >>>> >>>> >> mailing list archive at > >>> Nabble.com. > >>> >>>> >>>> > > >>> >>>> >>> > >>> >>> > >>
