Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi John, On Sun, Aug 30, 2020 at 7:51 PM John Morris wrote: > Is it the case that none of the XML files in your latest test-cases has as > many as 40 lines of XML constructs? (The significance of the number 40 for > me is that it was the first line at which I observed the erratic behaviour > that I reported. I suspect that, if I had truncated my XML files to have > fewer than 40 lines, I would not have observed any problems. Or have I > missed the point that you were making?) > I've just tested with my own XML and XSD documents (having more than 40 lines within my XML input documents), and I don't see with my test cases the sorts of bugs which you've found. I've attached my test cases to the Jira issue XERCESJ-1726. I'd suggest that, let's continue this discussion on the Jira issue itself. -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
HiThanks for your latest email.Is it the case that none of the XML files in your latest test-cases has as many as 40 lines of XML constructs? (The significance of the number 40 for me is that it was the first line at which I observed the erratic behaviour that I reported. I suspect that, if I had truncated my XML files to have fewer than 40 lines, I would not have observed any problems. Or have I missed the point that you were making?)Best regards,John.Sent from my Samsung device Original message From: Mukul Gandhi Date: 30/08/2020 12:44 (GMT+00:00) To: j-users@xerces.apache.org Subject: Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text I just played around, with this kind of XSD validation, but using somewhat simple XSD and XML input documents. Below are mentioned my XSD and XML input documents,XSD document,http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified"> XML input document (which is found as valid with above mentioned schema document), a b a0 c a5When I remove comments from above mentioned XML input document, I still see the XML input document as valid.The following XML input documents are found as invalid, a b a0 c a5 a b a0 c a5Therefore, fundamentally I don't see a bug with Xerces-J with the kind of XSD and XML input documents discussed within this thread & the Jira issue (those that are provided by John). -- Regards,Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
I just played around, with this kind of XSD validation, but using somewhat simple XSD and XML input documents. Below are mentioned my XSD and XML input documents, XSD document, http://www.w3.org/2001/XMLSchema; elementFormDefault="qualified"> XML input document (which is found as valid with above mentioned schema document), a b a0 c a5 When I remove comments from above mentioned XML input document, I still see the XML input document as valid. The following XML input documents are found as invalid, a b a0 c a5 a b a0 c a5 Therefore, fundamentally I don't see a bug with Xerces-J with the kind of XSD and XML input documents discussed within this thread & the Jira issue (those that are provided by John). -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi John, On Fri, Aug 28, 2020 at 1:47 PM JOHN Morris wrote: > I'm not sure what was happening before to give us those discrepancies > between our respective results though. Did you try comparing the files you > were using for the validations before with the new (but, from my point of > view, the same) files from the Jira case? > I've not been able to do, detailed comparison between the files posted by you within this mail thread and those attached on the Jira issue. I'd assume that they're same as claimed by you. But fetching your files from the Jira issue attachment, gives me same results that you've got (and I don't get same results by copying file contents from within the mail thread). For the purpose of making a possible fix to implementation as per the issue that we're discussing here, I'd suggest that we use the file attachment that you've provided within the Jira issue. I also suggest that, if you may reduce as much as possible the XML input file contents and also possibly remove most of the comments from the XSD document (and reattach the modified files on the Jira issue) that would be still ok for validation and replicates the same bug, I guess that'd be more easy for any Xerces-J developer to investigate the issue & possibly make a fix. > What can I expect to happen next? Is there some process where one of the > Xerces-J Developers (assuming that you are not already one of those) will > pick up this case? If so, how long does that process typically take, in > your experience? > As per my knowledge, the issue would be picked up (for analysis & a possible fix) by any Xerces-J developer depending who has available time (the Xerces-J developers are volunteers). > I wonder whether it would help my case, if you could add your supporting > information, that you had succeeded in replicating the problem, to the Jira > case itself? > I've made few comments on the Xerces-J Jira issue you've posted, some time ago. -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi John, On Wed, Aug 26, 2020 at 7:47 PM JOHN Morris wrote: Also, I wonder whether I could get you to send me the exact outputs from > your jaxp.SourceValidator runs on the two files test1.xml (9 errors) and > test2.xml (18 errors). I just saw that, you've created a Jira issue XERCESJ-1726 for this discussion. Thanks. I'm now able to reproduce the issues described by you within this discussion. I've now used, the XML documents (test1.xml & test2.xml) and an XSD document (test.xsd) that you've attached to the above mentioned Jira issue. With those files, the XML document test1.xml gives me 9 XSD validation errors, and the file test2.xml gives me no XSD validation errors. When I remove the XML comment from line 40 of the file test2.xml, I get 9 XSD validation errors for the file test2.xml after removal of XML comment. This looks to me strange, and may be a bug with Xerces-J and should be investigated I believe. > Is there a command-line option for jaxp.SourceValidator which would > produce additional lines in the output for the original XML from each line > in the input file, whether valid or not? > I guess that, there's no such command line option with jaxp.SourceValidator. Inherently, after validation using jaxp.SourceValidator, either we see line and column numbers with error descriptions for the instances that are involved in invalidity assessment for the XML and XSD documents, or if the XML document is valid the output displays nothing (except the time it took for the XSD validation to take place). -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi Mukul, Thanks for your reply. As a beginner, I wonder how I would go about "creating a Xerces-J jira issue" on my problem. Also, I wonder whether I could get you to send me the exact outputs from your jaxp.SourceValidator runs on the two files test1.xml (9 errors) and test2.xml (18 errors). Is there a command-line option for jaxp.SourceValidator which would produce additional lines in the output for the original XML from each line in the input file, whether valid or not? Best regards, John. > On 26 August 2020 at 05:19 Mukul Gandhi wrote: > > > Hi John, > > On Sun, Aug 23, 2020 at 7:23 PM JOHN Morris > wrote: > > > Where can we go from here? I can only think of asking you to double check > > that both of your test runs are not on the same XML data somehow. > > I've just double checked, and I get XSD validation errors on both of your XML > input documents when validated by the XSD document that you've provided. > > I get the count of XSD validation errors as 9 and 18, respectively for the > XML input documents test1.xml and test2.xml when both of these XML input > documents are validated by the XSD document test.xsd. > > It seems strange to me that, you don't get any XSD validation errors with > Xerces-J for the XML input document test2.xml. > > > ... or do you have any other further suggested steps in order to achieve > > consistency from our two sets of results? > > May be if someone else, could also cross check my findings, that would be > further helpful to this discussion. > > If you're still grappling with this issue, I could also suggest that you can > create a Xerces-J jira issue for this and attach your XML and XSD documents > to the jira issue. That shall help anyone else, to use your test case XML and > XSD documents and do an analysis. > > > -- > > > Regards, > Mukul Gandhi - To unsubscribe, e-mail: j-users-unsubscr...@xerces.apache.org For additional commands, e-mail: j-users-h...@xerces.apache.org
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi John, On Sun, Aug 23, 2020 at 7:23 PM JOHN Morris wrote: > Where can we go from here? I can only think of asking you to double check > that both of your test runs are not on the same XML data somehow. > I've just double checked, and I get XSD validation errors on both of your XML input documents when validated by the XSD document that you've provided. I get the count of XSD validation errors as 9 and 18, respectively for the XML input documents test1.xml and test2.xml when both of these XML input documents are validated by the XSD document test.xsd. It seems strange to me that, you don't get any XSD validation errors with Xerces-J for the XML input document test2.xml. ... or do you have any other further suggested steps in order to achieve > consistency from our two sets of results? > May be if someone else, could also cross check my findings, that would be further helpful to this discussion. If you're still grappling with this issue, I could also suggest that you can create a Xerces-J jira issue for this and attach your XML and XSD documents to the jira issue. That shall help anyone else, to use your test case XML and XSD documents and do an analysis. -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi Mukul, Thanks. I have followed the steps you suggested in using jave -Djava.endorsed.dirs=... and jaxp.SourceValidator (although I am using java version "1.8.0_261") but the validation results are unaffected. I still get 9 validation violations from test1.xml but none at all from the essentially identical test2.xml. Where can we go from here? I can only think of asking you to double check that both of your test runs are not on the same XML data somehow. (Recall that test2.xml should have those XML comment lines at lines 40, 158 and 195, displacing the previously erroneous lines to other line numbers, where they now seem (in my validation) harmless). ... or do you have any other further suggested steps in order to achieve consistency from our two sets of results? Thank you also for the clarification about the characters outside the UTF-8 encoding scheme. I should have thought about those accented letters. best regards, John. > On 23 August 2020 at 10:43 Mukul Gandhi wrote: > > > On Sun, Aug 23, 2020 at 2:13 PM JOHN Morris > wrote: > > > I think that the first steps in achieving convergence between the > > validation behaviour that you observe and mine is for me to try switching > > to jaxp.SourceValidator and to ensure that I am using the same environment > > as you. To that (second) end, I would ask you to provide a (Windows) batch > > file, embodying the environment that you use to run your tests (unless your > > tests were made using the environment in my batch file). > Following are the contents of batch file that I use, to run XSD 1.1 > validation using Xerces-J sample jaxp.SourceValidator, > > > @echo off > > java -Djava.endorsed.dirs=F:\xerces-2_12_1-xml-schema-1.1 > jaxp.SourceValidator -i %1 -a %2 -f -fx -xsd11 > > > (please replace F:\xerces-2_12_1-xml-schema-1.1 with Xerces-J's path on your > environment) > > The above batch file needs to be invoked as: xmlFile_name xsdFile_name > > I'm currently using java 1.8.0_241 from Oracle. > > > > > > > > I would also ask you which characters in my XML files were unable to be > > represented in the default UTF-8 encoding, in case they are the result > > typos that I haven't noticed. > When I specify the XML prolog as , for the XML document > that is validated, I get following errors, > > [Fatal Error] test1.xml:10:93: Invalid byte 2 of 3-byte UTF-8 sequence. > > (that is the line Copyright (C) 2005 - 2017 [joomla.fr](http://joomla.fr) et > Open Source Matters. Tous droits réservés within your XML document) > > Specifying the XML prolog as , > solves this problem for me. > > > -- > > > Regards, > Mukul Gandhi - To unsubscribe, e-mail: j-users-unsubscr...@xerces.apache.org For additional commands, e-mail: j-users-h...@xerces.apache.org
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
On Sun, Aug 23, 2020 at 2:13 PM JOHN Morris wrote: > I think that the first steps in achieving convergence between the > validation behaviour that you observe and mine is for me to try switching > to jaxp.SourceValidator and to ensure that I am using the same environment > as you. To that (second) end, I would ask you to provide a (Windows) batch > file, embodying the environment that you use to run your tests (unless your > tests were made using the environment in my batch file). > Following are the contents of batch file that I use, to run XSD 1.1 validation using Xerces-J sample jaxp.SourceValidator, *@echo off* *java -Djava.endorsed.dirs=F:\xerces-2_12_1-xml-schema-1.1 jaxp.SourceValidator -i %1 -a %2 -f -fx -xsd11* (please replace F:\xerces-2_12_1-xml-schema-1.1 with Xerces-J's path on your environment) The above batch file needs to be invoked as: xmlFile_name xsdFile_name I'm currently using java 1.8.0_241 from Oracle. > I would also ask you which characters in my XML files were unable to be > represented in the default UTF-8 encoding, in case they are the result > typos that I haven't noticed. > When I specify the XML prolog as , for the XML document that is validated, I get following errors, [Fatal Error] test1.xml:10:93: Invalid byte 2 of 3-byte UTF-8 sequence. (that is the line Copyright (C) 2005 - 2017 joomla.fr et Open Source Matters. Tous droits réservés within your XML document) Specifying the XML prolog as , solves this problem for me. -- Regards, Mukul Gandhi
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
Hi Mukul, Thanks for your further comments. I think that the first steps in achieving convergence between the validation behaviour that you observe and mine is for me to try switching to jaxp.SourceValidator and to ensure that I am using the same environment as you. To that (second) end, I would ask you to provide a (Windows) batch file, embodying the environment that you use to run your tests (unless your tests were made using the environment in my batch file). I would also ask you which characters in my XML files were unable to be represented in the default UTF-8 encoding, in case they are the result typos that I haven't noticed. Many thanks once again for your patience. Best regards, John. > On 23 August 2020 at 07:46 Mukul Gandhi wrote: > > On Sat, Aug 22, 2020 at 10:27 PM JOHN Morris > wrote: > > > > > > I note your reference to using jaxp.SourceValidator/ I'm not sure > > whether you are claiming that, if you use that, you don't see any of the > > problems that I have highlighted(?) > > > > > I didn't mean that. Its just that, my personal preference to do > > XSD validation with Xerces-J, for your examples, is to use Xerces-J sample > > jaxp.SourceValidator. > > > > > > I find that seemingly good XML constructs cause validation error > > WHEN THOSE XML CONSTRUCTS OCCUR AT CERTAIN PROBLEMATIC LINE NUMBERS e.g. > > line 44, in my test1.xml file). However, those exact same constructs DO NOT > > trigger errors if they appear at different XML line numbers. For example, > > if I insert an XML comment line before one of these constructs,formerly > > appearing at a problematic line number, so that it now appears at the > > subsequent line number, it no longer triggers an error!!! That was what the > > second test file was supposed to illustrate. > > > > > I don't experience these things. For me, both of your XML > > documents (test1.xml and test2.xml) when validated by your XSD document > > (test.xsd), produces same sort of errors. My validation attempts using > > Xerces-J, produces about 9 validation errors for both of the XML documents > > that are validated. I'm not sure, what is different at your side. > > > > > > As for using xs:pattern, my imagination was not rich enough to know > > how to achieve the same effect as I was trying to get with 'matches' but > > now with xs:pattern. Have you been able to visualise how to do that? > > > > > Let's say you wish to have written > > using xs:pattern instead of XPath 2.0 'matches' function. > > Following should be the steps to do this, > > 1) Create an XSD simple type definition > > > > > > > > 2) Write the xs:assert as following > > > > (the above suggested XSD fragments, are not tested from my side) > > Another minor point. I had to add the following XML prolog to your XML > documents that are to be validated, > > > (your XML documents contain few characters, that cannot be represented > with the default encoding UTF-8). > > > > -- > Regards, > Mukul Gandhi >
Re: Re: Possible Bug: Xerces 2.12.1 for XML Validation with XSD 1.1 Schema under Java - Plain Text
On Sat, Aug 22, 2020 at 10:27 PM JOHN Morris wrote: > I note your reference to using jaxp.SourceValidator/ I'm not sure whether > you are claiming that, if you use that, you don't see any of the problems > that I have highlighted(?) > I didn't mean that. Its just that, my personal preference to do XSD validation with Xerces-J, for your examples, is to use Xerces-J sample jaxp.SourceValidator. > I find that seemingly good XML constructs cause validation error WHEN > THOSE XML CONSTRUCTS OCCUR AT CERTAIN PROBLEMATIC LINE NUMBERS e.g. line > 44, in my test1.xml file). However, those exact same constructs DO NOT > trigger errors if they appear at different XML line numbers. For example, > if I insert an XML comment line before one of these constructs,formerly > appearing at a problematic line number, so that it now appears at the > subsequent line number, it no longer triggers an error!!! That was what the > second test file was supposed to illustrate. > I don't experience these things. For me, both of your XML documents (test1.xml and test2.xml) when validated by your XSD document (test.xsd), produces same sort of errors. My validation attempts using Xerces-J, produces about 9 validation errors for both of the XML documents that are validated. I'm not sure, what is different at your side. > As for using xs:pattern, my imagination was not rich enough to know how to > achieve the same effect as I was trying to get with 'matches' but now with > xs:pattern. Have you been able to visualise how to do that? > Let's say you wish to have written using xs:pattern instead of XPath 2.0 'matches' function. Following should be the steps to do this, 1) Create an XSD simple type definition 2) Write the xs:assert as following (the above suggested XSD fragments, are not tested from my side) Another minor point. I had to add the following XML prolog to your XML documents that are to be validated, (your XML documents contain few characters, that cannot be represented with the default encoding UTF-8). -- Regards, Mukul Gandhi
