On Tue, 2005-10-11 at 15:15 -0400, Daniel Veillard wrote: > I will try to take an example putting this in parallel. If you were testing > the Mozilla rendering on a set of complex HTML pages, would you consider > failure to match an exact rendering at the pixel level for those pages > to be wrong. Obviously no! This is the same for libxml2 processing on some > complex processin/transformations, it will usually give exactly the same > output on two runs in different environemnt, but failure to do so doesn't > mean it's broken.
I understand that and agree with you but one thing which still bothers me is when I run these regression test from 2.20 against code base of 2.22 and see around 51 test errors. Theoretically I should not see errors if ABI behavior is constant and running in same environment. Am I correct or missing something? Why I am looking from that point of view is if I include these test cases into LSB runtime for conformance, and let say spec is based on 2.22 and distro is running 2.24/2.25.. test cases which is confirming the specs should not fail in same environment. BTW the error messages I get when I run 2.20 against 2.22 code base is: ## XML regression tests File ./test/xhtml1 generated an error ## XML regression tests on memory Result for ./test/xhtml1 failed File ./test/xhtml1 generated an error ## XML entity subst regression tests File ./test/xhtml1 generated an error ## XML Namespaces regression tests ## Error cases regression tests ## Error cases stream regression tests ## Reader regression tests ## Reader entities substitution regression tests ## Reader on memory regression tests ## Walker regression tests ## SAX1 callbacks regression tests Got a difference for ./test/ent2 File ./test/ent2 generated an error Got a difference for ./test/ent7 File ./test/ent7 generated an error Got a difference for ./test/xml2 File ./test/xml2 generated an error ## SAX2 callbacks regression tests Got a difference for ./test/ent2 File ./test/ent2 generated an error Got a difference for ./test/ent7 File ./test/ent7 generated an error Got a difference for ./test/xml2 File ./test/xml2 generated an error ## XML push regression tests Result for ./test/xhtml1 failed File ./test/xhtml1 generated an error ## HTML regression tests Result for ./test/HTML/Down.html failed File ./test/HTML/Down.html generated an error Result for ./test/HTML/attrents.html failed File ./test/HTML/attrents.html generated an error Result for ./test/HTML/cf_128.html failed File ./test/HTML/cf_128.html generated an error Result for ./test/HTML/doc2.htm failed File ./test/HTML/doc2.htm generated an error Result for ./test/HTML/doc3.htm failed File ./test/HTML/doc3.htm generated an error Result for ./test/HTML/fp40.htm failed File ./test/HTML/fp40.htm generated an error Result for ./test/HTML/liclose.html failed File ./test/HTML/liclose.html generated an error Result for ./test/HTML/pre.html failed File ./test/HTML/pre.html generated an error Result for ./test/HTML/python.html failed File ./test/HTML/python.html generated an error Result for ./test/HTML/test2.html failed File ./test/HTML/test2.html generated an error Result for ./test/HTML/test3.html failed File ./test/HTML/test3.html generated an error Result for ./test/HTML/wired.html failed File ./test/HTML/wired.html generated an error ## Push HTML regression tests Result for ./test/HTML/Down.html failed File ./test/HTML/Down.html generated an error Result for ./test/HTML/attrents.html failed File ./test/HTML/attrents.html generated an error Result for ./test/HTML/cf_128.html failed File ./test/HTML/cf_128.html generated an error Result for ./test/HTML/doc2.htm failed File ./test/HTML/doc2.htm generated an error Result for ./test/HTML/doc3.htm failed File ./test/HTML/doc3.htm generated an error Result for ./test/HTML/fp40.htm failed File ./test/HTML/fp40.htm generated an error Result for ./test/HTML/liclose.html failed File ./test/HTML/liclose.html generated an error Result for ./test/HTML/pre.html failed File ./test/HTML/pre.html generated an error Result for ./test/HTML/python.html failed File ./test/HTML/python.html generated an error Result for ./test/HTML/test2.html failed File ./test/HTML/test2.html generated an error Result for ./test/HTML/test3.html failed File ./test/HTML/test3.html generated an error Result for ./test/HTML/wired.html failed File ./test/HTML/wired.html generated an error ## HTML SAX regression tests Got a difference for ./test/HTML/Down.html File ./test/HTML/Down.html generated an error Got a difference for ./test/HTML/attrents.html File ./test/HTML/attrents.html generated an error Got a difference for ./test/HTML/cf_128.html File ./test/HTML/cf_128.html generated an error Got a difference for ./test/HTML/doc2.htm File ./test/HTML/doc2.htm generated an error Got a difference for ./test/HTML/doc3.htm File ./test/HTML/doc3.htm generated an error Got a difference for ./test/HTML/fp40.htm File ./test/HTML/fp40.htm generated an error Got a difference for ./test/HTML/liclose.html File ./test/HTML/liclose.html generated an error Got a difference for ./test/HTML/pre.html File ./test/HTML/pre.html generated an error Got a difference for ./test/HTML/python.html File ./test/HTML/python.html generated an error Got a difference for ./test/HTML/reg1.html File ./test/HTML/reg1.html generated an error Got a difference for ./test/HTML/reg2.html File ./test/HTML/reg2.html generated an error Got a difference for ./test/HTML/reg3.html File ./test/HTML/reg3.html generated an error Got a difference for ./test/HTML/reg4.html File ./test/HTML/reg4.html generated an error Got a difference for ./test/HTML/script.html File ./test/HTML/script.html generated an error Got a difference for ./test/HTML/test2.html File ./test/HTML/test2.html generated an error Got a difference for ./test/HTML/test3.html File ./test/HTML/test3.html generated an error Got a difference for ./test/HTML/wired.html File ./test/HTML/wired.html generated an error ## Valid documents regression tests ## Validity checking regression tests ## General documents valid regression tests ## XInclude regression tests ## XInclude xmlReader regression tests ## XInclude regression tests stripping include nodes ## XInclude xmlReader regression tests stripping include nodes ## XPath expressions regression tests ## XPath document queries regression tests ## XPointer document queries regression tests ## xml:id regression tests ## URI parsing tests ## URI base composition tests ## Schemas regression tests Error for ./test/schemas/any3_0.xml on ./test/schemas/any3_0.xsd failed Error for ./test/schemas/bug303566_1.xml on ./test/schemas/bug303566_1.xsd failed Error for ./test/schemas/changelog093_0.xml on ./test/schemas/changelog093_1.xsd failed Result for ./test/schemas/derivation-ok-extension_0.xml on ./test/schemas/deriva tion-ok-extension_0.xsd failed Error for ./test/schemas/derivation-ok-extension_0.xml on ./test/schemas/derivat ion-ok-extension_0.xsd failed ## Relax-NG regression tests ## Relax-NG streaming regression tests ## Pattern regression tests ## C14N with comments regression tests ## C14N without comments regression tests ## C14N exclusive without comments regression tests ## Catalog and Threads regression tests Total 2557 tests, 51 errors, 0 leaks I also see one error message while running 2.22 code base against 2.22 code. ## Schemas regression tests Error for ./test/schemas/derivation-ok-extension_0.xml on ./test/schemas/derivation-ok-extension_0.xsd failed Thanks, Nilesh _______________________________________________ xml mailing list, project page http://xmlsoft.org/ [email protected] http://mail.gnome.org/mailman/listinfo/xml
