>I'm about to start filing bugs against the [conformance test] suite,
>but there's no category for it.

I think the ideal global solution is for OASIS to run a Bugzilla for
this, since it would allow filing of bugs against all conformance tests,
including those not originated here. Over time, expect more commingling
of test suites. Apache already has tests that were provided to OASIS by
people who were not direct Apache contributors.

If we do it here, we should pick one "product" (XalanC, XalanJ2, or a
separate new one) to hold all such bugs. There should be separate
components for "conformance" and "acceptance" tests. A few of the
acceptance tests have different results for different Xalan flavors, but
probably not enough to warrant split reporting.

Use of Bugzilla implies that fixing the tests proceeds along a similar
workflow to code bugs. One difference is that someday, all conformance
tests should be certified as normative by the XSL Working Group. Once
that starts happening, the tests cannot be changed, lest we be thought
to disrespect the WG's normativity.

I am CCing Jeff Kenton, Chair of the OASIS TC on XSLT/XPath Conformance
testing, so that he can put this on the agenda for the next meeting of
the committee. I am of the opinion that reports of incorrect tests
should be funneled into a forum or tracking system that addresses all
XSLT implementors.
.................David Marston

Reply via email to