I've implemented a new BugzillaTestletDriver that automates running any
Bugzilla*.java Testlet class or any Bugzilla*.xsl stylesheet that's checked
into tests/bugzilla. I hope (with some documentation) to get to the point
where most Bugzilla bugs will also have a simple 'regression' test that
gets checked in that we can automate.
cd xml-xalan/test
build all bugzilla
This should build all (which compiles all the normal tests, plus compiles
*.java in tests/bugzilla), and will then run the BugzillaTestletDriver over
this directory.
Quickly: Bugzilla tests should be short and simple reproductions of the
specific problem that a bug is reporting - the minimal amount of code to
show the problem. For some complex bugs, we may never bother to implement
a Bugzilla test for them if the code will be too big or require too much
maintenance.
-- If you need to call certain API's or want to control the process
yourself, then you create a Testlet subclass named
Bugzilla(bugnumber).java. See BugzillaNNNN.java for a cheap start. It
must extend or implement Testlet, and it must use the logger to report
results and output data - never System.err/.out!
-- If you can simply Process a .xsl stylesheet with some sort of .xml
document to show the bug (i.e. an exception is thrown or the output is
wrong), then you simply create a triplet of BugzillaNNNN.xsl/.xml/.out;
where the .out file is the 'gold' or expected output of the transformation,
if available. The BugzillaTestletDriver will simply use a
StylesheetTestlet to process and verify the output of these files.
-- The goal is that if you run a Bugzilla test now, it will fail (because
there's a bug in Xalan). When a developer fixes the bug, then the test
should pass (thus proving that they fixed this bug {or at least this
specific variant...}).
Now, of course, the real win is when people start creating more bugzilla
tests!
- Shane
P.S. Stay tuned for actual updated documentation (someday) and a similar
ExtensionTestletDriver and a tests/extension directory...
