David Van Couvering wrote: > I think this is a laudable goal. I agree it would need to be done > incrementally. I think it would really help to have a Wiki page where > work around this can be clarified and coordinated. Probably a JIRA to > hang subtasks off of too. > > Here are some questions I have around this > > - does moving to a JUnit harness imply removing canon-based testing and > only using assertion-based testing? Or is that a separate, orthogonal > goal (one I which I think is important, both for master file management > overhead as well as improving our ability to truly test backward > compatibility without having to sift through unhelpful diffs).
Orthogonal goal (and I agree a good one). It's possible to write a Junit test that uses sql scripts and a canon file, see my early experiments with JUnit. May not be the best approach, but it is possible, and would probably work for a large number of the existing tests. The downside with moving away from canon based test is who is going to re-write all the tests as assert based and ensure no cases were lost. As more required functionality is moved into the Junit framework, I'm assuming tests will move over, I see a possible staging something like: 0 (today) - all tests (incl JUnit) run as single tests in derby harness 1 - derby harness runs its tests and multiple Junit suites 2 - derby harness runs its tests and a single Junit suite 3 - some canon based tests moved to JUnit tests (using canons) 4 - majority of tests run as Junit tests in single suite from derbyall 5 - derby harness killed, all tests Junit > - Where does the responsibility lie to manage different results in > different environments? Not sure what you are asking here. If a test passes in one environment and fails in the other, I don't see how which harness is being used affects who is responsible (ie. those with the itch to see clean runs in that environment). Any complete move to JUnit would have to ensure we do not regress the community's ability to run tests where they can be run today (ie. the do no harm mantra). J2ME/CDC/Foundation is one concern here, but I'm assuming the current JUnit tests are running on J2ME. Thanks, Dan.
