Hi Dan,
We briefly discussed this compatiblity test suite late last month on a
thread titled "client/server compatibility testing". Here's the essence
of that proposal:
o As part of the checkin barrier, derbyall should just test the
compatibility suite against a Derby client and server running at the
current level on the same jdk version.
o The full set of combinations can be run on a weekly basis by some
authority.
o The full set of combinations should be run as part of the release
barrier.
I was hoping that the first point could be accomplished by some jiggery
pokery which ran the compatibility suite under the normal (non-junit)
test harness, producing a canon. This wouldn't wrench the existing
machinery around too much. However, it would require developers to
download the junit jar so that the test class would build and
execute...and this may be controversial.
The full compatibility suite (run weekly and before releases) would be a
collection of junit test runs--for which no canons would be necessary.
It would run under its own small ant- and junit-based harness with its
own instructions. I don't regard this harness as a permanent fixture. I
am cautiously hopeful that someone will tackle the problem of migrating
our existing tests to JUnit; and when they do, my temporary harness can
be thrown away and the compatiblity suite can be integrated with the
general mechanism. Back in May, an email thread titled "JUnit" discussed
the desirability of migrating our tests to the JUnit framework over
time. You described your experiments with JUnit and seemed hopeful that
this framework would reduce the time it takes to run derbyall.
Other than requiring developers to download the junit jar, I think I am
proposing something fairly modest here. However, the camel's snout would
be in the tent. With JUnit now visible to Derby developers, momentum
might build for someone to attempt the bigger migration. At that point,
we probably would have to iron out a lot of controversial issues.
Please help me understand what issues (besides the junit jar
requirement) you see in this first, smaller proposal to expose an ant-
and junit-based compatibility test suite.
Thanks,
-Rick
Daniel John Debrunner wrote:
Rick Hillegas wrote:
Thanks Jean, David, and Francois for your suggestion that I amend
BUILDING.txt. Now I'm back to an etiquette question: I'm hoping to check
in a JUnit assertion-based test. Since the test needs JUnit to build, my
patch will break everyone's build: they will have to download the junit
jar themselves. This could annoy a lot of folks. What is the etiquette
for breaking the build this way?
I think a vote would be needed to accept the patch.
Currently Derby's testing policy is its own harness, you seem to be
suggesting some change, but it's not clear what.
What's the expectation here, that these Junit tests need to be run
before any commit, in addition to derbyall? Or that they are separate
tests run by those that care to, like the current upgrade tests?
If they are required to be run, then are you providing a single command
that executes both derbyall and the junit tests?
Dan.
