I got suites.All to run with EMMA, but I had to add 7-8 lines to
derby_tests.policy and modify a couple of tests before it ran cleanly.
I think it would be good if this worked without editing the source, so
I'll make a Jira and upload a patch with these changes.
I'm thinking about adding some ant tasks for instrumenting classes/jars
and running the junit-tests with EMMA, making it easier to do this.
Vemund
Vemund Ostgaard wrote:
Hi,
I've been trying to run the Derby testsuites with EMMA to measure
codecoverage, and after troubling a little I have a few questions and
comments.
I hit some problems both with the old test harness and the new junit
testsuite in relation to the securitymanager. I haven't analyzed it
fully, but both seemed to be caused by the emma codebase wanting to
access files on the disk and/or system properties.
I also stumbled on a test for derbyrun.jar in the old harness. It
forks a java process running derbyrun.jar with the -jar option, so no
matter what you set your CLASSPATH env to you still only have
derbyrun.jar in the classpath. This test then failed when running with
instrumented jarfiles as it couldn't find the classes it needed from
emma.jar. There might be other tests with similar problems, this was
just the first one I stumbled on when running the derbytools subsuite.
Ideally I think the tests should be run in their normal configuration
(with security manager if that is the default), also when running
codecoverage measurements. It would also be good if it was possible to
run all the tests. My immediate thought was that this would require
modifications to policy files and possibly individual tests like the
test for derbyrun.jar, but there may be better ways to get around
these issues that I haven't thought about. I know that EMMA is being
used by others in the community, how did you get around these problems?
Is there in principle anything wrong with making changes to policy
files and/or tests to make it easier to run codecoverage measurements
with the EMMA tool?
Vemund
