Using EMMA for codecoverage analysis

[Vemund Ostgaard]

Hi,

I've been trying to run the Derby testsuites with EMMA to measure 
codecoverage, and after troubling a little I have a few questions and 
comments.

I hit some problems both with the old test harness and the new junit 
testsuite in relation to the securitymanager. I haven't analyzed it 
fully, but both seemed to be caused by the emma codebase wanting to 
access files on the disk and/or system properties.

I also stumbled on a test for derbyrun.jar in the old harness. It forks 
a java process running derbyrun.jar with the -jar option, so no matter 
what you set your CLASSPATH env to you still only have derbyrun.jar in 
the classpath. This test then failed when running with instrumented 
jarfiles as it couldn't find the classes it needed from emma.jar. There 
might be other tests with similar problems, this was just the first one 
I stumbled on when running the derbytools subsuite.

Ideally I think the tests should be run in their normal configuration 
(with security manager if that is the default), also when running 
codecoverage measurements. It would also be good if it was possible to 
run all the tests. My immediate thought was that this would require 
modifications to policy files and possibly individual tests like the 
test for derbyrun.jar, but there may be better ways to get around these 
issues that I haven't thought about. I know that EMMA is being used by 
others in the community, how did you get around these problems?

Is there in principle anything wrong with making changes to policy files 
and/or tests to make it easier to run codecoverage measurements with the 
EMMA tool?


Vemund