2009/11/4 Kelly O'Hair <kelly.oh...@sun.com>: > > I need some formal reviewers on these jdk/test/Makefile changes. > > More polish and changes may need to be done later, but there > is value in what I have now, and I need lots of help to improve > things (and fix some of our testcases). > > Here is the bugid and webrev: > > 6888927: Fix jdk jtreg tests to indicate which ones need othervm, allow for > use of samevm option > http://cr.openjdk.java.net/~ohair/openjdk7/jdk7-build-samevm-6888927/webrev/ > > The goal was to see if we could easily run most of the regression > tests in the jdk/test directory, in minimum time, and without so > much noise (e.g. do not run unstable or problematic tests). > This could then be used as a benchmark to validate some quality measure > of the jdk7 that was built. > > The primary changes include: > * New jdk/test/Makefile targets: jdk_all jdk_lang jdk_util jdk_nio ... > * Addition of a jdk/test/ProblemList file to be used by jdk/test/Makefile > > I tried to group all the tests (by directory names) into jdk_* sets, then > tried to balance them as to how long each set ran and if they could be run > with the jtreg -samevm option. The balancing was tricky, and will need more > polish, along with the virtual frame buffer idea. I also had to give up > on the awt, rmi, and swing tests until the Xvfb issues are figured out. > > I chose to not actually modify the tests themselves if they needed fixing, > it was just too overwhelming. So developers and teams may want to browse the > ProblemList for their favorite tests (which aren't my favorites ;^) and > consider what they might want to do. > > With this new jdk/test/Makefile, anybody can: > > cd jdk/test > gnumake [PRODUCT_HOME=${YOUR_JDK7_HOME}] [JT_HOME=${YOUR_JTREG_HOME}] > jdk_all > > And run about 3,000+ tests that SHOULD PASS in roughly 2 hours if your > machine isn't too slow and old. I was able to run it in 90 minutes on a > monster OpenSolaris AMD machine which had 16Gb RAM and 16 2.2Ghz cpus. > > Then I added "-j 4" to the gnumake command line and it only took 30mins! > So the various jdk_* make targets can be run in parallel. > > Please let me know what you think. > > -kto > > >
I like this idea, the time to run a jdk jtreg test has put me off in the past. BTW, jtreg is the one thing I wouldn't mind having in the tree as it's not something people are likely to have installed. IcedTea has a cut down copy and that means tests get run. I'm worried about the use of the IcedTea7 test list. These were added just because they were the ones that failed under my testing and that's a while ago. Just as may happen with your ProblemList, this hasn't been maintained recently so I'd be wary about including the list verbatim. With this patch in, IcedTea7 can just use the list provided here. We don't use test/Makefile at present at all, and it would be nice to do so. I need to look at the issues with that. -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8
