webrev:
http://cr.openjdk.java.net/~dholmes/8024140/webrev/
Similar to what was recently done on the hotspot side, this introduces
profile-based test groups for the jdk regression tests.
The primary groups are:
- jdk
- jre
- compact3
- compact2
- compact2_minimal
- compact1
- compact1_minimal
The minimal VM is only supported on compact1 and compact2.
The groups are defined hierarchically in two forms:
- The need_xxx groups list all the tests that have a dependency on a
specific profile. This is either because it tests a feature in that
profile, or the test infrastructure uses a feature in that profile.
- The primary groups are defined in terms of the other primary groups
combined with the needs_xxx groups (including and excluding them as
appropriate). For example the jre can run all tests from compact3, plus
those from needs_jre, but excluding those from need_jdk.
The bottom group defines all the actual tests to be considered, simply
by listing the top-level test directories, and then excluding all the
needs_xxx groups.
To select a group of tests you use :<group-name>
Eg to run only those tests that can run on compact1 use:
jtreg :compact1
Of course you still need to point jtreg at the right kind of runtime
image (and give it a full JDK as the compile-jdk!); and if testing the
minimal VM you need to tell jtreg to select it using -javaoptions:-minimal
These top-level groups are not as useful standalone as they are for
hotspot due to the fact that there are so many regression tests that are
problematic in one way or another. In most cases what is desired is a
way to run sub-set of tests for a given profile eg all jdk_core tests
that can run on compact1. At present this would require a new group to
be defined for each permutation and that is not scalable. I hope that in
the near future jtreg will allow you to implicitly define such ad-hoc
groups via the command-line, simply by listing groups to include and
groups to exclude.
The full jtreg group facility is only available in the most recent jtreg
builds, so you will need to grab the latest nightly build, or latest
sources.
Note: once this is in place, anyone writing regression tests will need
to be aware of whether that test is limited to certain profiles and
update the group file accordingly. Sometimes it is not the item being
tested that determines the minimum needed profile, but the test
infrastructure eg if it uses XML.
See TEST.groups for more information.
Note: The initial group definitions proposed here are not complete.
There are over 4000 tests of the "desktop" that can not be readily
executed manually and it may be that a number of these tests will
require a JDK rather than the full JRE. Our full test processes will
discover this and update the lists as needed. (Or we fix the tests to
not need full JDKs.)
Thanks,
David
