This should help to speed up tests significantly. Currently, if we run "make 
test" with a subset of tests, JTReg would still read the entirety of test root 
to report on tests that were not run. Even with current suite of tests it gets 
expensive. If you add more tests to suite -- for example, mounting a large test 
archive like pre-generated fuzzer tests -- it starts to take a lot of time.

The reasonable default for older jtreg is `-report:executed`. My own CI -- that 
has millions of tests mounted in `test/` -- was running with 
`JTREG="OPTIONS=-report:executed"` for years now. I think that should be the 
default way to run the tests in JDK tree.

[CODETOOLS-7903323](https://bugs.openjdk.org/browse/CODETOOLS-7903323) provides 
a new reporting option, `-report:files`, which makes this whole business 
easier. This requires new jtreg.

Motivational improvements:


$ time CONF=linux-x86_64-server-release make run-test 
TEST=gc/epsilon/TestHelloWorld.java

# Default JDK tree

# Baseline
real    0m9.086s
user    0m22.857s
sys     0m4.202s

# Patched
real    0m6.406s   ; +40% faster
user    0m17.156s
sys     0m3.193s

# +100K fuzzer tests in tree

# Baseline
real    3m1.997s
user    3m16.643s
sys     0m10.490s

# Patched
real    0m8.919s   ; 20x faster
user    0m17.904s
sys     0m4.860s


Additional testing:
 - [x] Ad-hoc timing tests (see above)
 - [x] Reproducer from 
[CODETOOLS-7903331](https://bugs.openjdk.org/browse/CODETOOLS-7903331)
 - [x] Eyeballing the test results on passing tests with REPEAT_COUNT > 1
 - [x] Eyeballing the test results on intermittently failing tests with 
RETRY_COUNT > 1

-------------

Commit messages:
 - Fix

Changes: https://git.openjdk.org/jdk/pull/11824/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=11824&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8294403
  Stats: 12 lines in 3 files changed: 11 ins; 0 del; 1 mod
  Patch: https://git.openjdk.org/jdk/pull/11824.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/11824/head:pull/11824

PR: https://git.openjdk.org/jdk/pull/11824

Reply via email to