On Tue, 21 Sep 2021 14:52:26 GMT, Aleksey Shipilev <sh...@openjdk.org> wrote:
> Now that OpenJDK has more or less complete `tier{1,2,3,4}` definitions, let's
> mention them in `testing.md`.
>
> Current patch is my braindump, I am open for suggestions :)
doc/testing.html line 77:
> 75: <li><p><code>tier1</code>: This test group is the first line of defense
> against bugs. Multiple developers run these tests every day. Normally, at
> least this tier is ran before integration. Because of the widespread, the
> tests in <code>tier1</code> are carefully selected and optimized to run fast,
> and to run in the most stable manner. The test failures in <code>tier1</code>
> are usually followed up on quickly, either with fixes, or adding relevant
> tests to problem list. <a href="../.github/workflows/">GitHub Actions
> workflows</a>, if enabled, run <code>tier1</code> tests.</p></li>
> 76: <li><p><code>tier2</code>: This test group covers even more ground. These
> contain, among other things, tests that either run for too long to be at
> <code>tier1</code>, test unstable and/or experimental features, test less
> essential JDK components.</p></li>
> 77: <li><p><code>tier3</code>: This test group covers more stressful tests,
> the tests for corner cases not covered by previous tiers, plus the tests that
> require GUIs. As such, this suite should either be run with low concurrency
> (<code>TEST_JOBS=1</code>), or without headful tests
> (<code>JTREG_KEYWORDS=\!headful</code>), or both.</p></li>
It's not clear to me that these descriptions are useful. Instead of "First line
of defense against bugs" then it might be better to say something about the
hotspot and core library tests that are typically run in tier1. Folks
contributing to some areas of the core, I/O, and networking libraries will need
to learn about tier2 as that is where many of the tests for these APIs and
implementations are. So I wouldn't use words like "unstable", "experimental",
"less essential" but maybe say that it includes tests that may require some
configuration, like changing firewall rules to allow multicast or other traffic.
-------------
PR: https://git.openjdk.java.net/jdk/pull/5615
