Anton Vinogradov created IGNITE-28823:
-----------------------------------------
Summary: Reduce duration of the "Check java code" GitHub Actions
job (commit-check.yml)
Key: IGNITE-28823
URL: https://issues.apache.org/jira/browse/IGNITE-28823
Project: Ignite
Issue Type: Task
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov
h2. Problem
The \{{check-java}} job ("Check java code on JDK 17") in
\{{.github/workflows/commit-check.yml}}
is the long pole of the "Code Style, Abandoned Tests, Javadocs" workflow: it
takes ~8.9 min
while every other job in the workflow (.NET, ducktape) finishes in under 1 min,
so the whole
workflow is gated by it.
The job runs three sequential Maven invocations:
|| Step || Time || Command ||
| codestyle + licenses | ~5.4 min | \{{mvnw test-compile
-Pall-java,licenses,lgpl,checkstyle,examples,scala,check-licenses}} |
| abandoned tests | ~2.4 min | \{{mvnw test
-Pcheck-test-suites,all-java,scala}} |
| javadocs | ~0.8 min | \{{mvnw ... install ... && mvnw initialize -Pjavadoc}} |
Two inefficiencies:
# The build is single-threaded (no \{{-T}}), although GitHub \{{ubuntu-latest}}
runners have 4 vCPUs.
# The "abandoned tests" step recompiles the entire reactor a second time.
maven-compiler-plugin
3.15 logs _"Recompiling the module because of changed dependency"_ for 28
modules (including
ignite-core, 3322 source files). Because this second invocation uses a
different profile set than
the codestyle step, the plugin's incremental engine treats reactor dependencies
as changed and
cascades a full recompile. As a result ignite-core and ~30 modules are compiled
twice per job.
h2. Proposed change
{code:none}
- name: Run codestyle and licenses checks
run: |
- ./mvnw test-compile -P...,check-licenses -B -V
+ ./mvnw test-compile -P...,check-licenses -B -V -T 1C
- name: Run abandoned tests checks.
+ # Reuse classes from the previous step; the differing profiles
otherwise trigger a full reactor recompile.
run : |
- ./mvnw test -Pcheck-test-suites,all-java,scala -B -V
+ ./mvnw test -Pcheck-test-suites,all-java,scala -B -V
-Dmaven.compiler.useIncrementalCompilation=false
{code}
* \{{-T 1C}} — parallel reactor build, applied to the codestyle step only.
* \{{-Dmaven.compiler.useIncrementalCompilation=false}} — switches the compiler
plugin to
timestamp-based staleness so the "abandoned tests" step reuses classes produced
by the codestyle
step instead of recompiling the whole reactor.
The "abandoned tests" step is intentionally *not* parallelized: the orphan-test
collection writes
to a shared file finalized by the last reactor module, so \{{-T}} would race.
h2. Why not merge the two steps into one invocation
Merging would pull \{{examples}}/\{{lgpl}} modules into the orphan-test check
(today it runs only
over \{{-Pall-java,scala}}) and could newly fail it. The chosen flag keeps the
orphan-check scope
identical: profile-conditional source roots (\{{java-lgpl}}) exist only in the
\{{examples}} module,
which is not part of the "abandoned tests" reactor.
h2. Local verification (full reactor)
|| || before || after ||
| step 2 recompiled modules | 30 (28 "changed dependency") | 2 (0 "changed
dependency") |
| step 1 under \{{-T 1C}} | n/a | 47 modules SUCCESS, 0 failures, checkstyle
included |
The reproduced cascade (30 modules / 28 "changed dependency") matches the CI
logs exactly, and the
flag eliminates it. Exact before/after wall-clock will be visible on this
issue's own PR run, since
the change modifies \{{commit-check.yml}}.
h2. Out of scope
Caching \{{~/.m2/repository}} (e.g. via \{{actions/setup-java}} \{{cache:
maven}}) would further cut
dependency resolution in both steps — left for a separate change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
