[
https://issues.apache.org/jira/browse/IGNITE-28823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov resolved IGNITE-28823.
---------------------------------------
Resolution: Fixed
[~alex_pl] [~zstan] Thanks for the review!
> Reduce duration of the "Check java code" GitHub Actions job (commit-check.yml)
> ------------------------------------------------------------------------------
>
> Key: IGNITE-28823
> URL: https://issues.apache.org/jira/browse/IGNITE-28823
> Project: Ignite
> Issue Type: Task
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> h2. Problem
> The \{{check-java}} job ("Check java code on JDK 17") in
> \{{.github/workflows/commit-check.yml}}
> is the long pole of the "Code Style, Abandoned Tests, Javadocs" workflow: it
> takes ~8.9 min
> while every other job in the workflow (.NET, ducktape) finishes in under 1
> min, so the whole
> workflow is gated by it.
> The job runs three sequential Maven invocations:
> || Step || Time || Command ||
> | codestyle + licenses | ~5.4 min | \{{mvnw test-compile
> -Pall-java,licenses,lgpl,checkstyle,examples,scala,check-licenses}} |
> | abandoned tests | ~2.4 min | \{{mvnw test
> -Pcheck-test-suites,all-java,scala}} |
> | javadocs | ~0.8 min | \{{mvnw ... install ... && mvnw initialize
> -Pjavadoc}} |
> Three inefficiencies:
> # The build is single-threaded (no \{{-T}}), although GitHub
> \{{ubuntu-latest}} runners have 4 vCPUs.
> # The "abandoned tests" step recompiles the entire reactor a second time.
> maven-compiler-plugin
> 3.15 logs _"Recompiling the module because of changed dependency"_ for 28
> modules (including
> ignite-core, 3322 source files). Because this second invocation uses a
> different profile set than
> the codestyle step, the plugin's incremental engine treats reactor
> dependencies as changed and
> cascades a full recompile. As a result ignite-core and ~30 modules are
> compiled twice per job.
> # The local Maven repository is not cached, so the codestyle step downloads
> ~343 MB
> (2074 artifacts) from Maven Central on every run.
> h2. Change
> {code:none}
> - name: Setup java
> uses: actions/setup-java@v4
> with:
> distribution: 'temurin'
> java-version: ${\{ matrix.java }}
> + cache: 'maven'
> - name: Run codestyle and licenses checks
> run: |
> - ./mvnw test-compile -P...,check-licenses -B -V
> + ./mvnw test-compile -P...,check-licenses -B -V -T 1C
> - name: Run abandoned tests checks.
> + # Reuse classes from the previous step; the differing profiles
> otherwise trigger a full reactor recompile.
> run : |
> - ./mvnw test -Pcheck-test-suites,all-java,scala -B -V
> + ./mvnw test -Pcheck-test-suites,all-java,scala -B -V
> -Dmaven.compiler.useIncrementalCompilation=false
> {code}
> * \{{cache: 'maven'}} on \{{setup-java}} — cache \{{~/.m2/repository}}
> between runs.
> * \{{-T 1C}} — parallel reactor build, applied to the codestyle step only.
> * \{{-Dmaven.compiler.useIncrementalCompilation=false}} — switches the
> compiler plugin to
> timestamp-based staleness so the "abandoned tests" step reuses classes
> produced by the codestyle
> step instead of recompiling the whole reactor.
> The "abandoned tests" step is intentionally *not* parallelized: the
> orphan-test collection writes
> to a shared file finalized by the last reactor module, so \{{-T}} would race.
> h2. Why not merge the two steps into one invocation
> Merging would pull \{{examples}}/\{{lgpl}} modules into the orphan-test check
> (today it runs only
> over \{{-Pall-java,scala}}) and could newly fail it. The chosen flag keeps
> the orphan-check scope
> identical: profile-conditional source roots (\{{java-lgpl}}) exist only in
> the \{{examples}} module,
> which is not part of the "abandoned tests" reactor.
> h2. Local verification (full reactor)
> || || before || after ||
> | step 2 recompiled modules | 30 (28 "changed dependency") | 2 (0 "changed
> dependency") |
> | step 1 under \{{-T 1C}} | n/a | 47 modules SUCCESS, 0 failures, checkstyle
> included |
> The reproduced cascade (30 modules / 28 "changed dependency") matches the CI
> logs exactly, and the
> flag eliminates it. The \{{~/.m2}} cache helps once it is populated (master
> pushes and subsequent PR
> runs); the first cold run still pays the download. Exact before/after
> wall-clock is visible on the
> PR's own CI run, since the change modifies \{{commit-check.yml}} itself.
> h2. PR
> [apache/ignite#13281|https://github.com/apache/ignite/pull/13281]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
