xiangfu0 opened a new pull request, #19885:
URL: https://github.com/apache/pinot/pull/19885
## Problem
"Pinot Integration Test Set 2" fails intermittently in the *Build Project*
step on PRs that touch no build file (seen on #18709 at f1e1b36578, run
38719560945 attempt 1; the rerun passed):
```
[INFO] java.nio.file.NoSuchFileException:
.../pinot-plugins/assembly-descriptor/target/it/simple-assembly/target/simple-assembly-0.0.1-SNAPSHOT-plugin.zip
[INFO] simple-assembly/pom.xml .......................... FAILED
(23.57 s)
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-invoker-plugin:3.10.1:verify (its) on project
assembly-descriptor: 1 build failed.
```
Two things make this failure both likely and undiagnosable:
1. **The IT build re-downloads ~500 artifacts from Central on every run.**
The invoker runs the `simple-assembly` IT against a fresh, isolated
`target/it-repo` so that it resolves `assembly-descriptor`, `pinot-spi` and
`pinot-bom` from the reactor build. But that repo starts empty, so every Maven
plugin of the IT's `package` lifecycle (compiler, resources, surefire, jar,
assembly) and all of their transitive dependencies are downloaded again into it
(locally: 492 downloads, 78 MB, ~40 s). Any transient download error fails the
IT. The outer build already has all but two of those files in its own local
repository.
2. **The console never shows the real error.** `maven-invoker-plugin` runs
the post-build hook (`verify.groovy`) in a `finally` block, so when the IT
build itself exits non-zero, the hook's `NoSuchFileException` for the missing
zip replaces the invoker's own "The build exited with code N. See
.../build.log" message. The real error only exists in
`target/it/simple-assembly/build.log` on the runner.
Reactor ordering is not the cause: `assembly-descriptor` inherits the
`provided` dependency on `pinot-spi` from `pinot-plugins`, `pinot-bom` is an
upstream of the root pom through the BOM import, and the failing job's log
shows `invoker:install` copying `pinot-spi/target/pinot-spi-1.6.0-SNAPSHOT.jar`
straight from the reactor after `pinot-spi` had been installed. `./mvnw -pl
pinot-plugins/assembly-descriptor -am validate` lists `Pinot BOM`, `Pinot`,
`Pinot Service Provider Interface`, `Pinot Plugins` ahead of this module.
## Fix
- `assembly-descriptor/pom.xml`: pass
`-Dmaven.repo.local.tail=${settings.localRepository}` to the IT build. Maven
3.9+ then reads everything missing from `it-repo` out of the outer build's
local repository (read-only) before going to Central. The Pinot artifacts
installed into `it-repo` still take precedence, so the isolation that the IT
relies on is unchanged. Older Maven versions ignore the property and keep
today's behavior.
- `simple-assembly/pom.xml`: pin the lifecycle plugins (resources, jar,
surefire, assembly) to the versions of the outer Pinot build via the existing
`@property@` filtering (compiler was already pinned), so they are found in the
tail. This also moves the IT from `maven-assembly-plugin` 3.7.1 to the version
Pinot's own plugin modules use.
- `verify.groovy`: when the zip is missing, fail with the last 100 lines of
the IT's `build.log` in the message, so the console shows the real error next
time.
## Verification
- `rm -rf pinot-plugins/assembly-descriptor/target && ./mvnw -T 1C -pl
pinot-plugins/assembly-descriptor -am verify -DskipTests` (Maven 3.10.0 via the
wrapper, the same version the `ubuntu-latest` runner ships):
`simple-assembly/pom.xml ... SUCCESS (5.2 s)`, previously 41.9 s on the same
machine. The IT's `build.log` shows 2 downloads (`picocli` 4.6.3, a
`maven-assembly-plugin` dependency that was not in the local cache) instead of
492, and `it-repo` holds 389 files instead of 2290. The IT runs
`assembly:3.8.0:single`.
- Appending garbage to the IT's `Simple.java` and running the invoker now
prints the `COMPILATION ERROR` block and the `Failed to execute goal ...
compile` line under `Building: simple-assembly/pom.xml` in the outer console,
where before only the `NoSuchFileException` appeared.
- `spotless:apply`, `license:format`, `checkstyle:check` and `license:check`
pass for the module with no changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
