MartijnVisser opened a new pull request, #170:
URL: https://github.com/apache/flink-shaded/pull/170
## What is the purpose of the change
Bumps the build and enforced Maven version for `flink-shaded` from 3.8.6 to
3.9.16, matching apache/flink#29009 for FLINK-40459.
## Brief change log
- `.github/workflows/ci.yml`: the `stCarolas/setup-maven` pin goes from
3.8.6 to 3.9.16
- `pom.xml`: the `release` profile's `requireMavenVersion` pin goes from
`[3.8.6]` to `[3.9.16]` (exact pin retained)
- `.mvn/wrapper/maven-wrapper.properties`, `mvnw`, `mvnw.cmd`: add a Maven
wrapper pinning Apache Maven 3.9.16 (maven-wrapper 3.3.4), matching Flink's
- `.gitignore`: ignore the downloaded `maven-wrapper.jar`
- `README.md`: document the required Maven version and the wrapper
- `.github/workflows/ci.yml`: normalize Maven 3.9 mojo banners before the
license check (see below)
The wrapper is added **script-only** — `maven-wrapper.jar` is deliberately
not checked in. `mvnw` downloads it on first use and verifies it against
`wrapperSha256Sum`, so no binary artifact enters the ASF source release.
Verified: a simulated `create_source_release.sh` tarball contains only the
three text files and zero binaries, and `apache-rat` reports 0 unapproved / 0
binaries. A previous attempt to add the wrapper here (0a4baad) was reverted 18
minutes later (8f9e83c); that one checked in the jar and shipped a
maven-wrapper 3.1.1 script against 3.3.2 properties, so its checksums were
never actually verified.
### The license check needs a fix, or it silently stops checking
Maven 3.9 renders mojo banners using the plugin's **goal prefix**
(`deploy:2.8.2:deploy`) instead of its **artifactId**
(`maven-deploy-plugin:2.8.2:deploy`). This was confirmed by decompiling
`ExecutionEventLogger.append(MessageBuilder, MojoExecution)` from both
distributions: 3.8.6 calls `MojoExecution.getArtifactId()`, 3.9.16 calls
`MojoDescriptor.getPluginDescriptor().getGoalPrefix()`.
`flink-ci-tools`' `DeployParser` and `DependencyParser` match on the
artifactId spelling, so under 3.9.16 they match nothing. `NoticeFileChecker`
then sees zero deployed modules, empties `modulesWithBundledDependencies`, and
every declared NOTICE line degrades to "not bundled, but listed" at `TOLERATED`
severity — so `severeIssueCount` stays 0 and **the build still exits 0**. The
gate does not fail; it stops checking. (`ShadeParser` survives, because its
regex starts with `.*`. `JarFileChecker` scans the deploy directory rather than
the log and is unaffected.)
Measured on this reactor:
| Build | exit code | `Extracted N modules that were deployed` |
|---|---|---|
| 3.8.6 | 0 | **17** |
| 3.9.16, unremediated | 0 | **0** |
| 3.9.16 + this PR's normalization | 0 | **17** |
No released `flink-ci-tools` fixes this — I checked the 1.18.0, 1.20.4 and
2.2.1 jars from Maven Central and all three carry byte-identical broken
patterns; the fix exists only on the unmerged apache/flink#29009 branch. So CI
rewrites the two affected banners back to the 3.8 spelling before invoking the
checker. Only `deploy:` and `dependency:` are rewritten, anchored to `^[INFO]
--- `; no other plugin in this build owns those goal prefixes, and the step is
a byte-identical no-op on 3.8.6-format logs. It should be removed once
`flink-ci-tools` is bumped to a release containing the FLINK-40459 parser fix.
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
It was validated with a local red/green differential of the full reactor
under 3.8.6 and 3.9.16, on JDK 11 (CI parity) and JDK 17, across the default,
`-Pinclude-netty-tcnative-static`, `-Plicense-check`, `-Dshade-sources` and
`-Prelease` profile combinations. Every combination compared **equivalent**:
identical deployed file sets, identical jar entry lists, identical SHA-256 per
uncompressed entry, identical aggregated `META-INF/NOTICE` and
`META-INF/DEPENDENCIES`, byte-identical `dependency-reduced-pom.xml` and
flattened poms, and an identical mojo execution sequence.
The only artifact difference anywhere is the manifest `Created-By: Apache
Maven 3.8.6` → `3.9.16` line in the nine non-Ant modules, plus the matching
`pom.properties` comment in `flink-shaded-force-shading`. The three
Ant-repackaged jars (`flink-shaded-netty`,
`flink-shaded-netty-tcnative-dynamic`, `flink-shaded-netty-tcnative-static`)
are unchanged, as expected — Ant's `<jar>` writes their manifest, so they carry
no Maven version at all.
Dependency resolution is unchanged. Two independent cold local repositories,
populated online by each Maven, are identical in all 2189 files, and
`dependency:tree` / `dependency:list` match exactly. This matters most for the
ZooKeeper modules, whose `<artifactSet><include>*</include></artifactSet>`
would put any resolution change straight into the shipped jar.
Also verified:
- `./mvnw clean verify` passes, and `./mvnw -v` downloads Apache Maven
3.9.16 with the wrapper's own SHA-256 verification
- negative check: with a system Maven 3.8.6, `mvn -N validate -Prelease`
now fails with `Detected Maven Version: 3.8.6 is not in the allowed range
[3.9.16,3.9.16]`, and with 3.9.16 it passes
- before this PR's pin change, `-Prelease` on 3.9.16 failed with `Detected
Maven Version: 3.9.16 is not in the allowed range [3.8.6,3.8.6]` and no other
error
- the old plugin set still loads and executes under 3.9.16 with no linkage
errors (enforcer 1.4.1, apache-rat 0.11, antrun 1.7, dependency 3.1.1, gpg
1.4), including negative controls confirming enforcer still detects an injected
dependency-convergence conflict and RAT still detects an unlicensed file.
MNG-6965 (plexus-utils no longer auto-injected) is a non-event here: every
plugin in the effective set declares plexus-utils itself
- a runtime probe over the produced artifacts on JDK 11 and 17 behaves
identically for both Maven versions: relocated classes resolve, no un-relocated
leaks, and the Jackson multi-release classes are still selected from
`META-INF/versions/`
- downstream, `flink-rpc/flink-rpc-akka` built against each artifact set
with the *same* Maven produces uber-jars identical in all 12323 entries
One benign difference shows up in `help:effective-pom` and is not caused by
this change: Maven 3.9's super-POM drops
`<releases><updatePolicy>never</updatePolicy>` from the built-in `central`
**pluginRepository**. Nothing in this repository sets `updatePolicy`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? The README is updated with the
required Maven version and the wrapper
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
