nzw921rx commented on PR #13283:
URL: https://github.com/apache/seatunnel/pull/13283#issuecomment-6907642888

   > Thanks for working on this. I reviewed the latest head `f1586236a2f3`.
   > 
   > # What this PR fixes
   > - User pain: the shaded `seatunnel-starter.jar` bundles a different 
`netty-buffer`/`netty-common` depending on how it is built -- 4.1.118.Final in 
the reactor (won through the *test-scoped* `seatunnel-transforms-v2` -> AWS SDK 
2.31.30 path) versus 4.1.104.Final in a standalone build against installed 
dependency-reduced POMs (won through `seatunnel-common` -> 
`seatunnel-shade-arrow` / Arrow 15.0.1), so the starter jar is not reproducible 
across build modes.
   > - Fix approach: adds a module-local 
`<netty.version>4.1.118.Final</netty.version>` property 
(`seatunnel-core/seatunnel-starter/pom.xml:32`) and a `<dependencyManagement>` 
block (`pom.xml:36-49`) that pins only `io.netty:netty-buffer` and 
`io.netty:netty-common`; no root-POM, `seatunnel-dist`, or code change.
   > - One-line summary: the pin does make the shaded starter jar 
deterministic, but it is a hand-copied snapshot of what the AWS SDK brings 
(root `pom.xml:207`), is decoupled from both producers (root `pom.xml:208` 
Arrow 15.0.1), manages only two Netty modules, introduces a fourth differently 
named property (`netty.version`) next to the three existing 
`netty-buffer.version` pins (imap-storage-file 4.1.60, e2e-base 4.1.89, 
`seatunnel-dist/pom.xml:149` 4.1.89), and is moot in the shipped CI 
distribution where `assembly-seatunnel-bin-ci.xml:239-240` copies 
netty-buffer/netty-common into `lib/`, which `seatunnel-cluster.sh:221` 
(`CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}`) places ahead of the starter jar -- 
so Zeta still loads a different Netty than the one this PR pins, and nothing in 
the build (`tools/dependencies/known-dependencies.txt` still lists both 4.1.104 
and 4.1.118) guards the new invariant.
   > 
   > # Runtime chain I rechecked
   > ```text
   > Maven dependency mediation for the seatunnel-starter shaded jar 
(build-time only; no runtime code changed)
   >   seatunnel-core/seatunnel-starter/pom.xml:36-49  <dependencyManagement> 
io.netty:netty-buffer / netty-common -> ${netty.version}
   >     seatunnel-core/seatunnel-starter/pom.xml:32   
<netty.version>4.1.118.Final</netty.version>
   >     overrides nearest-wins resolution from:
   >       seatunnel-starter -> seatunnel-transforms-v2 (test scope, 
pom.xml:103-108) -> software.amazon.awssdk:* 
(seatunnel-transforms-v2/pom.xml:51-58; root pom.xml:207, 2.31.30) -> netty 
4.1.118.Final  [reactor winner]
   >       seatunnel-starter -> seatunnel-api/engine -> seatunnel-common 
(seatunnel-common/pom.xml:64 seatunnel-shade-arrow) -> arrow-memory-netty (root 
pom.xml:662, arrow.version 15.0.1 at root pom.xml:208) -> netty 4.1.104.Final  
[standalone/dependency-reduced-POM winner]
   >   maven-shade-plugin (seatunnel-starter/pom.xml:128-170; no io.netty 
exclude) bundles the mediated netty-buffer/netty-common into 
starter/seatunnel-starter.jar
   > 
   > Runtime classpath in the CI distribution (unchanged by this PR; what Zeta 
actually loads)
   >   seatunnel-dist/pom.xml:149   
<netty-buffer.version>4.1.89.Final</netty-buffer.version>
   >   seatunnel-dist/pom.xml:1162-1167   io.netty:netty-buffer (provided) -> 
${netty-buffer.version}
   >   seatunnel-dist/src/main/assembly/assembly-seatunnel-bin-ci.xml:239-240   
include io.netty:netty-buffer:jar, io.netty:netty-common:jar -> lib/
   >   seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh:221   
CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}  (lib/ shadows the starter jar's 
4.1.118.Final)
   >   seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-cluster.sh:39    
APP_JAR=${APP_DIR}/starter/seatunnel-starter.jar
   > ```
   > 
   > # Findings
   > **Issue 1: Hard-coded 4.1.118.Final in the starter POM is a detached copy 
of the version the AWS SDK (root `software.amazon.awssdk.version` = 2.31.30) 
happens to bring; the root POM manages no `io.netty` artifact, so the next 
SDK/Arrow bump silently produces mixed Netty module versions on the starter 
classpath**
   > - Location: `seatunnel-core/seatunnel-starter/pom.xml:32`
   > - Why it matters: - Root cause: the value `4.1.118.Final` on 
`seatunnel-core/seatunnel-starter/pom.xml:32` is not derived from anything; it 
mirrors what `software.amazon.awssdk:*:2.31.30` (root `pom.xml:207`, pulled in 
via the test-scoped `seatunnel-transforms-v2`, 
`seatunnel-transforms-v2/pom.xml:51-58`) resolves today. The root 
`<dependencyManagement>` has no `io.netty` entry at all (only 
`arrow-memory-netty`, root `pom.xml:662`), and the three other Netty pins in 
the repo are each module-local and different (`seatunnel-dist/pom.xml:149` = 
4.1.89.Final, `imap-storage-file/pom.xml:38` = 4.1.60.Final, 
`connector-seatunnel-e2e-base/pom.xml:33` = 4.1.89.Final).
   > - Impact: the management block only covers `netty-buffer` and 
`netty-common` (lines 38-47). When `software.amazon.awssdk.version` or 
`arrow.version` is bumped, `netty-transport`/`netty-handler`/`netty-codec*` on 
the starter test classpath (and any future compile-scope Netty consumer in 
engine-server/engine-client) float to the new version while buffer/common are 
forced back to 4.1.118 -- exactly the kind of cross-module Netty mismatch 
(`NoSuchMethodError`/`AbstractMethodError` inside `io.netty.buffer`) that 
surfaces at runtime, not compile time. Nothing in the root POM reveals that 
this pin exists, so whoever does the bump has no reason to look here.
   > - Fix: manage Netty once at the root instead of per module -- import 
`io.netty:netty-bom:${netty.version}` (`<type>pom</type>`, 
`<scope>import</scope>`) in the root `<dependencyManagement>` with a single 
root `netty.version` property, and have seatunnel-dist, imap-storage-file and 
e2e-base reference that property instead of their private 
`netty-buffer.version`. If this change is deliberately starter-only, at minimum 
import `netty-bom` here rather than listing two artifacts, and add a comment 
tying the value to `software.amazon.awssdk.version` so the two are bumped 
together.
   > - Evidence: `seatunnel-core/seatunnel-starter/pom.xml:40  
<artifactId>netty-buffer</artifactId>`; 
`seatunnel-core/seatunnel-starter/pom.xml:45  
<artifactId>netty-common</artifactId>`; 
`seatunnel-core/seatunnel-starter/pom.xml:35  <!-- Use the same Netty version 
in reactor and standalone builds. -->`
   > - Suggested fix: This value is a hand-copied snapshot of what 
`software.amazon.awssdk:*:${software.amazon.awssdk.version}` (2.31.30) resolves 
today; the root POM has no `io.netty` management at all, and seatunnel-dist 
(4.1.89), imap-storage-file (4.1.60) and e2e-base (4.1.89) each carry their own 
`netty-buffer.version`. Managing only `netty-buffer`/`netty-common` here means 
the next AWS SDK or Arrow bump will move 
`netty-transport`/`netty-handler`/`netty-codec*` on this module's classpath 
while pinning buffer/common back to 4.1.118 -- a mixed-version Netty that fails 
at runtime. Could we instead import `io.netty:netty-bom` with a single 
`netty.version` property in the root `<dependencyManagement>` and point the 
other three modules at it? If this must stay starter-local, please at least 
import `netty-bom` rather than two artifacts, and document that this value must 
track `software.amazon.awssdk.version`.
   > - Severity: Medium
   > 
   > **Issue 2: The module-local pin only makes the shaded starter jar 
deterministic; in the shipped distribution `lib/netty-buffer-4.1.89.Final.jar` 
(from `seatunnel-dist`) is loaded ahead of `starter/seatunnel-starter.jar`, so 
Zeta still runs a different Netty than the one this PR pins**
   > - Location: `seatunnel-core/seatunnel-starter/pom.xml:36`
   > - Why it matters: - Root cause: the new `<dependencyManagement>` block in 
`seatunnel-core/seatunnel-starter/pom.xml:36-49` is scoped to this module only. 
`seatunnel-dist/pom.xml:149` still declares 
`<netty-buffer.version>4.1.89.Final</netty-buffer.version>` (used by the 
`provided` `netty-buffer` dependency at `pom.xml:1162-1167`), and 
`seatunnel-dist/src/main/assembly/assembly-seatunnel-bin-ci.xml:239-240` copies 
`io.netty:netty-buffer:jar` and `io.netty:netty-common:jar` into `/lib`. 
`seatunnel-cluster.sh:221` builds `CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}`, so 
`lib/` wins over the shaded starter jar for every `io.netty.buffer.*` / 
`io.netty.util.*` class.
   > - Impact: for a Zeta server started from the binary distribution, the 
Netty classes that actually execute are still `netty-buffer` 4.1.89 plus 
whatever transitive `netty-common` `seatunnel-dist` resolves, not 4.1.118. The 
4.1.118 classes bundled in the starter jar are dead weight at runtime, and 
`imap-storage-file` (which compiles against `netty-buffer` 4.1.60.Final as 
`provided`, see its `pom.xml:38,108-111`, and uses 
`io.netty.buffer.ByteBuf`/`Unpooled` in `CloudWriter.java:33-34`) still runs on 
a version nobody pinned deliberately. The PR title promises consistent Netty 
resolution for the starter, but the user-visible artifact (the distribution) 
keeps the same three-way mismatch.
   > - Fix: either pin the version once in the root `pom.xml` 
`<dependencyManagement>` (so `seatunnel-starter`, `seatunnel-dist` and 
`imap-storage-file` all inherit it) and drop the per-module 
`netty-buffer.version` properties, or, if the intent is that `lib/` is the 
single source of Netty for Zeta, add `io.netty:*` to the starter's 
`maven-shade-plugin` `<artifactSet><excludes>` (`pom.xml:132-156`) so the 
starter jar does not carry a second copy at all. Please also state in the PR 
which of the two layouts is intended.
   > - Evidence: `seatunnel-core/seatunnel-starter/pom.xml:40  
<artifactId>netty-buffer</artifactId>`; 
`seatunnel-core/seatunnel-starter/pom.xml:45  
<artifactId>netty-common</artifactId>`; 
`seatunnel-core/seatunnel-starter/pom.xml:35  <!-- Use the same Netty version 
in reactor and standalone builds. -->`
   > - Suggested fix: This `<dependencyManagement>` only affects the shaded 
`seatunnel-starter.jar`. In the binary distribution 
`seatunnel-dist/pom.xml:149` still ships `netty-buffer` 4.1.89.Final (plus a 
transitive `netty-common`) into `lib/` via 
`assembly-seatunnel-bin-ci.xml:239-240`, and `seatunnel-cluster.sh` builds 
`CLASS_PATH=${APP_DIR}/lib/*:${APP_JAR}`, so `lib/` wins over the starter jar 
at runtime. The Zeta server will therefore keep running 4.1.89 regardless of 
this pin, and `imap-storage-file` (compiled against 4.1.60 `provided`) is still 
on an unpinned version. Could we move the pin to the root POM 
`<dependencyManagement>` so `seatunnel-dist`/`imap-storage-file` inherit the 
same value (and remove their local `netty-buffer.version` properties), or 
alternatively exclude `io.netty:*` from the starter shade if `lib/` is meant to 
be the single Netty source?
   > - Severity: Medium
   > 
   > **Issue 3: Pinning only `netty-buffer` and `netty-common` with a 
hard-coded version decoupled from the AWS SDK / Arrow that actually produce 
them will yield mixed Netty module versions on the next SDK or Arrow bump**
   > - Location: `seatunnel-core/seatunnel-starter/pom.xml:40`
   > - Why it matters: - Root cause: `netty.version` at 
`seatunnel-core/seatunnel-starter/pom.xml:32` is a literal `4.1.118.Final` that 
mirrors the version the AWS SDK path (`netty-nio-client-2.31.30` -> 
`netty-transport/handler/codec-4.1.118.Final`, see 
`tools/dependencies/known-dependencies.txt:109-121`) currently brings, while 
Arrow 15.0.1 (`arrow.version`, root `pom.xml:208`) brings 4.1.104 
(`known-dependencies.txt:83-84`). Nothing ties the three together, and the 
management block covers just two of the ten `io.netty` artifacts that the AWS 
SDK path places on the starter classpath.
   > - Failure scenario: when the AWS SDK or Arrow is bumped (which is how 
Netty normally moves in this repo), `netty-transport`, `netty-handler`, 
`netty-codec*`, `netty-resolver` float to the new release but 
`netty-buffer`/`netty-common` are forced back to 4.1.118. Netty modules are 
only guaranteed compatible within one release; 
`netty-transport`/`netty-handler` regularly call new 
`netty-common`/`netty-buffer` methods, so the result is 
`NoSuchMethodError`/`NoClassDefFoundError` at runtime in whatever path 
exercises Netty (today the starter test classpath via 
`seatunnel-transforms-v2`, tomorrow any compile-scope Netty user). Nothing in 
the build fails fast on this, so it surfaces as a confusing runtime error 
rather than a build error.
   > - Fix: import `io.netty:netty-bom:${netty.version}` with 
`<type>pom</type><scope>import</scope>` instead of listing two artifacts, so 
every `io.netty:*` artifact resolves to the same release, and keep 
`netty.version` next to `arrow.version`/`software.amazon.awssdk.version` in the 
root POM with a comment stating which upstream it must track.
   > - Evidence: `seatunnel-core/seatunnel-starter/pom.xml:41  
<version>${netty.version}</version>`; 
`seatunnel-core/seatunnel-starter/pom.xml:46  
<version>${netty.version}</version>`; 
`seatunnel-core/seatunnel-starter/pom.xml:40  
<artifactId>netty-buffer</artifactId>`
   > - Suggested fix: Managing only `netty-buffer` and `netty-common` leaves 
`netty-transport`, `netty-handler`, `netty-codec*`, `netty-resolver` (all 
currently 4.1.118 via the AWS SDK path, see 
`tools/dependencies/known-dependencies.txt:112-121`) unmanaged. The moment the 
AWS SDK or Arrow is bumped those will float to a newer release while these two 
are forced back to 4.1.118, producing a mixed Netty set and 
`NoSuchMethodError`s at runtime. Could we import 
`io.netty:netty-bom:${netty.version}` (`<type>pom</type><scope>import</scope>`) 
instead, so all `io.netty:*` artifacts are kept on a single release?
   > - Severity: Medium
   > 
   > # Review conclusion
   > ### Conclusion: final approval remains pending on the previous comments
   > 
   > **2. Remaining follow-ups**
   > - Issue 1: Hard-coded 4.1.118.Final in the starter POM is a detached copy 
of the version the AWS SDK (root `software.amazon.awssdk.version` = 2.31.30) 
happens to bring; the root POM manages no `io.netty` artifact, so the next 
SDK/Arrow bump silently produces mixed Netty module versions on the starter 
classpath
   > - Issue 2: The module-local pin only makes the shaded starter jar 
deterministic; in the shipped distribution `lib/netty-buffer-4.1.89.Final.jar` 
(from `seatunnel-dist`) is loaded ahead of `starter/seatunnel-starter.jar`, so 
Zeta still runs a different Netty than the one this PR pins
   > - Issue 3: Pinning only `netty-buffer` and `netty-common` with a 
hard-coded version decoupled from the AWS SDK / Arrow that actually produce 
them will yield mixed Netty module versions on the next SDK or Arrow bump
   > 
   > Thanks for the updates so far. Happy to discuss the remaining points.
   
   Thanks for the review. This PR intentionally addresses a specific build 
inconsistency: reactor and standalone builds of `seatunnel-starter` resolve 
different Netty versions.
   
   The local validation covered dependency resolution and the actual shaded 
JARs:
   
   | Build | Before | After |
   | --- | --- | --- |
   | Reactor | 4.1.118.Final | 4.1.118.Final |
   | Standalone with installed dependency-reduced POMs | 4.1.104.Final | 
4.1.118.Final |
   
   All **702 Netty class entries** are byte-for-byte identical between the 
patched JARs and match the unmodified reactor JAR. The 42-module reactor 
verification and all 94 existing starter tests also passed, with the validation 
settings documented in the PR.
   
   Regarding issues 1 and 3, broader Netty alignment is worth considering, but 
the future SDK/Arrow upgrade scenario does not demonstrate a regression in this 
patch. Introducing a root-level BOM and updating the dist, storage, and E2E 
modules would expand the compatibility scope considerably. I would prefer to 
handle that in a separate PR.
   
   Regarding issue 2, the cited `assembly-seatunnel-bin-ci.xml` configuration 
already exists before this change. The regular `assembly-seatunnel-bin.xml` 
does not explicitly include these two standalone Netty JARs in the same way, so 
the CI assembly alone does not establish which version the release distribution 
loads. That runtime classpath concern deserves separate verification and does 
not invalidate the starter artifact consistency demonstrated above.
   
   I would like to keep this PR focused on making standalone builds match the 
existing reactor output. If there is a concrete regression introduced by the 
current patch, please provide the dependency tree, artifact comparison, or 
reproducer so I can investigate it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to