markap14 commented on PR #11325:
URL: https://github.com/apache/nifi/pull/11325#issuecomment-4681500612
## Experimental commit: temporarily revert Jetty 12.1.10 → 12.1.9
Pushed `451d60b` to test the hypothesis that the recent system-tests flakes
are a server-side HTTP/2 regression introduced by the Jetty 12.1.9 → 12.1.10
bump in NIFI-15993 (2026-06-03), not by anything in this PR.
### Why I think Jetty 12.1.10 is the trigger
- The failures all surface as the JDK `java.net.http.HttpClient` receiving
an HTTP/2 `RST_STREAM` with code `CANCEL` from the in-JVM Jetty server during
the request body upload of a replicated cluster request:
```
java.io.IOException: RST_STREAM received Stream cancelled
at
java.net.http/jdk.internal.net.http.Stream.incompleteRequestBodyReset(Stream.java:730)
at
java.net.http/jdk.internal.net.http.Stream.incoming_reset(Stream.java:712)
```
- I counted status codes in `nifi-request.log` for the failing test: **zero
HTTP 421 responses on either node**, which rules out
`ProxyHeaderValidatorCustomizer` / `HostPortValidatorCustomizer` as the source
of the reset.
- Correlation with main-branch `system-tests` workflow history:
- 2026-06-03 13:05 UTC: main run **SUCCESS** (last green run).
- 2026-06-03 21:48 UTC: `f5b9c13` NIFI-15993 bumps **Jetty 12.1.9 →
12.1.10** (and several other unrelated deps).
- 2026-06-09 02:21 UTC: main run **FAILURE** — first system-tests run on
main after the Jetty bump.
- Every `system-tests` workflow run on main since then has failed.
- Jetty 12.1.10's notable HTTP/2 changes per the [release
notes](https://github.com/jetty/jetty.project/releases/tag/jetty-12.1.10)
include #15009 "Make processing of RST_STREAM more lenient" and #15161 "Reduce
memory footprint for persistent HttpConnections", both of which touch HTTP/2
stream/connection lifecycle.
### What this commit is and isn't
This is **not** intended to be merged as-is. The PR remains a Draft. If this
commit's CI run shows the flakes disappear, we will:
1. File an upstream Jetty bug with a minimal repro.
2. Decide whether to pin Jetty to 12.1.9 in `main` until it's fixed, or wait
for a 12.1.11 with a targeted fix.
If the flakes persist on 12.1.9, the hypothesis is wrong and we keep digging.
The existing `LoadBalanceIT` batch-size reduction in this PR stays in place
as defense in depth either way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
