gnehil opened a new pull request, #11330:
URL: https://github.com/apache/seatunnel/pull/11330
### Purpose of this pull request
Fix a Doris sink stream load failure that occurs when FE is slow to return
the 307 redirect.
The stream load client sends `Expect: 100-continue` and relies on FE's 307
redirect to reach BE. `HttpUtil` builds the HttpClient without a custom
`HttpRequestExecutor`, so the default `waitForContinue` timeout of **3s**
applies. When FE is busy (heavy load / FullGC / network jitter) and the 307
arrives later than 3s:
1. The client stops waiting and sends the non-repeatable `InputStreamEntity`
body to FE;
2. When the 307 finally arrives, `RedirectExec` detects the consumed entity
(`RequestEntityProxy.isRepeatable == false`), silently refuses to follow the
redirect, and returns the raw 307 response to the caller;
3. The load fails with `STREAM_LOAD_FAILED ... HTTP/1.1 307 Temporary
Redirect`.
This PR raises the `waitForContinue` timeout to **60s** by setting
`.setRequestExecutor(new HttpRequestExecutor(60_000))` on the client builder,
aligned with the same fix in doris-flink-connector
(apache/doris-flink-connector#522).
`HttpUtil` is the single client-building choke point on the sink side, so
this covers all FE 307 redirect paths: `DorisStreamLoad` (write / pre-commit /
abort) and `DorisCommitter` (2PC commit / abort).
### Does this PR introduce _any_ user-facing change?
No. No config option, public API, or dependency changes — only the internal
wait-for-continue timeout is raised from 3s to 60s, matching
doris-flink-connector's default behavior.
### How was this patch tested?
Added `HttpUtilTest#testStreamLoadClientWaitsForSlowFeRedirect`, which
reproduces the scenario end-to-end:
- A raw-socket FE stub reads the request headers, delays the 307 redirect by
4s (longer than the old 3s default, far shorter than the new 60s) without
sending `100 Continue`. (A JDK `HttpServer` cannot simulate this — it
auto-replies `100 Continue` before the handler runs.)
- A JDK `HttpServer` BE stub receives the redirected request and returns 200.
- The client sends an `HttpPut` built by `HttpPutBuilder` with a
non-repeatable `InputStreamEntity`.
- Asserts the request follows the 307 to BE, returns 200, and BE receives
the full body.
Before the fix this test fails with `expected: <200> but was: <307>`
(instrumentation confirmed the body was prematurely sent to FE and BE was never
hit); after the fix it passes.
Also ran the full connector-doris unit test suite (111 tests, all passing),
including the existing `DorisStreamLoadTest` / `DorisCommitterTest`
307-handling regressions, plus `./mvnw spotless:apply` and `./mvnw -DskipTests
verify`.
### Check list
* [x] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
— N/A, no new dependency
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs — N/A, no
user-facing change
* [x] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR. — N/A, no incompatible change
* [ ] If you are contributing the connector code, please check that the
following files are updated: — N/A, bug fix in existing connector
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
