surafel58 opened a new pull request, #11974: URL: https://github.com/apache/seatunnel/pull/11974
### Purpose of this pull request Closes #11911. Follow-up to #11827 (raised by @SEZ9 in that review). The checkpoint flush added in #11827 makes remote-write success gate the checkpoint. Two consequences motivated this change: - A single transient failure (transport error, or a `5xx`/`429` response) could fail a checkpoint, and on Flink's default `tolerableCheckpointFailureNumber=0` restart the whole job. - After a restore, the source replays from the last successful checkpoint and re-sends the buffered samples. If the receiver rejects a re-sent sample as a duplicate or out-of-order (`400`), the flush would fail the checkpoint again and loop the job. This adds bounded retry and tolerates the replay case, **reusing the connector's existing HTTP retry options** (`retry`, `retry_backoff_multiplier_ms`, `retry_backoff_max_ms` from `HttpCommonOptions`) rather than introducing new ones. What changed: - `PrometheusSink` wires the existing retry options into `HttpParameter`, which activates the base `HttpClientProvider`'s transport-`IOException` retry (previously `retry` was never set, so it was effectively disabled). `retry` defaults to `3` when unset, so retries are on by default. - `PrometheusWriter.flush()` additionally retries retryable HTTP statuses (`5xx` and `429`) — the base retryer cannot see these because they come back as responses, not exceptions — with exponential backoff capped at `retry_backoff_max_ms`. Other `4xx` responses fail fast. A `400` the receiver reports as a duplicate/out-of-order sample is treated as delivered, so a replay after restore does not fail the checkpoint or loop the job. The delivery guarantee remains at-least-once. ### Does this PR introduce _any_ user-facing change? Yes, a behavior improvement, no new options. - Before: the Prometheus sink never wired the `retry` option into the HTTP client, so a transient remote-write failure was not retried at all; a `5xx` response or a replay-time duplicate `400` failed the flush (and, via the #11827 checkpoint flush, could fail/loop the job). - After: transient transport failures and `5xx`/`429` responses are retried up to `retry` times (default `3`) with exponential backoff, and a duplicate/out-of-order `400` is tolerated as delivered. The `retry` option now covers `5xx`/`429` in addition to `IOException`. The Prometheus sink docs (EN and ZH) are updated: the `retry` option description and default, and the "Checkpoint Flush and Failure Handling" section. ### How was this patch tested? Added unit tests in `PrometheusWriterTest`: - `shouldRetryRetryableFailureThenSucceed` — a `503` then `204` delivers the batch (two attempts). - `shouldThrowAfterRetriesExhausted` — a persistent `503` throws after `retry + 1` attempts. - `shouldFailFastOnNonRetryable4xx` — a `403` throws after a single attempt despite `retry=5`. - `shouldTreatDuplicateOrOutOfOrder400AsDelivered` — a duplicate `400` returns normally and clears the buffer. The full `connector-prometheus` module test suite passes locally (14 tests, 0 failures) on JDK 8. ### Check list * [x] If necessary, please update the documentation to describe the new feature. (Prometheus sink docs updated, EN and ZH) * [ ] New Jar binary package: N/A * [ ] New connector: N/A (modifies an existing connector; no new options, no plugin-mapping / seatunnel-dist / plugin_config changes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
