SEZ9 commented on PR #10502: URL: https://github.com/apache/seatunnel/pull/10502#issuecomment-5738728615
@CloverDew thanks for the explanation. On the Flink test: agreed that `testDirtyDataCollectionFailure` in `DirtyDataCollectionIT` already exercises the dirty-data path on Zeta, Spark and Flink. The narrower gap is that it does not pin the "explicit invalid dirty config must fail fast instead of degrading to `NoOp`" contract on the Flink starter path specifically. I'm fine keeping that as a non-blocking follow-up rather than a blocker for this PR. On docs: I understand the wish to wait for the feature to stabilize, but this PR already introduces user-facing contract — the `dirty.collector` / `dirty.validator` env keys, the sink-overrides-env precedence rule, the `counting` type's `threshold` / `fail_on_threshold` options, and the collector/validator SPI extension points. Once that lands in `seatunnel-api`, users will start relying on it, so matching `docs/en` and `docs/zh` coverage needs to ship in the same PR. It does not have to be exhaustive; a minimal page covering the keys, precedence, the `counting` options and how to plug in a custom collector/validator is enough to start, and we can iterate from there. Also, the current head `915fe2f` is reported as conflicting with `dev`, so a sync and conflict resolution is needed before another full pass makes sense. Remaining asks to unblock: 1. Sync with the latest `dev` and resolve the merge conflict. 2. Add `docs/en` / `docs/zh` coverage for the items listed above. Ping me here once both are pushed and I'll re-review the new head. The Flink starter-path fail-fast test can follow separately. <!-- streview-comment:1158 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
