SEZ9 commented on PR #10502: URL: https://github.com/apache/seatunnel/pull/10502#issuecomment-5724339615
Hi @CloverDew, thanks for the explanation on the test coverage. On the Flink test: given that `testDirtyDataCollectionFailure` in `DirtyDataCollectionIT` runs against Zeta, Spark and Flink, I'm fine keeping a dedicated Flink starter-path fail-fast test (explicit invalid dirty config must fail instead of degrading to `NoOp`) as a recommended follow-up rather than a blocker. On docs, I'd still like to have them in this PR. The change already introduces a user-facing contract: the `dirty.collector` / `dirty.validator` env keys, the sink-level-overrides-env precedence rule, the fail-fast behavior for invalid config, the `counting` collector type with its threshold / `fail_on_threshold` options, and the `DirtyRecordCollector` / `DirtyDataValidator` SPI extension points. Once merged, users will hit those without any reference, so matching `docs/en` and `docs/zh` coverage should land together with the code. A minimal page is fine to start: the two config keys, the precedence rule, the `counting` options, and a short note on how to plug in a custom collector/validator. Happy to help trim the scope if that's the main blocker. Two concrete asks before the next round: 1. The latest head `915fe2f` currently has a merge conflict against `dev`, so please sync with `dev` and resolve it first. 2. Add the `docs/en` / `docs/zh` coverage described above. Please ping me once those are pushed and I'll re-review the new commit. No rush; I just wanted to make sure the remaining items are clear. <!-- streview-comment:1134 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
