SEZ9 commented on issue #10681: URL: https://github.com/apache/seatunnel/issues/10681#issuecomment-5754547207
Thanks for the detailed status update, @junjunclub — that clarifies the current state nicely. To summarize where things stand on the `--dry-run=<level>` proposal: - Layer 0 (`--dry-run static`) and the shared Layer 1 framework (`--dry-run connect`) are merged, with `--check` kept as the compatibility path for static validation and `--dry-run connect` running through the CLI validation path without submitting a cluster job. - The dry-run hooks live in separate opt-in interfaces (`SupportSourceDryRunValidation` / `SupportSinkDryRunValidation`) rather than in `TableSourceFactory` / `TableSinkFactory`, and connectors that don't implement them are reported as `SKIPPED` instead of failing the whole connect run. `DryRunConnectValidator` orchestrates source → transform → sink validation on top of those hooks. - @goutamadwant's MongoDB source (metadata-only) and Kafka sink (schema/metadata checks, no record production or transaction init) PRs are building on those Layer 1 interfaces, which matches the connector-by-connector rollout. That all sounds consistent with the progressive-validation direction. A few concrete asks so we can keep this coordinated: 1. @junjunclub — could you add a short checklist in this issue listing which connectors currently implement the Layer 1 interfaces and which are still `SKIPPED`? That will make it easy for contributors to pick up the next connectors. 2. @goutamadwant — please cross-link both connector PRs to this issue and confirm they only use the shared SPI (no changes to `DryRunConnectValidator` or the interfaces). If either PR does need an SPI change, please call it out explicitly so @junjunclub can weigh in before merge. 3. For the later layers (schema compatibility and beyond), let's agree on the level names and failure semantics in this issue before any implementation PR is opened, so the CLI surface stays consistent with `static` and `connect`. Appreciate everyone pushing this forward. <!-- streview-comment:1200 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
