AnishMahto opened a new pull request, #56042: URL: https://github.com/apache/spark/pull/56042
### What changes were proposed in this pull request? Introduce dataclass for unresolved AutoCDC flow (`AutoCdcFlow`) and resolved AutoCDC flow (`AutoCdcMergeFlow`). Add wiring to analyze an `AutoCdcFlow` to an `AutoCdcMergeFlow`. A small refactor was additionally made on the `UnresolvedFlow` and `ResolvedFlow` class hierarchy. ### Why are the changes needed? Support AutoCDC flow registration and analysis. AutoCDC flow execution will be supported in a future PR. Previously, an `UnresolvedFlow` additionally always represented an untyped-flow; a flow where do not yet know its execution-type, i.e streaming, append-once, etc. `AutoCdcFlow` is a specialized flow with support for only streaming flows, hence it represents a flow whose execution-type we know at construction. It is still unresolved at registration time, and needs to go through resolution to determine its position in the DAG and its input/outut schemas. Hence we introduce the intermediary child `UntypedFlow` for `UnresolvedFlow`, which all previous flows are classified as during registration. An `AutoCdcFlow` directly implements `UnresolvedFlow` (skipping `UntypedFlow in its inheritance chain) because it is not untyped. ### Does this PR introduce _any_ user-facing change? No, the AutoCDC feature is not released anywhere yet. ### How was this patch tested? `ConnectValidPipelineSuite` and `AutoCdcFlowSuite` ### Was this patch authored or co-authored using generative AI tooling? Co-authored. Generated-by: Claude-Opus-4.7-thinking-xhigh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
