Anish Mahto created SPARK-56956:
-----------------------------------
Summary: AutoCDC Flow Execution; Introduce Flow Dataclasses
Key: SPARK-56956
URL: https://issues.apache.org/jira/browse/SPARK-56956
Project: Spark
Issue Type: Sub-task
Components: Declarative Pipelines
Affects Versions: 4.2.0, 4.3.0
Reporter: Anish Mahto
Introduce new flow data classes to represent an unresolved AutoCDC flow, and a
resolved AutoCDC SCD1 flow.
Eventually, PySpark/SQL APIs will register an unresolved AutoCDC flow into the
`GraphRegistrationContext`, and the `CoreDataflowNodeProcessor` will resolve to
an SCD1 (or in the future SCD2) flow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
