zozo123 opened a new issue, #74196:
URL: https://github.com/apache/airflow/issues/74196

   ### Description
   Extend the Unity asset examples beyond COPY INTO, beginning with explicit 
inlets/outlets on SQL and job operators. Evaluate typed 
input_tables/output_tables convenience only if examples demonstrate an 
ergonomic or validation gap beyond existing BaseOperator arguments.
   
   ### Use case/motivation
   SQL, jobs and pipelines may write known Unity tables even when automatic SQL 
lineage is unavailable. Authors need consistent asset identities and success 
semantics without maintaining a separate inferred SQL parser.
   
   ### Delivery and acceptance
   - After #74191 settles the UnityTableIdentity API, add examples for 
DatabricksSQLStatementsOperator and RunNow/SubmitRun using explicit 
inlets/outlets.
   - Preserve caller-provided outlets, including explicit empty lists; define 
mapped-task and templated-name behavior.
   - Validate workspace consistency where the connection is available; 
distinguish declared outputs from outputs actually verified by the provider.
   - Emit output asset events only after remote work is confirmed successful. 
Fire-and-forget submission must not be presented as a confirmed table refresh.
   - Cover synchronous and deferrable completion, remote failure/cancellation 
and retry behavior. Treat task-success events as task-success events; do not 
claim a new Delta commit for a no-op job.
   - If typed convenience parameters are justified, add them to one operator 
first, then reuse the proven design. Extract shared helpers only after real 
duplication.
   - Reconcile asset identity with existing OpenLineage conventions without 
promising automatic lineage for arbitrary SQL.
   
   ### Related issues
   Follow-up to #74191. Its current docs already recommend explicit outlets for 
SQL/job operators; this request starts from that supported capability, rather 
than assuming it is absent. Pipeline-specific extensions can follow 
demonstrated demand.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to