ashb commented on code in PR #72043: URL: https://github.com/apache/airflow/pull/72043#discussion_r3956545480
########## go-sdk/adr/0007-native-dag-interface.md: ########## @@ -0,0 +1,140 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + --> + +# 7. Native Dag interface + +Date: 2026-09-07 + +## Status + +Proposed. Supersedes the interfaces proposed in #67155 and #70158, reshaped by the design review +on #72043. Nothing below exists on `main`. + +## Why + +A native Dag is authored entirely in Go: the author owns the schedule, the tasks, and the +dependencies, and the SDK serializes all of it into the Dag JSON a Python Dag would produce. There is +one Dag type, as in Python — the Mixed Lang case registers task handlers instead +([ADR 6](0006-mixed-lang-task-handler-interface.md)), because it defines no Dag. Dependencies between +Go functions have to be typed rather than looked up by task ID, and a Dag should read like Go rather +than like transliterated Python. + +## Example + +Both examples build the same graph. Which form an author writes depends on whether the edge carries a +value. + +### Data dependencies: the TaskFlow equivalent + +`airflow.Inputs` passes an upstream's return value in and declares the edge in one call, the way +calling one TaskFlow function with another's output does in Python. + +```go +dag := airflow.Dag(airflow.DagSpec{DagId: "etl", Schedule: "@daily"}) + +extracted := dag.Task(extract) +transformed := dag.Task(transform, airflow.Inputs(extracted)) +dag.Task(load, airflow.Inputs(transformed), airflow.TaskSpec{Retries: 2}) + +registry.AddDags(dag) +``` + +```python +# the Python Dag this mirrors +extracted = extract() +transformed = transform(extracted) +load(transformed) +``` + +```go +func extract(ctx context.Context) (Result, error) { + return Result{Message: "native Dag data"}, nil +} + +func transform(ctx context.Context, extracted Result) (Result, error) { + return Result{Message: "transformed " + extracted.Message}, nil +} + +func load(ctx context.Context, transformed Result) error { return nil } +``` + +### Order-only dependencies: the `>>` and `<<` equivalent + +`Before` and `After` draw an edge and pass nothing, for tasks that must be ordered but exchange no Review Comment: Elsewhere in this PR you have `task.Then(task2)` -- we likely shouldn't have both, right? Lets be consistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
