henry3260 opened a new pull request, #73936: URL: https://github.com/apache/airflow/pull/73936
`airflow.DagSpec` and `airflow.TaskSpec` are the two structs a Dag author fills in. They describe the same attributes `airflow-core/src/airflow/serialization/schema.json` declares, and Python owns that schema — so hand-maintaining them means they drift from it in silence. This generates both from the schema and commits the result, the way `pkg/execution/genmodels/models.gen.go` is already generated from the supervisor schema. Implements decision 12 of [ADR 8](https://github.com/apache/airflow/blob/main/go-sdk/adr/0008-native-dag-interface.md), on top of the constructors added in #73767. ### How it is generated Three `go:generate` directives on `go-sdk/airflow/spec.go`: 1. `internal/genspec` rewrites the Python-owned schema into the schema the specs generate from, writing it to `.build/go-sdk/spec.schema.json` (a build artifact; the Python schema is never touched). 2. `go-jsonschema` writes `airflow/spec.gen.go` from it (committed). 3. `genspec -license` puts the Apache header back, which `go-jsonschema` does not emit, so `go generate` is idempotent and a drift check means something. ### Why the schema cannot be generated from as-is It is the *serialized* shape, not the authoring shape, so `internal/genspec/authoring.go` carries three tables, each entry with the reason it exists: - **Exclusions** — properties that must not reach an author: paths and task lists the bundle fills in (`fileloc`, `tasks`), grid colours and template fields of a Python operator class (`ui_color`, `template_fields`), derived flags (`has_on_*_callback`), and properties with no Go authoring type yet (`params`, `inlets`). A property *not* on the list generates, so a property added on the Python side surfaces rather than vanishing. - **Type overrides** — the schema types a moment in time and a duration as a number of seconds and an integral count as a JSON number. These become `time.Time`, `time.Duration` and `int` through `go-jsonschema`'s `goJSONSchema` extension, which it reads before a `$ref`. - **Injection** — `Schedule` over the serialized `timetable`. Injecting into the schema, rather than hand-writing the field, is what keeps every field in one struct declaration — and that is what lets `TaskSpec` implement the sealed `TaskOption`, whose only method is unexported and so belongs to the `airflow` package. `trigger_rule` is typed as a plain `string` with no values named, so `TriggerRule` and its constants are hand-written in `spec.go` and the generated field is given that type. A field is a pointer only where the schema default is not the Go zero value — the rule `genmodels` already applies — so an author writes `Tags: []string{...}` and `Retries: 3`, while `DoXcomPush *bool` can still tell "unset" from "false". ### Drift check `scripts/ci/prek/check_go_sdk_spec_drift.py` regenerates and asks Git whether anything changed. It watches the Python schema, `spec.gen.go` and `internal/genspec`. A drifted file is left regenerated in the working tree and the report points at the exclusion list, so a new property is a review decision rather than an accident. ### Also here The Apache-header insertion that `genmodels/gen` carried is now `internal/genlicense`, used by both generators. Pure extraction — `models.gen.go` is unchanged by it. `TestTriggerRuleConstantsMatchPython` is their tripwire against Airflow's own enum, since the drift check cannot see a value set the schema does not name. ### Not in scope - `weight_rule` needs a named type and constants of its own, as `TriggerRule` has. - `retry_exponential_backoff` is excluded: the schema types it as a `number` while `BaseOperator`'s attribute is a `bool`. - A serializer has to convert the `time.Duration` fields to the seconds the schema carries; no serialization code is added here. ### Checks run - `go test ./...` in `go-sdk`, and `go generate ./airflow/...` twice to confirm it is idempotent - `uv run --project scripts pytest scripts/tests/` (1518 passed) - the `TriggerRule` tripwire verified to fail with a constant removed - `prek run --from-ref main --stage pre-commit` and `--stage manual` (the latter skipping `upgrade-important-versions`, which rewrites unrelated pinned versions) - `breeze ci selective-check`: `run-go-sdk-tests` and `run-scripts-tests`, both run above --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 5) Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
