kacpermuda opened a new pull request, #66992:
URL: https://github.com/apache/airflow/pull/66992
Introduces a single `[openlineage] lineage_controls` configuration option
that replaces
the patchwork of per-feature toggles with one coherent rule language.
## What changed
**New config: `lineage_controls`** — a JSON array of rule objects, each with
a match scope
and one or more control flags:
```ini
[openlineage]
lineage_controls = [
{"operator": "airflow.providers.http.operators.http.HttpOperator", "emit":
false},
{"dag_id": "expensive_dag", "extract_metadata": false},
{"dag_id": "expensive_dag", "task_id": "send_report", "extract_metadata":
true},
{"dag_id": "reporting_dag", "emit_dag_events": false},
{"dag_id": "^staging_.*", "match_mode": "regex", "emit": false},
{"dag_id": "audit_dag", "include_full_task_info": true},
{"source_code": false}
]
```
**Match scopes** (priority order, most-specific wins): `dag_id + task_id` →
`dag_id` →
`operator` → global. Within a tier, last rule wins.
**Control flags**: `emit`, `emit_task_events`, `emit_dag_events`,
`extract_metadata`,
`source_code`, `hook_lineage`, `include_full_task_info`.
**`match_mode: "regex"`** — match scope values are treated as `re.fullmatch`
patterns.
**Audit logging** — every non-default resolved flag is logged at INFO with
the exact rule
that caused it, giving a clear audit trail.
**Contradiction detection** — when two rules in the same priority tier set
the same flag to
opposite values, a WARNING is logged (last value still wins).
## Backwards compatibility
Legacy options (`disabled_for_operators`, `disable_source_code`,
`include_full_task_info`,
`selective_enable`) are **not removed**.
- When `lineage_controls` is **absent/empty**: legacy options work exactly
as before — zero behaviour change.
- When `lineage_controls` is **set**: legacy options are automatically
translated into equivalent rules and prepended before the explicit rules (so
explicit rules win within each tier). A `DeprecationWarning` is issued listing
the translated options.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (claude-sonnet-4-5)
Generated-by: Claude Code (claude-sonnet-4-5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
