hanxdatadog opened a new pull request, #71793: URL: https://github.com/apache/airflow/pull/71793
Optionally emit Dag tags as structured log context (logging counterpart to `dag_tags_in_metrics`) ## What does this PR do? - Adds a new `[logging] dag_tags_in_logs` config option (boolean, default `False`). - When enabled, `task_runner.startup()` binds Dag-tag-derived fields into structlog contextvars right after the Dag is parsed, so every subsequent log line for that task attempt carries them. - Reuses `build_dag_metric_tags()` (the same helper backing `[metrics] dag_tags_in_metrics`, added in #68568) so tags of the form `key:value` split into a key/value field; plain tags map to an empty-string value — keeping the same tag-parsing convention across metrics and logs. - Built-in TI identifier keys (`ti_id`, `dag_id`, `task_id`, `run_id`, `try_number`, `map_index`) always win on collision with a same-named Dag tag. ## Motivation Dag tags (e.g. `team:data-eng`, `env:prod`) are already surfaced in metrics via `dag_tags_in_metrics`, but not in structured logs — users who want this today have to write a custom Airflow listener plugin to bind the context themselves. This closes that gap with a small, opt-in, native option, mirroring the metrics precedent. ## Testing - Unit tests: 4 new tests added to `task-sdk/tests/task_sdk/execution_time/test_task_runner.py` covering: flag off (no leak into log context), flag on with mixed `key:value`/bare tags, flag on with no tags, and the reserved-key collision guard. - Ran via `breeze testing core-tests`: 7/7 pass (4 new + 3 pre-existing `dag_tags_in_metrics`/`stats_tags` tests, confirming no regression). - Static checks: `ruff format`, `ruff check`, `mypy-task-sdk`, YAML lint, newsfragment validation — all pass via `prek`. ## Additional Notes - Intentionally scoped to `task_runner.py` + config + tests only — no changes to the Listener Plugin API, executors, or Celery. Dag tags are already available in-process at this point (the Dag is parsed just above), so no new DB/ORM access is introduced. - Newsfragment filename uses a `00000` placeholder pending the real PR number (renamed before this PR is opened). - Related: #37901 (Dag tags → observability tooling, currently Sentry-scoped) and #68568 (the metrics counterpart of this change). ## Who will it impact? Opt-in only (default `False`) — no behavior change for existing deployments unless explicitly enabled. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (Claude Code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
