seanghaeli opened a new pull request, #76560:
URL: https://github.com/apache/airflow/pull/76560
Building on #76527, which gives each asset watcher its own log file. This
adds `GET /api/v2/triggers/{trigger_id}/logs` to read any trigger's logs,
whatever its kind: a deferred task's trigger lines from the task's log, a
callback's from the callback's log, and a watcher's from its own files. Stacked
on #76527, so only the last commit is new until that merges.
- Returns a trigger's latest lines in the task log shape. It reads them live
from the triggerer, from remote logging and from the local disk, merged in time
order. Nothing is stored twice.
- A watcher's files are read newest first. Reading stops at `tail_lines`, 64
MiB or the 50 newest triggerer jobs, and an unreachable remote store costs one
timeout, not one per job.
- Access follows the owner's log permission: task logs on the Dag for a
deferred task or a callback. A watcher needs read access to one of its assets
and task logs on a Dag scheduled on it, directly or through an asset name or
URI reference. A trigger you can see but whose logs you can't read returns 403.
- With CloudWatch, reading a watcher's history back from remote also needs a
small amazon provider change that reads log streams by prefix (separate PR).
How to review: one commit. Read `get_trigger_logs` in
`routes/public/triggers.py`, then `utils/log/trigger_log_reader.py`. About 630
lines of source; the rest is generated code, docs and tests.
Breeze on Postgres with CloudWatch remote logging and the amazon provider
change merged in locally: a healthy watcher, a watcher with expired
credentials, a deferred task and a deadline callback, then FAB users with one
role each:
```text
# after this PR
=== GET /api/v2/triggers (admin)
total_entries=4
id=1 kind=asset_watcher status=failing host=530e4d5dd729 dag_id=None
fails=48 last_event=None | watchers
expired_credentials_asset/expired_credentials_watcher
last_error='ConnectionError: the security token included in the
request is expired'
id=2 kind=asset_watcher status=running host=530e4d5dd729 dag_id=None
fails=0 last_event=None | watchers healthy_asset/healthy_watcher
id=4 kind=callback status=running host=530e4d5dd729
dag_id=example_deadline_callback fails=0 last_event=2026-11-04T22:35:11.843344Z
| callback 01a1a8b3 dag=example_deadline_callback
run=manual__2026-11-04T22:34:57.853996+00:00 cb_state=running
id=3 kind=task status=running host=530e4d5dd729
dag_id=example_deferred_task fails=0 last_event=None | task
example_deferred_task.wait_for_trigger try=1 ti_state=deferred
=== GET /api/v2/triggers/{id}/logs?tail_lines=3 (admin): each trigger's own
lines
--- trigger 1 (asset_watcher, failing): watcher expired_credentials_watcher
HTTP 200
3 lines;
sources=['http://530e4d5dd729:8794/log/triggers/1/3/trigger.log',
'triggers/1/3/trigger.log']
2026-11-04T22:36:03.249751Z creds cleanup called
2026-11-04T22:36:04.250440Z Trigger ID 1 exited with error the
security token included in the request is expired
2026-11-04T22:36:04.250791Z Trigger exited without sending an event.
Dependent tasks will be failed.
lines of another trigger: 0
--- trigger 2 (asset_watcher, running): watcher healthy_watcher
HTTP 200
3 lines;
sources=['http://530e4d5dd729:8794/log/triggers/2/3/trigger.log',
'triggers/2/3/trigger.log']
2026-11-04T22:36:00.461555Z healthy poll 48
2026-11-04T22:36:02.462200Z healthy poll 49
2026-11-04T22:36:04.462986Z healthy poll 50
lines of another trigger: 0
--- trigger 3 (task, running): task example_deferred_task.wait_for_trigger
HTTP 200
3 lines;
sources=['http://530e4d5dd729:8794/log/dag_id=example_deferred_task/run_id=manual__2026-11-04T22:34:41.439301+00:00/task_id=wait_for_trigger/attempt=1.log.trigger.3.log',
'dag_id=example_deferred_task/run_id=manual__2026-11-04T22:34:41.439301+00:00/task_id=wait_for_trigger/attempt=1.log.trigger.3.log']
2026-11-04T22:36:00.658323Z wait_for_trigger poll 36
2026-11-04T22:36:02.659030Z wait_for_trigger poll 37
2026-11-04T22:36:04.659714Z wait_for_trigger poll 38
lines of another trigger: 0
--- trigger 4 (callback, running): callback of example_deadline_callback
HTTP 200
3 lines;
sources=['http://530e4d5dd729:8794/log/triggerer_callbacks/example_deadline_callback/manual__2026-11-04T22:34:57.853996+00:00/01a1a8b3-8509-7d23-af9c-c921ac1bc315/callback.log',
'triggerer_callbacks/example_deadline_callback/manual__2026-11-04T22:34:57.853996+00:00/01a1a8b3-8509-7d23-af9c-c921ac1bc315/callback.log']
2026-11-04T22:35:58.832372Z tick 17
2026-11-04T22:36:01.832909Z tick 18
2026-11-04T22:36:04.833463Z tick 19
lines of another trigger: 0
=== access rule (FAB): HTTP code of GET /api/v2/triggers/{id}/logs per user
admin 1:200 2:200 3:200 4:200
reads_deferred_task_dag 1:404 2:404 3:200 4:404
reads_deadline_dag 1:404 2:404 3:404 4:200
reads_assets_no_task_logs 1:403 2:403 3:404 4:404
reads_assets_and_task_logs 1:403 2:200 3:404 4:404
no_access 1:403 2:403 3:403 4:403
no token: 401
role r_task: ['can_read on DAG Runs', 'can_read on
DAG:example_deferred_task', 'can_read on Task Instances', 'can_read on Task
Logs']
role r_deadline: ['can_read on DAG Runs', 'can_read on
DAG:example_deadline_callback', 'can_read on Task Logs']
role r_asset_nologs: ['can_read on Assets', 'can_read on
DAG:to_healthy_consumer']
role r_asset_logs: ['can_read on Assets', 'can_read on
DAG:to_healthy_consumer', 'can_read on Task Logs']
role r_nothing: []
```
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change below checkbox to `[X]` followed by the name of the tool, uncomment
the "Generated-by".
-->
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
