ChickenBenny opened a new pull request, #75387:
URL: https://github.com/apache/airflow/pull/75387
* related: #74869
---
A follow-up to #74869.
The supervisor gets `execution_timeout` from `TIRunContext` (the serialized
Dag), but the Python task runner still read `task.execution_timeout` from the
Dag file it parses. If the two differed, for example because the Dag file
changed after it was serialized, the supervisor backstop could silently never
be armed. Now the runner reads `TIRunContext` too, so both use the same value.
```text
┌─ API server ───────────────────────────────┐
│ ti_run: TIRunContext.execution_timeout │
└────────────────────────────────────────────┘
│
▼
┌─ Supervisor ───────────────────────────────┐
│ backstop timer = TIRunContext value │
└────────────────────────────────────────────┘
│ StartupDetails(ti_context)
▼
┌─ Python task runner ───────────────────────┐
│ before: SIGALRM timer = task value │
│ after: SIGALRM timer = TIRunContext value │
└────────────────────────────────────────────┘
```
Notes:
- A `[core] default_task_execution_timeout` set only on the worker no longer
applies to Python tasks. It now comes from where the Dag is serialized, same as
for language SDK tasks.
- Changing `task.execution_timeout` at runtime on the worker, for example in
`pre_execute` or in a cluster policy that only runs on the worker, no longer
has any effect.
- There's no fallback to `task.execution_timeout`, since that would bring
the mismatch back. The runner now relies on the API server sending
`execution_timeout`, so this must be backported to v3-4-test together with
#74869 and released with it.
- `BaseAsyncOperator` still applies `task.execution_timeout` through
`asyncio.wait_for`. That duplicate timer is left for the follow-up that makes
the supervisor the only clock.
- `create_runtime_ti` in `tests_common` now fills `execution_timeout` in the
run context like the Execution API does, so tests that set it on the task keep
working.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
