Hi everyone,

While working on a custom backfill DAG to trigger another DAG today, I was
surprised to find that there is no way to override the DAG-level
dagrun_timeout for an individual run.

The DAG-level timeout works well for regular scheduled runs. However,
manual runs and custom backfills can process larger batches of data and
need more time. I’d like to set a different timeout for those runs without
changing the DAG definition or affecting other runs.

I opened PR #76311 <https://github.com/apache/airflow/pull/76311> to
support this. The idea is straightforward: persist an optional timeout
override on each DAG run and have the scheduler use it when set, falling
back to the DAG-level timeout otherwise. Omitting the override preserves
the existing behavior.

The diff is fairly large because the parameter needs to be supported across
the APIs, clients, UI, backfills, and SDK, along with tests and generated
schemas. The core change is a nullable database column and the scheduler
logic to select the effective timeout. Support in TriggerDagRunOperator is
planned as a separate follow-up.

I’m sharing this here for visibility and to hear any objections or concerns
I may have missed. Does anyone see a problem with allowing this override,
or an edge case we should consider?

Thanks,
Hussein

Reply via email to