brki opened a new issue, #55675: URL: https://github.com/apache/airflow/issues/55675
### Description For the description below, assume that the DAG: * is scheduled to run once a day * has been paused more than one day * has `max_active_runs=1` * was not running when it was paused (e.g. all tasks are success or failed) When triggering a paused DAG, it would be nice if the manually triggered run starts before any scheduled run. If the DAG has parameters, there is a "Unpause DAG when triggered" toggle, but this doesn't help. This is the behavior currently seen, running with the official Helm chart, using the Celery scheduler: * With "Unpause DAG when triggered" = true: regularly scheduled run starts running, the manually triggered run can run only after the other one completes (naturally, or because marked success / failure) * With "Unpause DAG when triggered" = false: only the manually triggered run is scheduled - until the DAG is unpaused, at which time the regularly scheduled run starts running, and the manually triggered run can run only after the other one completes (naturally, or because marked success / failure) ### Use case/motivation We have and Integration environment, but most of the DAGs are usually paused on Integration. Some of the DAGs process a lot of data, and take a long time to run. These DAGs are run on Integration only occasionally, for example when testing changes. These DAGs often have parameters to reduce the amount of data processed (e.g. only process the list of ids (1,2,3), or only process the source data within a given date/time range). It's an annoyance for us, and counterintuitive, that when we trigger a paused DAG, the manually triggered run, with the parameters we have specified, is not the one that starts running - it's the regularly scheduled DAG run that goes first (obviously, without the parameters that have been specified). We have to then mark the regularly scheduled run as fail / success, before our parameterized manual run starts. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
