[ https://issues.apache.org/jira/browse/AIRFLOW-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tylar Murray updated AIRFLOW-2535: ---------------------------------- Description: `TriggerDagRunOperator` does not respect the `max_active_runs` setting. This in itself is not a huge issue, but the scheduler breaks down if `max_active_runs` is exceeded. Task scheduling becomes absurdly slow, quickly leading to perpetual DAG buildup. `TriggerDagRunOperator` could throw exception if `max_active_runs` will be exceeded, or maybe dags could be put into a `queued` state rather than directly into `running`? Alternatively: the scheduler probably shouldn't slow task scheduling when `max_active_runs` is exceeded. was: `TriggerDagRunOperator` does not respect the `max_active_runs` setting. This in itself is not a huge issue, but the scheduler breaks down if `max_active_runs` is exceeded. Task scheduling becomes absurdly slow quickly leading to perpetual DAG buildup. `TriggerDagRunOperator` could throw exception if `max_active_runs` will be exceeded, or maybe dags could be put into a `queued` state rather than directly into `running`? Alternatively: the scheduler probably shouldn't slow task scheduling when `max_active_runs` is exceeded. > TriggerDagRunOperator ignores max_active_runs, leading to scheduler breakdown > ----------------------------------------------------------------------------- > > Key: AIRFLOW-2535 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2535 > Project: Apache Airflow > Issue Type: Bug > Components: operators, scheduler > Affects Versions: 1.9.0 > Environment: CeleryExec, Rabbitmq, mysql > Reporter: Tylar Murray > Priority: Minor > > `TriggerDagRunOperator` does not respect the `max_active_runs` setting. This > in itself is not a huge issue, but the scheduler breaks down if > `max_active_runs` is exceeded. Task scheduling becomes absurdly slow, quickly > leading to perpetual DAG buildup. > `TriggerDagRunOperator` could throw exception if `max_active_runs` will be > exceeded, or maybe dags could be put into a `queued` state rather than > directly into `running`? > Alternatively: the scheduler probably shouldn't slow task scheduling when > `max_active_runs` is exceeded. -- This message was sent by Atlassian JIRA (v7.6.3#76005)