[
https://issues.apache.org/jira/browse/AIRFLOW-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312673#comment-17312673
]
Chris edited comment on AIRFLOW-2535 at 3/31/21, 7:43 PM:
----------------------------------------------------------
This is making me avoid the TriggerDagRunOperator like the plague. I think I'll
stick with Sensors until this gets sorted out. I'm on Airflow 1.10.10. Will
this happen in v2.0+?
was (Author: cdabel):
This is making me avoid the TriggerDagRunOperator like the plague. I think I'll
stick with Sensors until this gets sorted out.
> TriggerDagRunOperator ignores max_active_runs, leading to scheduler breakdown
> -----------------------------------------------------------------------------
>
> Key: AIRFLOW-2535
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2535
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators, scheduler
> Affects Versions: 1.9.0
> Environment: CeleryExec, Rabbitmq, mysql
> Reporter: Tylar Murray
> Priority: Minor
>
> `TriggerDagRunOperator` does not respect the `max_active_runs` setting. This
> in itself is not a huge issue, but the scheduler breaks down if
> `max_active_runs` is exceeded. Task scheduling becomes absurdly slow, quickly
> leading to perpetual DAG buildup.
> `TriggerDagRunOperator` could throw exception if `max_active_runs` will be
> exceeded, or maybe dags could be put into a `queued` state rather than
> directly into `running`?
> Alternatively: the scheduler probably shouldn't slow task scheduling when
> `max_active_runs` is exceeded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
