[ 
https://issues.apache.org/jira/browse/AIRFLOW-1104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564331#comment-16564331
 ] 

ASF subversion and git services commented on AIRFLOW-1104:
----------------------------------------------------------

Commit ed972042a864cd010137190e0bbb1d25a9dcfe83 in incubator-airflow's branch 
refs/heads/master from Dan Fowler
[ https://gitbox.apache.org/repos/asf?p=incubator-airflow.git;h=ed97204 ]

[AIRFLOW-1104] Update jobs.py so Airflow does not over schedule tasks (#3568)

This change will prevent tasks from getting scheduled and queued over
the concurrency limits set for the dag

> Concurrency check in scheduler should count queued tasks as well as running
> ---------------------------------------------------------------------------
>
>                 Key: AIRFLOW-1104
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1104
>             Project: Apache Airflow
>          Issue Type: Bug
>         Environment: see https://github.com/apache/incubator-airflow/pull/2221
> "Tasks with the QUEUED state should also be counted below, but for now we 
> cannot count them. This is because there is no guarantee that queued tasks in 
> failed dagruns will or will not eventually run and queued tasks that will 
> never run will consume slots and can stall a DAG. Once we can guarantee that 
> all queued tasks in failed dagruns will never run (e.g. make sure that all 
> running/newly queued TIs have running dagruns), then we can include QUEUED 
> tasks here, with the constraint that they are in running dagruns."
>            Reporter: Alex Guziel
>            Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to