[ https://issues.apache.org/jira/browse/AIRFLOW-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on AIRFLOW-2760 started by Kevin Yang. ------------------------------------------- > DAG parsing loop coupled with scheduler loop > -------------------------------------------- > > Key: AIRFLOW-2760 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2760 > Project: Apache Airflow > Issue Type: Improvement > Reporter: Kevin Yang > Assignee: Kevin Yang > Priority: Major > > Currently DAG parsing loop is coupled with scheduler loop, meaning that if > scheduler loop became slow, we will parse DAG slower. > As a simple producer and consumer pattern, we shall have them decoupled and > completely remove the scheduling bottleneck placed by DAG parsing--which is > identified in Airbnb as the current biggest bottleneck. -- This message was sent by Atlassian JIRA (v7.6.3#76005)