Dan Davydov created AIRFLOW-2156: ------------------------------------ Summary: Parallelize Celery Executor Key: AIRFLOW-2156 URL: https://issues.apache.org/jira/browse/AIRFLOW-2156 Project: Apache Airflow Issue Type: Improvement Components: celery Reporter: Dan Davydov Assignee: Dan Davydov
The CeleryExecutor doesn't currently support parallel execution to check task states since Celery does not support this. This can greatly slow down the Scheduler loops since each request to check a task's state is a network request. The Celery Executor should parallelize these requests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)