Bo Blanton created AIRFLOW-6012:
-----------------------------------
Summary: Kubernetes Executer very slow to Clean finished tasks
Key: AIRFLOW-6012
URL: https://issues.apache.org/jira/browse/AIRFLOW-6012
Project: Apache Airflow
Issue Type: Improvement
Components: scheduler
Affects Versions: 1.10.6
Reporter: Bo Blanton
The new clean up task in the K8 executor queries the DB for all running
tasks/dags for each tasks to check for safe labels (tasks and dagid) as they
may have needed to be altered for k8 safety.
For hundereds of tasks finishing, this is a very long process (lots of DB calls
and a large loop).
There's an improvement we can make to to quickly exit the loop if we don't need
to check for the label saftly
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
