[
https://issues.apache.org/jira/browse/AIRFLOW-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Kim updated AIRFLOW-3001:
-------------------------------
Summary: Accumulative tis slow allocation of new schedule (was:
accumulative tis slow allocation of new schedule)
> Accumulative tis slow allocation of new schedule
> ------------------------------------------------
>
> Key: AIRFLOW-3001
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3001
> Project: Apache Airflow
> Issue Type: Improvement
> Components: scheduler
> Affects Versions: 1.10.0
> Reporter: Jason Kim
> Priority: Major
>
> I have created very long term schedule in short interval (2~3 years as 10 min
> interval)
> So, dag would be bigger and bigger as scheduling goes on.
> Finally, at critical point (I don't know exactly when it is), the allocation
> of new task_instances get slow and then almost stop.
> I found that in this point, many slow query logs had occurred. (I was using
> mysql as meta repository)
> queries like this
> "SELECT * FROM task_instance WHERE dag_id = '~' and execution_date = '~'"
> I could resolve this issue by adding new index consist of dag_id and
> execution_date
> So, I wanted 1.10 branch to be modified to create task_instance table with
> the index.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
