[ https://issues.apache.org/jira/browse/AIRFLOW-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458921#comment-16458921 ]
John Arnold commented on AIRFLOW-2393: -------------------------------------- yeah it gets slow for us too. ~100 parallel subdags per dag run, 1-4 tasks per subdag. > UI tree view struggles with large dags (60 tasks)x25 dag histories > ------------------------------------------------------------------ > > Key: AIRFLOW-2393 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2393 > Project: Apache Airflow > Issue Type: Improvement > Components: ui > Affects Versions: 1.9.0 > Reporter: Badger > Priority: Major > > Hi, > We are noticing the tree view is taking a long time to render as our DAG has > become more complex. We will need to start breaking our dag apart in order to > continue to use the user interface. > The basic problem is that a reasonably complex DAG (60 operators) x the > standard 25 dag run histories on the tree view causes a 350MB json response > (compressed to 8MB) to be downloaded, this then needs the browser to render > it. > On quick observation this appears to be because, the response appears to > contain all meta-data for each task. > Is this something others think is a problem. We occasionally have to refresh > due to memory errors and have already increased the RAM allocated to the box. > A suggestion might be to load specific instance history when a user hovers > over the task, rather than exporting all of the history on page load. I'd > look at contributing a PR but haven't had chance to take a look at this area > of the code base. > Thanks -- This message was sent by Atlassian JIRA (v7.6.3#76005)