nclaeys commented on issue #57776: URL: https://github.com/apache/airflow/issues/57776#issuecomment-3699003442
I have been testing this on Airflow 3.1.5 and I still see issues when working with big dags. I am noticing improvements (e.g. load on the database) in some areas but still fail to work well with big dags compared to what was possible in Airflow 2. As the ticket described there are 2 issues when working with dags that have many tasks: - performance degradation: api responses and rendering is really slow (mainly grid view at the moment, the issue with task view is tackled in: https://github.com/apache/airflow/issues/59598) - memory usage increases dramatically. In my test case I go from using 700Mb to 2.4Gb by just opening a big dag (600+ tasks and a couple of dag runs) and only 1 user. This is causing huge issues at our big Airflow environments, which did not exist on Airflow 2. @pierrejeambrun @potiuk Is there plan for tackling these UI/API server issues? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
