bugraoz93 commented on issue #44652: URL: https://github.com/apache/airflow/issues/44652#issuecomment-3281254874
I’m not certain about the root cause yet, but I think I may have reproduced the issue. In my case, the failure occurs on the liveness probe for the DagFileProcessor. Testing with around 200 Dags. When the DagFileProcessor restarts due to the liveness probe failing, it removes serialised Dags and reparses the entire DagBag, which takes around 5-15 minutes (It is up to # of Dags and their complexity, of course) to catch the heartbeat again. I will investigate this problem further. As Elad mentioned, it would be great if you could provide more information, which would be better -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
