potiuk commented on issue #67515: URL: https://github.com/apache/airflow/issues/67515#issuecomment-4559927202
And to add.. the measures you have done are completely broken - by definition. You measured something completely different and wrong. In order to parse Dag and tasks needed by the Dag - the tasks should actually **import** all the relevant imports that given Hook or Operator needs. So what you are mostly saving is not the time that you will save, but when you do it. Lazy importing is only effective when you do not actually import things. So you are just **delaying** stuff not avoiding it. And again - you mix CLI startup time with time needed to parse or execute the task. And the latter is basically unaffected by lazy import. So I recommend you to do tests with **actual** airflow executing **actual** tasks. and do similar comparision you did on syntetic, pretty unrealistic benchmark that shows different things that the actual usage in Airflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
