pingzh commented on PR #30104: URL: https://github.com/apache/airflow/pull/30104#issuecomment-1469333404
> > This will add workload for the dag file parsing and even can create lots of import_errors (it happens in our case, we had to add lots of file paths to the .airflowignore, however, it isn't very scalable given there are several thousands of files in our repo). > > Can you share a bit more about the uae case? This sounds like many lib files. I had many similar cases and it was easy solveable by placing lib folders and excluding the folder in .airflowignore It was as easy as one line in .airflowignore that ignores any file with /lib/ in it's path. hi @eladkal thanks for chiming in. Using the `airflowignore` can meet the needs when the dag folder structure is properly and strictly lay out. Having this pluggable callable can give the infra maintainer more flexibility compared to the airflowignore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
