potiuk commented on PR #36035: URL: https://github.com/apache/airflow/pull/36035#issuecomment-1837604071
I introduced two diagrams now - one basic, and one with standalond DAG file processor. While it is not yet fullly - mutlitenant, this already gives some good properties (like scheduler not having accesss DAG files at all) and having this picture described now is a good idea - and reflecting the current architecture. I also added a "dashed" line showing "executors" -> link between scheduler and workers and it looks better in Right to Lefte form - it also shows nicely the progression of things that happen with the tasks - scheduler being on the left of workers and triggerers. I converted the script to be entirely in pre-commit and added hash check so that it will not be running unnecessarily even in CI with `--all-files`. This way we will not recreate the images when not needed - only when the diagram sources change. I left two types of users for now. - I still think it is a good idea even for "basic" diagram. I also hope we will regenerate more diagrams using the same approach - celery , kubernetes, logging etc. - they will be so much -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
