potiuk commented on issue #36510: URL: https://github.com/apache/airflow/issues/36510#issuecomment-1872934038
Just to add a bit more context: * https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/overview.html#basic-airflow-architecture - shows the basic architecture of Airflow where DAG folder is synchronized and used with Triggerer (this image has been updated recently in 2.8.0 but it reflects how it should work) * Triggerrer is classified as "untrusted" component - i.e one that is supposed to run the "Dag Author" code and in the future - even in multi-tennant architecture it should read both DAG files and folder. You can see it in the next diagram https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/overview.html#separate-dag-processing-architecture - where we explained our security model and how we plan to implement tenancy isolation. Trigger will be part of the tenant and it will need access to DAG files, it will also communicate via internal API with the DB. So there is no particular reason for Trigger NOT to have access to the DAG files * When you update new Triggers in the DAG folder, you should restart Triggerer to pick those changes. This is something that should be deployment controlled - for example you should either be able to restart the triggerer manually or it should have a file watcher (if designated folders contain Trigger code provided by the user). But this is a deployment setup not airflow feature. So all-in-all: * Composer's Triggerer should have access to DAG files * Composer's Triggerer should be restartable when Trigger sources change - either manually or via some kind of watching mechanism Nothing to do in Airlfow - you should raise an issue to Composer team about it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
