[ https://issues.apache.org/jira/browse/AIRFLOW-6853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johannes Wienke updated AIRFLOW-6853: ------------------------------------- Summary: ExternalTaskSensor check_existence does not work with packaged DAGs (was: ExternalTaskSensor does not work with packaged DAGs) > ExternalTaskSensor check_existence does not work with packaged DAGs > ------------------------------------------------------------------- > > Key: AIRFLOW-6853 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6853 > Project: Apache Airflow > Issue Type: Bug > Components: operators > Affects Versions: 1.10.9 > Reporter: Johannes Wienke > Priority: Major > > The current implementation of ExternalTaskSensor uses the file location of a > DAG to determine if it still exists. This is realized by the following code > fragment: > {code:python} > if not os.path.exists(dag_to_wait.fileloc): > raise AirflowException('The external DAG ' > '{} was > deleted.'.format(self.external_dag_id) > {code} > In case the DAG to check is part of a Zip archive, fileloc looks something > like: {{/usr/local/airflow/dags/analytics.zip/create_keyframes.py}}. This can > obviously not be checked with {{os.path.exists}} and therefore the sensor is > currently unable to handle packaged DAGs at all. > This also contradicts the explicit note in the DAG model: > {noformat} > # The location of the file containing the DAG object > # Note: Do not depend on fileloc pointing to a file; in the case of a > # packaged DAG, it will point to the subpath of the DAG within the > # associated zip. > {noformat} > I suspect, the sensor should instead use {{is_active}}. -- This message was sent by Atlassian Jira (v8.3.4#803005)