[
https://issues.apache.org/jira/browse/AIRFLOW-6011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976872#comment-16976872
]
Aaron Liblong commented on AIRFLOW-6011:
----------------------------------------
That documentation on packaged DAGs says the following:
{quote}
{code:java}
my_dag1.py
my_dag2.py
package1/__init__.py
package1/functions.py
{code}
{quote}
{quote}Airflow will scan the zip file and try to load my_dag1.py and
my_dag2.py. It will not go into subdirectories as these are considered to be
potential packages.
{quote}
My issue is that Airflow _did_ look in the subdirectory, and specifically for
files containing the character sequence "DAG".
> Airflow treats local dependencies as DAG files if they contain the string
> "DAG"
> -------------------------------------------------------------------------------
>
> Key: AIRFLOW-6011
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6011
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG, dependencies
> Affects Versions: 1.10.2
> Reporter: Aaron Liblong
> Priority: Major
>
> Our Airflow instance uses what I believe is a common pattern of having a
> directory for DAG dependencies within the main {{dags}} directory itself.
> E.g.:
> {code:java}
> dags
> | - dag1.py
> | - dag2.py
> | - dependencies
> | | - dependency1.py
> | | - dependency2.py
> {code}
>
> Then {{dag1.py}} contains something like {{import dependency1}}.
> The dependencies are normally not treated like DAG files themselves, as
> expected. However, when they contain the string "DAG", even in a comment,
> they _will_ be treated as DAG files. I'm guessing this is the result of
> Airflow doing a recursive grep for that exact (case-sensitive) string. This
> is not intuitive behaviour for users, and is hard to debug, since users will
> reasonably expect non-directive comments not to have an effect on how their
> code will being treated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
