It's fine (and required if you do task assignment) to create the DAG
object; just return None from the function (return early) so that there is
no variable containing a DAG object visible within the module.
On Tue, Oct 9, 2018 at 12:21 PM Shah Altaf wrote:
> @*jmeickle *can you share your code
@*jmeickle *can you share your code or any sample? How are you returning
nothing?
I may have spoken too soon. I can't seem to set dag to None, it fails with
*Tried to create relationships between tasks that don't have DAGs yet*
On Tue, Oct 9, 2018 at 12:57 PM Shah Altaf wrote:
>
>
>
Thanks all, following your advice I've decided to do this right now with a
decorator. It looks crude so any suggestions are welcome.
The decorator function checks an environment variable that we set on our
servers and if it matches any of the passed in environment names, it'll
allow the DAG to
This would be a very desirable feature. It's not just an issue of
differing environments, but also changing requirements. A dag may be
used for a time, and then removed from the environment. But if it's
still in the airflow db, it will get a row on the UI with a black "I'
indicating that the
I like James solution better, but the initial thought I had was to deploy
airflowignore files to the environments to filter out files that should not
be processed when filling the DagBag.
Chris
On Mon, Oct 8, 2018 at 10:22 AM James Meickle
wrote:
> As long as the Airflow process can't find the
As long as the Airflow process can't find the DAG as a top-level object in
the module, it won't be registered. For example, we have a function that
returns DAGs; the function returns nothing if it's not in the right
environment.
On Sun, Oct 7, 2018 at 2:31 PM Shah Altaf wrote:
> Hi all,
>
>
Hi all,
tl;dr - Is it possible to conditionally hide a DAG from the UI based on an
environment variable?
Our team has a single repo with several DAGs in it and we deploy it across
multiple 'environments' (think dev, test, and other integration
environments). While most DAGs are meant to run
