airflow BashOperator use beeline to insert data to hive with chinese,data be garbled

with BashOperator ,i use beeline to insert data into hive ,hql with chinese characters ,after dag run success,hive data contain unreadable code. python : # -*- coding: utf-8 -*- import airflow from airflow.models import DAG from airflow.operators.bash_operator import BashOperator from airflow.ope

Re: Pinning dependencies for Apache Airflow

I am still not convinced that pinning is bad. I re-read again the whole mail thread and the thread from 2016 to read all the arguments, but I stand by pinning. I am - of course - not sure about graduation argument. I wo

Re: Can a DAG be conditionally hidden from the UI?

It's fine (and required if you do task assignment) to create the DAG object; just return None from the function (return early) so that there is no variable containing a DAG object visible within the module. On Tue, Oct 9, 2018 at 12:21 PM Shah Altaf wrote: > @*jmeickle *can you share your code o

Re: Pinning dependencies for Apache Airflow

Erik, the Airflow task execution code itself of course must run somewhere with Airflow installed, but if the task is making a database query or a web request or running something in Docker there's separation between the environments and maybe you don't care about Python dependencies at all except t

Re: Can a DAG be conditionally hidden from the UI?

@*jmeickle *can you share your code or any sample? How are you returning nothing? I may have spoken too soon. I can't seem to set dag to None, it fails with *Tried to create relationships between tasks that don't have DAGs yet* On Tue, Oct 9, 2018 at 12:57 PM Shah Altaf wrote: > > > Tha

Re: Can a DAG be conditionally hidden from the UI?

Thanks all, following your advice I've decided to do this right now with a decorator. It looks crude so any suggestions are welcome. The decorator function checks an environment variable that we set on our servers and if it matches any of the passed in environment names, it'll allow the DAG to be

Re: Pinning dependencies for Apache Airflow

This is maybe a stupid question, but is it even possible to run tasks in an environment where Airflow is not installed? Kind regards, Erik From: Matt Davis Sent: Monday, October 8, 2018 10:13:34 PM To: dev@airflow.incubator.apache.org Subject: Re: Pinning depe