hi, all,
due to some reasons, we need rename a dag.
is it possible to keep the dag run history which used the old name?
thanks.
michael
Hi, airflow dev,
We set up several dags running on the airflow server. Most dags run fine, but
there is one dag which doesn't start at the scheduled time.
Then we found in the "dag" table, there is a "last_expired" column. The
not-started dag has a value, but it's empty for all other dags.
r an expected behavior.
Any comments are welcome.
Thanks.
Michael
From: Michael Gong
Sent: Friday, March 3, 2017 8:56 PM
To: dev@airflow.incubator.apache.org
Subject: Re: Airflow running different with different user id ?
Hi, Dan,
Thanks for the e
what's the difference between the Airflow 1.8.0-incubating release and the
Airflow 1.8.0 release ?
From: Bolke de Bruin
Sent: Monday, March 20, 2017 5:30 PM
To: dev@airflow.incubator.apache.org; annou...@apache.org
Subject: [ANNOUNCE] Apache Airflow 1.8.0-incub
feature, ie. the "run_as_user" parameter ?
Any suggestions ?
Thanks.
Michael
From: Maxime Beauchemin
Sent: Monday, March 6, 2017 5:15 PM
To: dev@airflow.incubator.apache.org
Cc: Michael Gong
Subject: Re: Airflow running different with different
Hi,
Does anyone know where I can find the documentation for the coming airflow
1.8.0 release ?
thanks.
Michael
From: Michael Gong
Sent: Monday, March 6, 2017 2:37 PM
To: dev; Michael Gong
Subject: Re: Airflow running different with different user id ?
Hi
Hi, Dan,
The only doc mentioning "run_as" was in the airflow.operators.HiveOperator.
Is this what you mean "run_as"?
If not, can you provide more information about it? We are very interested to
know more .
Thanks.
Michael
Sent from my PP•KING™ smartphone
On Mar 3, 2017 3
Hi, Dan,
The only doc mentioning "run_as" was in the airflow.operators.HiveOperator.
Is this what you mean "run_as"?
If not, can you provide more information about it? We are very interested to
know more .
Thanks.
Michael
Sent from my PP•KING™ smartphone
On Mar 3, 2017 3
From: Dan Davydov
Sent: Friday, March 3, 2017 8:44 PM
To: dev@airflow.incubator.apache.org
Subject: Re: Airflow running different with different user id ?
Within a couple of weeks.
On Fri, Mar 3, 2017 at 12:34 PM, Michael Gong wrote:
> When approximately will it be released?
>
> Se
When approximately will it be released?
Sent from my PP•KING™ smartphone
On Mar 3, 2017 1:42 PM, Dan Davydov wrote:
Yes it is starting on 1.8.0 which will be released soon, you can look in
the documentation/grep for "run_as".
On Mar 3, 2017 8:50 AM, "Michael Gong" wrote:
Hi,
Suppose I have 1 airflow instance running 2 different DAGs, is it possible to
specify the 2 DAGs running under 2 different ids ?
Any advises are welcomed.
Thanks.
Michael
ortant thing is that
every Airflow process that is part of the same Airflow cluster should be
configured similarly (point to the same metadata db, point to the same DAGS
folder, have the same executor configuration, ...)
On Thu, Jan 5, 2017 at 7:32 AM, Michael Gong wrote:
> Hi,
>
>
> He
Hi,
Here is the cases:
I have 2 separate airflow instances: 2 dag instance, 2 airflow server, 2
airflow webservers.
But they share the same MySQL db.
>From the 1st airflow webserver, I can see the dag instances from the 2nd
>airflow.
But it is greyed out , with a message:
"This DAG isn'
bject: Re: airflow dags share the same database?
Some suggestions:
* SQLSensor pointing to the other database
* Put a file somewhere, where it is being polled by the other dag/task
* Trigger the dag1 from task2 (easier with the Rest API in master)
Bolke
> On 4 Jan 2017, at 21:08, Michael
Hi,
Say due to some policy, I have to run 2 dag instances on 2 different servers.
If I want task 1 from dag1 depends on task 2 from dag2, how can I do it ?
One idea I can think of is that the 2 dag instances share the same MySQL db,
will it work ?
Any ideas are welcomed.
thanks.
Micha
7, 2016 at 11:48 AM, Michael Gong wrote:
> Rob,
>
> thanks for reply.
>
>
> If I have multiple installations of airflow on different servers, will
> this dependency be done ?
>
>
> For the second setup, ie. 1 installation of airflow and it is distributed,
> can you
workers on their own nodes?
If the latter, ExternalTaskSensor should still do the trick for you, as I
believe it just checks airflow's operational database for the status of
this external task.
On Thu, Nov 17, 2016 at 10:40 AM, Michael Gong wrote:
> Hi,
>
>
> Is it possibl
Hi,
Is it possible that 1 task can depend on another task running on another DAG on
another sever?
For example, I have task1 in dag1, and task2 in dag2.
If both dag1 and dag2 running on the same airflow instance on the same server,
airflow provides the ExternalTaskSensor operator.
But if
http://alembic.zzzcomputing.com/en/latest/tutorial.html
Max
On Wed, Nov 2, 2016 at 8:45 AM, Michael Gong wrote:
> Hi,
>
>
> I am trying to add a new table to airflow db. Airflow is using
> sqlAlchemy to manage the db.
>
> But I am not familiar with that.
>
>
> Can anyone provide some guidance ?
>
>
> Thanks.
>
> Michael
>
Hi,
I am trying to add a new table to airflow db. Airflow is using sqlAlchemy to
manage the db.
But I am not familiar with that.
Can anyone provide some guidance ?
Thanks.
Michael
Hi,
I have a MySQL table, which will be stored some static information. The
information could be different for different airflow runs, so I hope to use
python code to initialize it whenever airflow starts.
Where is the best place to put such code ?
Is the class DagBag's __init__() a good can
Hi,
When I clear the downstream tasks of 1 task from 1 dag, can it also clear its
downstream in another dag ?
For example:
dag1: task1
dag2: task2 (which is an ExternalTaskSensor, and depends on task1)
When I clear task1's downstream, is it possible that task2 is also cleared ?
Any comm
22 matches
Mail list logo
