I have a simple pipeline that consists of of two tasks. The second task
fetches an FTP file and stores on Google Cloud. The first task is an
external dag that waits for the file to be ready. I am wondering if I can
pass the execution date to the external dag. Right now, my code uses the
dag.start_
On 2019/04/01 17:14:28, Maxime Beauchemin wrote:
> Hey!
>
> I wanted to point out that there's awesome work taking place in FAB around
> a new REST API provided by the framework, and ways to extend it.
>
> Daniel Gaspar (cced) is working on this currently, and looking for input on
> design /
I'll comment on the PR tomorrow, but from a quick scan i didn't see anything
about OpenAPI/Swagger which would be a great addition.
-a
On 1 April 2019 18:14:28 BST, Maxime Beauchemin
wrote:
>Hey!
>
>I wanted to point out that there's awesome work taking place in FAB
>around
>a new REST API pro
Hey!
I wanted to point out that there's awesome work taking place in FAB around
a new REST API provided by the framework, and ways to extend it.
Daniel Gaspar (cced) is working on this currently, and looking for input on
design / implementation.
Check it out and chime in on the PR
https://github
Awesome and Thanks, Ash
Best wish.
-- Jiajie
From: Ash Berlin-Taylor
Sent: Monday, April 1, 2019 23:43
To: dev@airflow.apache.org
Subject: 1.10.3beta2 "snapshot" for available testing
Hi Everyone,
Following on form 1.10.3b1 I've pulled in another 84 (!) commits,
Awesome and Thanks for what you did.
Best wish.
-- Jiajie
From: Ash Berlin-Taylor
Sent: Monday, April 1, 2019 23:43
To: dev@airflow.apache.org
Subject: 1.10.3beta2 "snapshot" for available testing
Hi Everyone,
Following on form 1.10.3b1 I've pulled in another 84
Hi Everyone,
Following on form 1.10.3b1 I've pulled in another 84 (!) commits, mostly around
doc fixes or code-reorg. I will start the RC process this week.
In order to distinguish it from an actual (apache) release it is:
1. Marked as beta (python package managers do not install beta versions
That got fixed - thanks for the reminder :)
> On 29 Mar 2019, at 17:07, Jiajie Zhong wrote:
>
> Awesome Ash ??, expecting beta2.
> And don't forget some minor bug still in beta1
>
> https://issues.apache.org/jira/browse/AIRFLOW-3623?focusedCommentId=16803866&page=com.atlassian.jira.plugin.syst
If this is a common pattern, then I think it's worth considering the level
of effort to bake this into airflow, both for developer convenience and
safety. If it's baked into airflow, a developer working on dags in the
cluster will be saved from accidentally overriding important callbacks that
feed
I'm not opposed to the idea of building on the existing statsd metrics, but
they are really not anywhere close to the richness and configurability that
I think I would want for my use case.
If you look at the existing metrics, they are all about looking at the
health of an airflow cluster in aggre
Hey Stephan, not sure if you've seen it or not, but Airflow has some built
in support for exporting statsd metrics. We run everything in Kubernetes
and are also heavy users of Prometheus. We've had a pretty solid experience
using the statsd-exporter (https://github.com/prometheus/statsd_exporter),
The recording has been uploaded here:
https://www.youtube.com/watch?v=UBtXaPEg9yE
The text may not be clear at various points, sorry about that; I'll try and get
around to writing up a blog post or two of things.
-ash
> On 30 Mar 2019, at 05:07, Kevin Yang wrote:
>
> Thanks Ash, love to see
Great news! Happy to remove the Python 2 code ;-)
Btw that list (https://jira.apache.org/jira/browse/AIRFLOW-4196) was a
brain-dump. If there’s anything missing, feel free to create more issues.
Bas
On 1 Apr 2019, at 11:03, Driesprong, Fokko
mailto:fo...@driesprong.frl>> wrote:
Hi all,
I'm h
> For example, say we want to support retention in some Airflow tables such
> as task_instance, dag_run and log, it seems reasonable to me to create a
> DAG to periodically clean up the tables
I guess you mean something like
https://github.com/teamclairvoyant/airflow-maintenance-dags/tree/master/
Hi all,
I'm happy to inform you all that the vote has been passed! From Airflow 2.0
we will drop support for Python 2 (and this is not an April Fools' joke :-).
With a total 18 +1 votes, and a single +0 vote (at least a positive zero)
from Ash.
With the input of Bas Harenslak, I've composed a no
15 matches
Mail list logo