Committers, please resolve any open Cold Case PRs that you still have on
your list below :
https://cwiki.apache.org/confluence/display/AIRFLOW/Cold-Case+PR+Resolution
In the past 6 weeks, we've cleaned up over 70 (from over 110 down to 36! We
have 16 more to go to get to our goal of ~20.
Dan, Max
Thanks very much for the detailed reply Maxime,
Like you suggested, I had solved the issue by adding an unused import of an
airflow class. It took me a while to get to this solution though. I had to
look into the airflow code to the place you pointed to realize that this
import was necessary to ad
Chris and WePayEng,
Thanks for hosting another great Airflow meet-up.
Can all of the speakers post their slides online and add links to those
talks in response to this email (and also on our Wiki)?
-s
Here, I added warning log messages and docs about it:
https://github.com/apache/incubator-airflow/pull/1889
Max
On Thu, Nov 17, 2016 at 2:45 AM, Tamara Mendt wrote:
> Thanks very much for the detailed reply Maxime,
>
> Like you suggested, I had solved the issue by adding an unused import of an
We'll be posting the video recording shortly. IT is working on it. :)
Will post link on the meetup and mailing list.
On Thu, Nov 17, 2016 at 10:12 AM, Siddharth Anand
wrote:
> Chris and WePayEng,
> Thanks for hosting another great Airflow meet-up.
>
> Can all of the speakers post their slides on
Hi,
Is it possible that 1 task can depend on another task running on another DAG on
another sever?
For example, I have task1 in dag1, and task2 in dag2.
If both dag1 and dag2 running on the same airflow instance on the same server,
airflow provides the ExternalTaskSensor operator.
But if
Here are our slides:
https://docs.google.com/presentation/d/1NG1P86HRlX43qTVucCTOsFqIbCvYdOhq_np90VlbVRc/edit?usp=sharing
I don't think I have permissions to edit the wiki
On Thu, Nov 17, 2016 at 10:41 AM, Chris Riccomini
wrote:
> We'll be posting the video recording shortly. IT is working on i
Are you saying you have multiple installations of Airflow, or simply that
your airflow setup is distributed, with celery workers on their own nodes?
If the latter, ExternalTaskSensor should still do the trick for you, as I
believe it just checks airflow's operational database for the status of
thi
http://blog.gitter.im/2016/11/17/join-the-dark-side/
Rob,
Wiki Access granted.
-s
On Thu, Nov 17, 2016 at 10:49 AM, Rob Froetscher
wrote:
> Here are our slides:
> https://docs.google.com/presentation/d/1NG1P86HRlX43qTVucCTOsFqIbCvYd
> Ohq_np90VlbVRc/edit?usp=sharing
>
> I don't think I have permissions to edit the wiki
>
> On Thu, Nov 17, 2016 at
Rob,
thanks for reply.
If I have multiple installations of airflow on different servers, will this
dependency be done ?
For the second setup, ie. 1 installation of airflow and it is distributed, can
you advise how to set it up ? I am not familiar with that.
Thanks.
_
We are thinking about the same thing. We are going to write a custom plugins
similar to the ExternalTaskSensor and check for the task success on the
different backend database.
Please let me know if you have better idea.
Thanks,
Catherine
On 11/17/16, 11:48 AM, "Michael Gong" wrote:
Rob,
Hey Michael,
If you have separate installations with separate operational databases,
then the ExternalTaskSensor will not do what you need. However, I'm not
sure I understand why you would need separate Airflow installations that
rely upon each other. Using Celery, you should be able to scale a si
Thanks, I've added the slides to the links page. I didn't see a specific
page for that meetup for slides in general.
On Thu, Nov 17, 2016 at 11:39 AM, siddharth anand wrote:
> Rob,
> Wiki Access granted.
> -s
>
> On Thu, Nov 17, 2016 at 10:49 AM, Rob Froetscher <
> rfroetsc...@lumoslabs.com>
> w
Good Afternoon,
I have implemented an authentication backend for Airflow to authenticate
against an Okta server. Would this be something that is worth adding into
Airflow?
Brian Yang
Yes! We were just about to start writing one of our own.
-Jakob
On 17 November 2016 at 14:30, Brian Yang wrote:
> Good Afternoon,
>
> I have implemented an authentication backend for Airflow to authenticate
> against an Okta server. Would this be something that is worth adding into
> Airflow?
>
Cool! Submit a PR.. and Jakob can help test it and merge it :-) Always a
positive when there are multiple members of the community using the same
integrations.
-s
On Thu, Nov 17, 2016 at 2:31 PM, Jakob Homan wrote:
> Yes! We were just about to start writing one of our own.
>
> -Jakob
>
> On 17
That's as good a place as any. You are also welcome to tweet your slides
out as well (and mention @ApacheAirflow) so we can retweet via the Apache
Airflow twitter account.
-s
On Thu, Nov 17, 2016 at 2:04 PM, Rob Froetscher
wrote:
> Thanks, I've added the slides to the links page. I didn't see a
*This is a brainstorm email thread about Airflow 2.0!*
I wanted to share some ideas around what I would like to do in Airflow 2.0
and would love to hear what others are thinking. I'll compile the ideas
that are shared in this thread in a Wiki once the conversation fades.
-
As we near the last month of the year and the 9th month in Apache
Incubation, I'd like to share some thoughts and gauge community feedback.
At the beginning of this year, we saw a good deal of interest in Airbnb's
Airflow project. As more companies started using it for business critical
work, it b
20 matches
Mail list logo
