Re: NYC Airflow Meetup

[siddharth anand]

Great!
Thanks for creating it - I've just joined so you can add me as an
organizer.

I've linked to it on :

   - https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
   - https://twitter.com/ApacheAirflow/status/827743162789605382
   - https://cwiki.apache.org/confluence/display/AIRFLOW/Meetups
   -


-s

On Fri, Feb 3, 2017 at 11:01 AM, Joseph Napolitano <
joseph.napolit...@blueapron.com.invalid> wrote:

> Hi all,
>
> I want to thank everyone for attending NYC's first Airflow meetup at Blue
> Apron.  It was a huge success and we're glad to have met everyone.
>
> As suggested, we decided to create an official NYC Meetup page, sponsored
> by Blue Apron.  We'll add Sid and Max as Organizers.  Let us know if you
> want to help organize.
>
> https://www.meetup.com/NYC-Apache-Airflow-incubating-Meetup/
>
> I planned on taking video of the presentations, but it completely slipped
> my mind!  I'll upload my slides to Slideshare and provide a small writeup
> to complement them.
>
> We're committed to Airflow at Blue Apron and we love the project.  Now that
> our infrastructure is taking shape, we'll have time to contribute back to
> the project.  We have top-down support at Blue Apron to dedicate company
> time for it.
>
> Feel free to connect anytime!
> https://www.linkedin.com/in/joenap
>
> Thanks again,
> *Joe Napolitano *| Sr. Data Engineer
> www.blueapron.com | 5 Crosby Street, New York, NY 10013
>

Re: NYC Airflow Meetup

[Jeremiah Lowin]

Thanks for hosting, Joe & Blue Apron! Great to meet so many folks and
looking forward to joining you again.

Jeremiah

On Fri, Feb 3, 2017 at 2:02 PM Joseph Napolitano
 wrote:

> Hi all,
>
> I want to thank everyone for attending NYC's first Airflow meetup at Blue
> Apron.  It was a huge success and we're glad to have met everyone.
>
> As suggested, we decided to create an official NYC Meetup page, sponsored
> by Blue Apron.  We'll add Sid and Max as Organizers.  Let us know if you
> want to help organize.
>
> https://www.meetup.com/NYC-Apache-Airflow-incubating-Meetup/
>
> I planned on taking video of the presentations, but it completely slipped
> my mind!  I'll upload my slides to Slideshare and provide a small writeup
> to complement them.
>
> We're committed to Airflow at Blue Apron and we love the project.  Now that
> our infrastructure is taking shape, we'll have time to contribute back to
> the project.  We have top-down support at Blue Apron to dedicate company
> time for it.
>
> Feel free to connect anytime!
> https://www.linkedin.com/in/joenap
>
> Thanks again,
> *Joe Napolitano *| Sr. Data Engineer
> www.blueapron.com | 5 Crosby Street, New York, NY 10013
>

Re: Airflow 1.8.0 Release Candidate 1

[Jeremiah Lowin]

For what it's worth -- everything running smoothly after 24+ hours in a
production(ish) environment.

On Thu, Feb 2, 2017 at 11:25 PM Jayesh Senjaliya 
wrote:

> Thank You Bolke for all the efforts you are putting in !!
>
> I have deployed this RC now.
>
> On Thu, Feb 2, 2017 at 3:02 PM, Jeremiah Lowin  wrote:
>
> > Fantastic work on this Bolke, thank you!
> >
> > We've deployed the RC and will report if there are any issues...
> >
> > On Thu, Feb 2, 2017 at 4:32 PM Bolke de Bruin  wrote:
> >
> > > Now I am blushing :-)
> > >
> > > Sent from my iPhone
> > >
> > > > On 2 Feb 2017, at 22:05, Boris Tyukin  wrote:
> > > >
> > > > LOL awesome!
> > > >
> > > > On Thu, Feb 2, 2017 at 4:00 PM, Maxime Beauchemin <
> > > > maximebeauche...@gmail.com> wrote:
> > > >
> > > >> The Apache mailing doesn't support images so here's a link:
> > > >>
> > > >> http://i.imgur.com/DUkpjZu.png
> > > >> ​
> > > >>
> > > >> On Thu, Feb 2, 2017 at 12:52 PM, Boris Tyukin <
> bo...@boristyukin.com>
> > > >> wrote:
> > > >>
> > > >>> Bolke, you are our hero! I am sure you put a lot of your time to
> make
> > > it
> > > >>> happen
> > > >>>
> > > >>> On Thu, Feb 2, 2017 at 2:50 PM, Bolke de Bruin 
> > > >> wrote:
> > > >>>
> > >  Hi All,
> > > 
> > >  I have made the (first) RELEASE CANDIDATE of Airflow 1.8.0
> available
> > > >> at:
> > >  https://dist.apache.org/repos/dist/dev/incubator/airflow/ ,
> public
> > > >> keys
> > >  are available at
> > > https://dist.apache.org/repos/dist/release/incubator/
> > >  airflow/ . It is tagged with a local version “apache.incubating”
> so
> > it
> > >  allows upgrading from earlier releases. This should be considered
> of
> > >  release quality, but not yet officially vetted as a release yet.
> > > 
> > >  Issues fixed:
> > >  * Use static nvd3 and d3
> > >  * Python 3 incompatibilities
> > >  * CLI API trigger dag issue
> > > 
> > >  As the difference between beta 5 and the release candidate is
> > > >> relatively
> > >  small I hope to start the VOTE for releasing 1.8.0 quite soon (2
> > > >> days?),
> > > >>> if
> > >  the vote passes also a vote needs to happen at the IPMC
> mailinglist.
> > > As
> > >  this is our first Apache release I expect some comments and
> required
> > >  changes and probably a RC 2.
> > > 
> > >  Furthermore, we now have a “v1-8-stable” branch. This has version
> > >  “1.8.0rc1” and will graduate to “1.8.0” when we release. The
> > > >> “v1-8-test”
> > >  branch now has version “1.8.1alpha0” as version and “master” has
> > > >> version
> > >  “1.9.0dev0”. Note that “v1-8-stable” is now closed. This means
> that,
> > > >> per
> > >  release guidelines, patches accompanied with an ASSIGNED Jira and
> a
> > >  sign-off from a committer. Only then the release manager applies
> the
> > > >>> patch
> > >  to stable (In this case that would be me). The release manager
> then
> > > >>> closes
> > >  the bug when the patches have landed in the appropriate branches.
> > For
> > > >>> more
> > >  information please see: https://cwiki.apache.org/
> > >  confluence/display/AIRFLOW/Airflow+Release+Planning+and+
> > >  Supported+Release+Lifetime  > >  confluence/display/AIRFLOW/Airflow+Release+Planning+and+
> > >  Supported+Release+Lifetime> .
> > > 
> > >  Any questions or suggestions don’t hesitate to ask!
> > > 
> > >  Cheers
> > >  Bolke
> > > >>>
> > > >>
> > >
> >
>

NYC Airflow Meetup

[Joseph Napolitano]

Hi all,

I want to thank everyone for attending NYC's first Airflow meetup at Blue
Apron.  It was a huge success and we're glad to have met everyone.

As suggested, we decided to create an official NYC Meetup page, sponsored
by Blue Apron.  We'll add Sid and Max as Organizers.  Let us know if you
want to help organize.

https://www.meetup.com/NYC-Apache-Airflow-incubating-Meetup/

I planned on taking video of the presentations, but it completely slipped
my mind!  I'll upload my slides to Slideshare and provide a small writeup
to complement them.

We're committed to Airflow at Blue Apron and we love the project.  Now that
our infrastructure is taking shape, we'll have time to contribute back to
the project.  We have top-down support at Blue Apron to dedicate company
time for it.

Feel free to connect anytime!
https://www.linkedin.com/in/joenap

Thanks again,
*Joe Napolitano *| Sr. Data Engineer
www.blueapron.com | 5 Crosby Street, New York, NY 10013

Re: Airflow Meetup @ Paypal (San Jose)

[Jayesh Senjaliya]

yeah, I think we should have both. since we only have 2 presentations,
we will have plenty of time for round table and Q&A.

- Jayesh

On Fri, Feb 3, 2017 at 9:49 AM, Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:

> I like the idea of a panel talking about direction for the project. The
> panel could be taking questions from a moderator and from the audience. I'm
> open sitting on the panel.
>
> I could also give a talk on our A/B testing framework's complex DAG if I
> can't get the engineers working on it to do it :)
>
> Maybe we can do both, and just make the Q/A section we usually have a panel
> Q/A instead of a Max Q/A.
>
> Max
>
> On Fri, Feb 3, 2017 at 5:07 AM, Bolke de Bruin  wrote:
>
> > I might. But I would maybe be more interested in a kind of round table /
> > panel session to discuss directions? Does that make sense? Or would you
> > like me to talk about a specific subject?
> >
> > - Bolke.
> >
> > > On 3 Feb 2017, at 03:42, siddharth anand  wrote:
> > >
> > > Cool! I've tweeted it out using the ApacheAirflow account and also
> added
> > it
> > > to https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
> > >
> > > FYI,
> > > I was mistaken about the drive between Strata and PayPal. I had used
> the
> > > wrong venue. Strata is at the SJ Convention Center this year.  It's
> still
> > > very close.. 5 miles (12 minutes - reverse commute).
> > >
> > > https://goo.gl/maps/nwxmkYsNFKQ2
> > >
> > > BTW, I heard Bolke may be attending ;-) Bolke, would you like to speak
> at
> > > the Meetup?
> > >
> > > Jakob (other committers), will be down here for Strata?
> > >
> > > -s
> > >
> > > On Thu, Feb 2, 2017 at 5:02 PM, Jayesh Senjaliya 
> > > wrote:
> > >
> > >> Sure,
> > >> I have created event on Meetup :
> > >> https://www.meetup.com/Bay-Area-Apache-Airflow-
> > Incubating-Meetup/events/
> > >> 237412864/
> > >>
> > >> Thanks for helping on this Siddharth.
> > >> Jayesh
> > >>
> > >>
> > >>
> > >> On Wed, Feb 1, 2017 at 7:50 PM, siddharth anand 
> > wrote:
> > >>
> > >>> IMHO, I'd publish the meet-up. You still have 6 weeks to find a 3rd
> > >>> speaker. If Bolke and Alex are traveling all the way for Strata,
> > perhaps
> > >>> one of them can speak :-)
> > >>>
> > >>> -s
> > >>>
> > >>> On Wed, Feb 1, 2017 at 1:48 PM, Russell Jurney <
> > russell.jur...@gmail.com
> > >>>
> > >>> wrote:
> > >>>
> >  Maybe start a new thread with a title "Call for Speakers for Meetup
> on
> > >>> Mar
> >  14" ?
> > 
> >  On Wed, Feb 1, 2017 at 11:59 AM Jayesh Senjaliya <
> jhsonl...@gmail.com
> > >
> >  wrote:
> > 
> > > Yes, we are still waiting for more speakers.
> > >
> > > can anybody from Airbnb present ?
> > >
> > > anybody else ?
> > >
> > >
> > > - Jayesh
> > >
> > > On Tue, Jan 31, 2017 at 8:16 PM, siddharth anand <
> san...@apache.org>
> > > wrote:
> > >
> > >> Jayesh,
> > >> Looks good. No need to vote. Just publish a new event with details
> > >> on
> >  the
> > >> meet-up page:
> > >> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/
> > >>
> > >> Please add a short abstract as well for the talks and find a 3rd
> >  speaker.
> > >> Please be sure to record the meet-up so that we can publish it.
> > >> Once
> >  the
> > >> meet-up event is up, please respond to this email! We can help
> > >>> promote
> > > it.
> > >> I suggest picking a start time after the Strata talks end but not
> > >>> super
> > >> late either.
> > >>
> > >> -s
> > >>
> > >> On Tue, Jan 31, 2017 at 9:19 AM, Jayesh Senjaliya <
> > >>> jhsonl...@gmail.com
> > >
> > >> wrote:
> > >>
> > >>> HI All,
> > >>>
> > >>> √ I have approval from Paypal to host Airflow meetup.  How about
> >  March
> > >> 14th
> > >>> ? Please vote.
> > >>>
> > >>> √ we will have food and drinks.
> > >>> Please let me know if anybody has any special request, I will try
> > >>> to
> > >>> accommodate :)
> > >>>
> > >>> For presentations:
> > >>> 1) Disk recommission using airflow with overall automation of
> >  "Hadoop
> > >> Node
> > >>> and Disk Remediation". - Jayesh Senjaliya ( Paypal )
> > >>> 2) Predictive Analytics with Airflow and PySpark - ( Russell
> > >>> Jurney
> >  )
> > >>>
> > >>>
> > >>> Please send request to present to this email thread if you are
> > > interested
> > >>> in presenting.
> > >>>
> > >>> Thanks
> > >>> Jayesh
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Jan 26, 2017 at 4:08 PM, Russell Jurney <
> > >> russell.jur...@gmail.com>
> > >>> wrote:
> > >>>
> >  Cool!
> > 
> >  On Wed, Jan 25, 2017 at 11:23 PM Jayesh Senjaliya <
> > > jhsonl...@gmail.com
> > >>>
> >  wrote:
> > 
> > > Hi Russell,
> > >
> > > yes, I will be presenting from Paypa

Re: Airflow Meetup @ Paypal (San Jose)

[Maxime Beauchemin]

I like the idea of a panel talking about direction for the project. The
panel could be taking questions from a moderator and from the audience. I'm
open sitting on the panel.

I could also give a talk on our A/B testing framework's complex DAG if I
can't get the engineers working on it to do it :)

Maybe we can do both, and just make the Q/A section we usually have a panel
Q/A instead of a Max Q/A.

Max

On Fri, Feb 3, 2017 at 5:07 AM, Bolke de Bruin  wrote:

> I might. But I would maybe be more interested in a kind of round table /
> panel session to discuss directions? Does that make sense? Or would you
> like me to talk about a specific subject?
>
> - Bolke.
>
> > On 3 Feb 2017, at 03:42, siddharth anand  wrote:
> >
> > Cool! I've tweeted it out using the ApacheAirflow account and also added
> it
> > to https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
> >
> > FYI,
> > I was mistaken about the drive between Strata and PayPal. I had used the
> > wrong venue. Strata is at the SJ Convention Center this year.  It's still
> > very close.. 5 miles (12 minutes - reverse commute).
> >
> > https://goo.gl/maps/nwxmkYsNFKQ2
> >
> > BTW, I heard Bolke may be attending ;-) Bolke, would you like to speak at
> > the Meetup?
> >
> > Jakob (other committers), will be down here for Strata?
> >
> > -s
> >
> > On Thu, Feb 2, 2017 at 5:02 PM, Jayesh Senjaliya 
> > wrote:
> >
> >> Sure,
> >> I have created event on Meetup :
> >> https://www.meetup.com/Bay-Area-Apache-Airflow-
> Incubating-Meetup/events/
> >> 237412864/
> >>
> >> Thanks for helping on this Siddharth.
> >> Jayesh
> >>
> >>
> >>
> >> On Wed, Feb 1, 2017 at 7:50 PM, siddharth anand 
> wrote:
> >>
> >>> IMHO, I'd publish the meet-up. You still have 6 weeks to find a 3rd
> >>> speaker. If Bolke and Alex are traveling all the way for Strata,
> perhaps
> >>> one of them can speak :-)
> >>>
> >>> -s
> >>>
> >>> On Wed, Feb 1, 2017 at 1:48 PM, Russell Jurney <
> russell.jur...@gmail.com
> >>>
> >>> wrote:
> >>>
>  Maybe start a new thread with a title "Call for Speakers for Meetup on
> >>> Mar
>  14" ?
> 
>  On Wed, Feb 1, 2017 at 11:59 AM Jayesh Senjaliya  >
>  wrote:
> 
> > Yes, we are still waiting for more speakers.
> >
> > can anybody from Airbnb present ?
> >
> > anybody else ?
> >
> >
> > - Jayesh
> >
> > On Tue, Jan 31, 2017 at 8:16 PM, siddharth anand 
> > wrote:
> >
> >> Jayesh,
> >> Looks good. No need to vote. Just publish a new event with details
> >> on
>  the
> >> meet-up page:
> >> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/
> >>
> >> Please add a short abstract as well for the talks and find a 3rd
>  speaker.
> >> Please be sure to record the meet-up so that we can publish it.
> >> Once
>  the
> >> meet-up event is up, please respond to this email! We can help
> >>> promote
> > it.
> >> I suggest picking a start time after the Strata talks end but not
> >>> super
> >> late either.
> >>
> >> -s
> >>
> >> On Tue, Jan 31, 2017 at 9:19 AM, Jayesh Senjaliya <
> >>> jhsonl...@gmail.com
> >
> >> wrote:
> >>
> >>> HI All,
> >>>
> >>> √ I have approval from Paypal to host Airflow meetup.  How about
>  March
> >> 14th
> >>> ? Please vote.
> >>>
> >>> √ we will have food and drinks.
> >>> Please let me know if anybody has any special request, I will try
> >>> to
> >>> accommodate :)
> >>>
> >>> For presentations:
> >>> 1) Disk recommission using airflow with overall automation of
>  "Hadoop
> >> Node
> >>> and Disk Remediation". - Jayesh Senjaliya ( Paypal )
> >>> 2) Predictive Analytics with Airflow and PySpark - ( Russell
> >>> Jurney
>  )
> >>>
> >>>
> >>> Please send request to present to this email thread if you are
> > interested
> >>> in presenting.
> >>>
> >>> Thanks
> >>> Jayesh
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Jan 26, 2017 at 4:08 PM, Russell Jurney <
> >> russell.jur...@gmail.com>
> >>> wrote:
> >>>
>  Cool!
> 
>  On Wed, Jan 25, 2017 at 11:23 PM Jayesh Senjaliya <
> > jhsonl...@gmail.com
> >>>
>  wrote:
> 
> > Hi Russell,
> >
> > yes, I will be presenting from Paypal side.
> > Once i have official approval from Paypal, I will sent out
> >>> email.
> > I am basically going by the steps what Siddharth outlined
> >>> earlier
> > in
> >>> the
> > thread.
> >
> > Thanks
> > Jayesh
> >
> > On Wed, Jan 25, 2017 at 7:50 PM, Russell Jurney <
>  russell.jur...@gmail.com>
> > wrote:
> >
> >> Is someone from Paypal likely to speak? Should we start a
> >> new
> >> thread
> > with a
> >> call for another speaker? There was mention of three bei

Re: Airflow Meetup @ Paypal (San Jose)

[Bolke de Bruin]

I might. But I would maybe be more interested in a kind of round table / panel 
session to discuss directions? Does that make sense? Or would you like me to 
talk about a specific subject?

- Bolke.

> On 3 Feb 2017, at 03:42, siddharth anand  wrote:
> 
> Cool! I've tweeted it out using the ApacheAirflow account and also added it
> to https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
> 
> FYI,
> I was mistaken about the drive between Strata and PayPal. I had used the
> wrong venue. Strata is at the SJ Convention Center this year.  It's still
> very close.. 5 miles (12 minutes - reverse commute).
> 
> https://goo.gl/maps/nwxmkYsNFKQ2
> 
> BTW, I heard Bolke may be attending ;-) Bolke, would you like to speak at
> the Meetup?
> 
> Jakob (other committers), will be down here for Strata?
> 
> -s
> 
> On Thu, Feb 2, 2017 at 5:02 PM, Jayesh Senjaliya 
> wrote:
> 
>> Sure,
>> I have created event on Meetup :
>> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/
>> 237412864/
>> 
>> Thanks for helping on this Siddharth.
>> Jayesh
>> 
>> 
>> 
>> On Wed, Feb 1, 2017 at 7:50 PM, siddharth anand  wrote:
>> 
>>> IMHO, I'd publish the meet-up. You still have 6 weeks to find a 3rd
>>> speaker. If Bolke and Alex are traveling all the way for Strata, perhaps
>>> one of them can speak :-)
>>> 
>>> -s
>>> 
>>> On Wed, Feb 1, 2017 at 1:48 PM, Russell Jurney >> 
>>> wrote:
>>> 
 Maybe start a new thread with a title "Call for Speakers for Meetup on
>>> Mar
 14" ?
 
 On Wed, Feb 1, 2017 at 11:59 AM Jayesh Senjaliya 
 wrote:
 
> Yes, we are still waiting for more speakers.
> 
> can anybody from Airbnb present ?
> 
> anybody else ?
> 
> 
> - Jayesh
> 
> On Tue, Jan 31, 2017 at 8:16 PM, siddharth anand 
> wrote:
> 
>> Jayesh,
>> Looks good. No need to vote. Just publish a new event with details
>> on
 the
>> meet-up page:
>> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/
>> 
>> Please add a short abstract as well for the talks and find a 3rd
 speaker.
>> Please be sure to record the meet-up so that we can publish it.
>> Once
 the
>> meet-up event is up, please respond to this email! We can help
>>> promote
> it.
>> I suggest picking a start time after the Strata talks end but not
>>> super
>> late either.
>> 
>> -s
>> 
>> On Tue, Jan 31, 2017 at 9:19 AM, Jayesh Senjaliya <
>>> jhsonl...@gmail.com
> 
>> wrote:
>> 
>>> HI All,
>>> 
>>> √ I have approval from Paypal to host Airflow meetup.  How about
 March
>> 14th
>>> ? Please vote.
>>> 
>>> √ we will have food and drinks.
>>> Please let me know if anybody has any special request, I will try
>>> to
>>> accommodate :)
>>> 
>>> For presentations:
>>> 1) Disk recommission using airflow with overall automation of
 "Hadoop
>> Node
>>> and Disk Remediation". - Jayesh Senjaliya ( Paypal )
>>> 2) Predictive Analytics with Airflow and PySpark - ( Russell
>>> Jurney
 )
>>> 
>>> 
>>> Please send request to present to this email thread if you are
> interested
>>> in presenting.
>>> 
>>> Thanks
>>> Jayesh
>>> 
>>> 
>>> 
>>> 
>>> On Thu, Jan 26, 2017 at 4:08 PM, Russell Jurney <
>> russell.jur...@gmail.com>
>>> wrote:
>>> 
 Cool!
 
 On Wed, Jan 25, 2017 at 11:23 PM Jayesh Senjaliya <
> jhsonl...@gmail.com
>>> 
 wrote:
 
> Hi Russell,
> 
> yes, I will be presenting from Paypal side.
> Once i have official approval from Paypal, I will sent out
>>> email.
> I am basically going by the steps what Siddharth outlined
>>> earlier
> in
>>> the
> thread.
> 
> Thanks
> Jayesh
> 
> On Wed, Jan 25, 2017 at 7:50 PM, Russell Jurney <
 russell.jur...@gmail.com>
> wrote:
> 
>> Is someone from Paypal likely to speak? Should we start a
>> new
>> thread
> with a
>> call for another speaker? There was mention of three being
> needed.
>> 
>> On Wed, Jan 25, 2017 at 5:33 PM Jayesh Senjaliya <
>>> jhsonl...@gmail.com>
>> wrote:
>> 
>>> Yes I am waiting for response from facilities about it,
>>> most
>> likely
 by
>>> early next week.
>>> 
>>> Thanks
>>> Jayesh
>>> 
>>> On Wed, Jan 25, 2017 at 4:52 PM, Russell Jurney <
>> russell.jur...@gmail.com>
>>> wrote:
>>> 
 Boris, would you be able to attend an evening meetup on
>>> the
>>> nights
 of
>>> 3/15
 or 3/16? I think attendance would be better on one of
>>> those
>> days,
 as
>> many
 people don't attend the tutorial days.
>>>