Re: Subsequent Airflow Meetup: 2017/01/11

Hi George, Confirmed - would like give a talk. Thanks, Kevin Mandich On Tue, Jan 3, 2017 at 5:40 AM, Dan Davydov wrote: > Confirmed. > > On Sun, Jan 1, 2017 at 9:16 PM, George Leslie-Waksman < > geo...@cloverhealth.com.invalid> wrote: > > > Sorry for the delayed response, end of year and holid

Re: Airflow 1.8.0 Alpha 1

Hey Bolke, Thanks for taking this on. I'm definitely up for running stuff in our environments to verify everything is working. Can I ask that you create a 1.8 alpha 1 branch in the git repo? This will make it easier for us to track what changes are getting cherry picked into the branch, and will

Re: (INFRA-13207) Please create Airflow in /repos/dist/release/incubator/

Hey Bolke, I've created it: https://dist.apache.org/repos/dist/release/incubator/airflow I've also added a KEYS file with my PGP key in it. Cheers, Chris On Sun, Jan 1, 2017 at 3:08 AM, Bolke de Bruin wrote: > Jakob, Chris, > > Can you please create create ‘airflow’ in /repos/dist/release/

Re: Airflow 1.8.0 Alpha 1

All very reasonable to me, one reason we may not have hit the bugs in our production is because we are running off a different merge base and our cherries aren't 1-1 with what we are running in production (we still test them but we can't run them in production), that being said I don't think I auth

Re: Airflow 1.8.0 Alpha 1

Hi Dan et al, That sounds good to me, however I will be pretty critical of the changes in the scheduler and the cleanliness of the patches. This is due to the fact I have been chasing quite some bugs in master that were pretty hard to track down even with a debugger at hand. I’m surprised that

Re: Airflow 2.0

Right, this conversation turned into more of a wishlist / roadmap brainstorm which is also very useful. Let's treat this thread for the value it has from that perspective Seems like very few things needed are breaking changes. Re-packaging (breaking Airflow into smaller python packages) would be t

Re: Jan 2017 Airflow Podling Report Posted

Thank you for putting together the report. I have entered my mentor sign-off. On a side note, it's very exciting to see the push for an Apache release as the new year begins! Chris Nauroth On Sat, Dec 31, 2016 at 12:58 PM, siddharth anand wrote: > Hi Folks! > Here's the quarterly Podling Repo

Re: Python 3

Per Celery's documentation , Redis and RabbitMQ are the best choices. We use Redis at Airbnb and it's been a breeze, I don't think we've ever had any issues coming from that layer.

Re: Airflow 1.8.0 Alpha 1

@bolke thanks for taking the lead on this. Let's get these cherries in master and move forward with a release. Happy new year everyone! Max On Tue, Jan 3, 2017 at 7:30 AM, Alex Van Boxel wrote: > If they should make the first alpha, maybe they should be rebased so they > can be merged in. > >

Re: Jan 2017 Airflow Podling Report Posted

Thanks for taking the time to do this while most were on the holiday break! Max On Sat, Dec 31, 2016 at 12:58 PM, siddharth anand wrote: > Hi Folks! > Here's the quarterly Podling Report for Apache Airflow. Feel free to > suggest edits. If you are a committer/maintainer, you can directly edit i

Re: Airflow 1.8.0 Alpha 1

If they should make the first alpha, maybe they should be rebased so they can be merged in. On Tue, Jan 3, 2017 at 2:39 PM Dan Davydov wrote: > I have also started on this effort, recently Alex Guziel and I have been > pushing Airbnb's custom cherries onto master to get Airbnb back onto master >

Re: Airflow Github Mirror is not synchronizing

The repo mirror is syncing now. -s On Sat, Dec 31, 2016 at 11:22 AM siddharth anand wrote: > FYI! > I've reopened my earlier JIRA issue. It looks like multiple Apache > Projects are reporting the same. > https://issues.apache.org/jira/browse/INFRA-12949 > > Newly merged changes won't be available

Re: Python 3

We are using Airflow with Python 3.5 and RabbitMQ (CeleryExecutor). We're using Celery version 3.1.24. For the BROKER_URL connection string we use "pyamqp://:@host:5671/vhost" and for CELERY_RESULT_BACKEND we use "rpc://". Is Redis the preferred/better supported backend? Cheers, Conor Nash Data

Re: Subsequent Airflow Meetup: 2017/01/11

Confirmed. On Sun, Jan 1, 2017 at 9:16 PM, George Leslie-Waksman < geo...@cloverhealth.com.invalid> wrote: > Sorry for the delayed response, end of year and holidays stole my attention > for a bit. > > With the new year, I was just looking to pick things back up and solicit > presenters for the m

Re: Airflow 1.8.0 Alpha 1

I have also started on this effort, recently Alex Guziel and I have been pushing Airbnb's custom cherries onto master to get Airbnb back onto master in order for us to do a release. I think it might make sense to wait for these two commits to get merged in since they would be quite nice to have fo

Re: Airflow 1.8.0 Alpha 1

Hey Alex, I have noticed the same, and it is also the reason why we have Alpha versions. For now I have noticed the following: * Tasks can get in limbo between scheduler and executor: https://github.com/apache/incubator-airflow/pull/1948