Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Yes I think so. 

Bolke

Sent from my iPhone

> On 1 Aug 2017, at 00:59, Maxime Beauchemin  wrote:
> 
> Alright so here's the INSTALL file:
> https://github.com/apache/incubator-airflow/pull/2492
> 
> I'm thinking I'll cherry pick this in the 1.8-test branch and tar the whole
> repo, crank up the RC number and publish to the same location as before.
> 
> Am I on the right track?
> 
> Max
> 
> On Thu, Jul 27, 2017 at 11:12 AM, Chris Riccomini 
> wrote:
> 
>> Oky.. then I guess we can address the feedback above. Owe you some beers,
>> Max.
>> 
>> On Thu, Jul 27, 2017 at 10:56 AM, Bolke de Bruin 
>> wrote:
>> 
>>> Yep
>>> 
>>> Sent from my iPhone
>>> 
 On 27 Jul 2017, at 19:00, Maxime Beauchemin <
>> maximebeauche...@gmail.com>
>>> wrote:
 
 I need to re-package it with build instructions. I'm pretty sure this
>>> means
 another vote. I have time carved up to work on this today/tomorrow.
 
 Max
 
 On Thu, Jul 27, 2017 at 9:18 AM, Chris Riccomini <
>> criccom...@apache.org>
 wrote:
 
> IMO, given the level of effort for 1.8.2, and how long it's taken, we
> should not be re-voting right now unless something horrific happened
>> to
>>> the
> release.
> 
> On Tue, Jul 25, 2017 at 9:54 PM, Pavel Martynov 
> wrote:
> 
>> Hi, folks!
>> 
>> AIRFLOW-935 issue marked as resolved and fix version is 1.8.2, but
>> this
>> commit contained in master branch only and not tagged by 1.8.2rc2.
>> Can fix of this issue be released in 1.8.2?
>> 
>> 2017-07-26 2:27 GMT+03:00 George Leslie-Waksman <
>> geo...@cloverhealth.com.invalid>:
>> 
>>> I've checked and we are no longer relying on the previous
>>> LatestOnlyOperator behavior for any of our DAGs.
>>> 
>>> This is not a dealbreaker (though I will need to keep it in mind).
>>> 
>>> Thanks for asking,
>>> --George
>>> 
>>> On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
>>> maximebeauche...@gmail.com> wrote:
>>> 
 [AIRFLOW-1296] is part of 1.8.2.
 
 Is this a dealbreaker for 1.8.2?
 
 Max
 
 On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
 geo...@cloverhealth.com.invalid> wrote:
 
> I hope that it's not too late for me to chime in but there is a
>>> breaking
> change in the behavior of LatestOnlyOperator.
> 
> The change was introduced in
> PR: https://github.com/apache/incubator-airflow/pull/2365
> Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
>>> tasks
> 
> Prior to this change, the LatestOnlyOperator would skip direct
>>> downstream
> but not indirect downstream; now it skips indirect downstream.
> 
> This breaks the use of LatestOnlyOperator with TriggerRules that
>> do
>> not
> propagate skips.
> 
> --George
> 
> On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
> 
>> I think I'm gathering a good picture of what is expected here.
> I'll
>>> try
> to
>> update the Confluence page as I go.
>> 
>> I'm hoping to get started tomorrow and package it early next
> week.
>> 
>> Max
>> 
>> On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand <
>> san...@apache.org>
>> wrote:
>> 
>>> FYI, can anyone pictorially describe the release process (and
>> post
>>> it
> on
>>> the apache airflow wiki)? I think that would eliminate a lot of
> confusion
>>> in the future and avoid a rehash of this email thread on the
> next
>> release.
>>> 
>>> -s
>>> 
>>> On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah <
> hit...@apache.org
>>> 
> wrote:
>>> 
 To add, the main source tarball should have instructions to
 generate
>> the
 sdist and bdist versions. Additionally, as part of the
> release
> process
>> if
 the plan is to publish to pypi (after the IPMC vote
> succeeds),
>>> then
> the
 appropriate bits also need to be verified/voted upon. There
> are
>>> not
>>> exactly
 counted as the official release bits but they do need to be
 verified
> as
 part of the voting process to ensure that the bits do indeed
>> map
>>> to
> the
 source release, license/notice files are correct, etc.
 
 thanks
 -- Hitesh
 
 
 On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
 bdbr...@gmail.com>
 wrote:

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Maxime Beauchemin]

I need to re-package it with build instructions. I'm pretty sure this means
another vote. I have time carved up to work on this today/tomorrow.

Max

On Thu, Jul 27, 2017 at 9:18 AM, Chris Riccomini 
wrote:

> IMO, given the level of effort for 1.8.2, and how long it's taken, we
> should not be re-voting right now unless something horrific happened to the
> release.
>
> On Tue, Jul 25, 2017 at 9:54 PM, Pavel Martynov 
> wrote:
>
> > Hi, folks!
> >
> > AIRFLOW-935 issue marked as resolved and fix version is 1.8.2, but this
> > commit contained in master branch only and not tagged by 1.8.2rc2.
> > Can fix of this issue be released in 1.8.2?
> >
> > 2017-07-26 2:27 GMT+03:00 George Leslie-Waksman <
> > geo...@cloverhealth.com.invalid>:
> >
> > > I've checked and we are no longer relying on the previous
> > > LatestOnlyOperator behavior for any of our DAGs.
> > >
> > > This is not a dealbreaker (though I will need to keep it in mind).
> > >
> > > Thanks for asking,
> > > --George
> > >
> > > On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
> > > maximebeauche...@gmail.com> wrote:
> > >
> > > > [AIRFLOW-1296] is part of 1.8.2.
> > > >
> > > > Is this a dealbreaker for 1.8.2?
> > > >
> > > > Max
> > > >
> > > > On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
> > > > geo...@cloverhealth.com.invalid> wrote:
> > > >
> > > > > I hope that it's not too late for me to chime in but there is a
> > > breaking
> > > > > change in the behavior of LatestOnlyOperator.
> > > > >
> > > > > The change was introduced in
> > > > > PR: https://github.com/apache/incubator-airflow/pull/2365
> > > > > Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> > > tasks
> > > > >
> > > > > Prior to this change, the LatestOnlyOperator would skip direct
> > > downstream
> > > > > but not indirect downstream; now it skips indirect downstream.
> > > > >
> > > > > This breaks the use of LatestOnlyOperator with TriggerRules that do
> > not
> > > > > propagate skips.
> > > > >
> > > > > --George
> > > > >
> > > > > On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> > > > > maximebeauche...@gmail.com> wrote:
> > > > >
> > > > > > I think I'm gathering a good picture of what is expected here.
> I'll
> > > try
> > > > > to
> > > > > > update the Confluence page as I go.
> > > > > >
> > > > > > I'm hoping to get started tomorrow and package it early next
> week.
> > > > > >
> > > > > > Max
> > > > > >
> > > > > > On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand <
> > san...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > FYI, can anyone pictorially describe the release process (and
> > post
> > > it
> > > > > on
> > > > > > > the apache airflow wiki)? I think that would eliminate a lot of
> > > > > confusion
> > > > > > > in the future and avoid a rehash of this email thread on the
> next
> > > > > > release.
> > > > > > >
> > > > > > > -s
> > > > > > >
> > > > > > > On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah <
> hit...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > To add, the main source tarball should have instructions to
> > > > generate
> > > > > > the
> > > > > > > > sdist and bdist versions. Additionally, as part of the
> release
> > > > > process
> > > > > > if
> > > > > > > > the plan is to publish to pypi (after the IPMC vote
> succeeds),
> > > then
> > > > > the
> > > > > > > > appropriate bits also need to be verified/voted upon. There
> are
> > > not
> > > > > > > exactly
> > > > > > > > counted as the official release bits but they do need to be
> > > > verified
> > > > > as
> > > > > > > > part of the voting process to ensure that the bits do indeed
> > map
> > > to
> > > > > the
> > > > > > > > source release, license/notice files are correct, etc.
> > > > > > > >
> > > > > > > > thanks
> > > > > > > > -- Hitesh
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
> > > > bdbr...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Thanks Hitesh. We discussed it with John Ament on the IPMC.
> > > > Python
> > > > > > has
> > > > > > > > the
> > > > > > > > > notion of 3 types of distributions, “source”, “sdist”,
> > “bdist”,
> > > > > > > contrary
> > > > > > > > to
> > > > > > > > > Java that knows only two (source, bdist). We used to vote
> on
> > > > > “sdist”,
> > > > > > > > which
> > > > > > > > > was deemed incorrect.
> > > > > > > > >
> > > > > > > > > So, Max, indeed we need to vote on a tar.gz that contains
> > build
> > > > > > > > > instructions in INSTALL to get to “sdist”. The build
> > > instructions
> > > > > > > should
> > > > > > > > > also contain instruction how to run the license checks by
> > > Apache
> > > > > Rat.
> > > > > > > > Most
> > > > > > > > > of the work probably goes in the build instructions and
> > > verifying
> > > > > > they
> > > > > > > > > work, but it should not be much.
> > > > > > > > >
> > > > > > > > > Any other clarification 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Chris Riccomini]

IMO, given the level of effort for 1.8.2, and how long it's taken, we
should not be re-voting right now unless something horrific happened to the
release.

On Tue, Jul 25, 2017 at 9:54 PM, Pavel Martynov  wrote:

> Hi, folks!
>
> AIRFLOW-935 issue marked as resolved and fix version is 1.8.2, but this
> commit contained in master branch only and not tagged by 1.8.2rc2.
> Can fix of this issue be released in 1.8.2?
>
> 2017-07-26 2:27 GMT+03:00 George Leslie-Waksman <
> geo...@cloverhealth.com.invalid>:
>
> > I've checked and we are no longer relying on the previous
> > LatestOnlyOperator behavior for any of our DAGs.
> >
> > This is not a dealbreaker (though I will need to keep it in mind).
> >
> > Thanks for asking,
> > --George
> >
> > On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
> > maximebeauche...@gmail.com> wrote:
> >
> > > [AIRFLOW-1296] is part of 1.8.2.
> > >
> > > Is this a dealbreaker for 1.8.2?
> > >
> > > Max
> > >
> > > On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
> > > geo...@cloverhealth.com.invalid> wrote:
> > >
> > > > I hope that it's not too late for me to chime in but there is a
> > breaking
> > > > change in the behavior of LatestOnlyOperator.
> > > >
> > > > The change was introduced in
> > > > PR: https://github.com/apache/incubator-airflow/pull/2365
> > > > Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> > tasks
> > > >
> > > > Prior to this change, the LatestOnlyOperator would skip direct
> > downstream
> > > > but not indirect downstream; now it skips indirect downstream.
> > > >
> > > > This breaks the use of LatestOnlyOperator with TriggerRules that do
> not
> > > > propagate skips.
> > > >
> > > > --George
> > > >
> > > > On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> > > > maximebeauche...@gmail.com> wrote:
> > > >
> > > > > I think I'm gathering a good picture of what is expected here. I'll
> > try
> > > > to
> > > > > update the Confluence page as I go.
> > > > >
> > > > > I'm hoping to get started tomorrow and package it early next week.
> > > > >
> > > > > Max
> > > > >
> > > > > On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand <
> san...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > FYI, can anyone pictorially describe the release process (and
> post
> > it
> > > > on
> > > > > > the apache airflow wiki)? I think that would eliminate a lot of
> > > > confusion
> > > > > > in the future and avoid a rehash of this email thread on the next
> > > > > release.
> > > > > >
> > > > > > -s
> > > > > >
> > > > > > On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah  >
> > > > wrote:
> > > > > >
> > > > > > > To add, the main source tarball should have instructions to
> > > generate
> > > > > the
> > > > > > > sdist and bdist versions. Additionally, as part of the release
> > > > process
> > > > > if
> > > > > > > the plan is to publish to pypi (after the IPMC vote succeeds),
> > then
> > > > the
> > > > > > > appropriate bits also need to be verified/voted upon. There are
> > not
> > > > > > exactly
> > > > > > > counted as the official release bits but they do need to be
> > > verified
> > > > as
> > > > > > > part of the voting process to ensure that the bits do indeed
> map
> > to
> > > > the
> > > > > > > source release, license/notice files are correct, etc.
> > > > > > >
> > > > > > > thanks
> > > > > > > -- Hitesh
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
> > > bdbr...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks Hitesh. We discussed it with John Ament on the IPMC.
> > > Python
> > > > > has
> > > > > > > the
> > > > > > > > notion of 3 types of distributions, “source”, “sdist”,
> “bdist”,
> > > > > > contrary
> > > > > > > to
> > > > > > > > Java that knows only two (source, bdist). We used to vote on
> > > > “sdist”,
> > > > > > > which
> > > > > > > > was deemed incorrect.
> > > > > > > >
> > > > > > > > So, Max, indeed we need to vote on a tar.gz that contains
> build
> > > > > > > > instructions in INSTALL to get to “sdist”. The build
> > instructions
> > > > > > should
> > > > > > > > also contain instruction how to run the license checks by
> > Apache
> > > > Rat.
> > > > > > > Most
> > > > > > > > of the work probably goes in the build instructions and
> > verifying
> > > > > they
> > > > > > > > work, but it should not be much.
> > > > > > > >
> > > > > > > > Any other clarification required?
> > > > > > > >
> > > > > > > > Bolke
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> with best regards, Pavel Martynov
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Pavel Martynov]

Hi, folks!

AIRFLOW-935 issue marked as resolved and fix version is 1.8.2, but this
commit contained in master branch only and not tagged by 1.8.2rc2.
Can fix of this issue be released in 1.8.2?

2017-07-26 2:27 GMT+03:00 George Leslie-Waksman <
geo...@cloverhealth.com.invalid>:

> I've checked and we are no longer relying on the previous
> LatestOnlyOperator behavior for any of our DAGs.
>
> This is not a dealbreaker (though I will need to keep it in mind).
>
> Thanks for asking,
> --George
>
> On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
>
> > [AIRFLOW-1296] is part of 1.8.2.
> >
> > Is this a dealbreaker for 1.8.2?
> >
> > Max
> >
> > On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
> > geo...@cloverhealth.com.invalid> wrote:
> >
> > > I hope that it's not too late for me to chime in but there is a
> breaking
> > > change in the behavior of LatestOnlyOperator.
> > >
> > > The change was introduced in
> > > PR: https://github.com/apache/incubator-airflow/pull/2365
> > > Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> tasks
> > >
> > > Prior to this change, the LatestOnlyOperator would skip direct
> downstream
> > > but not indirect downstream; now it skips indirect downstream.
> > >
> > > This breaks the use of LatestOnlyOperator with TriggerRules that do not
> > > propagate skips.
> > >
> > > --George
> > >
> > > On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> > > maximebeauche...@gmail.com> wrote:
> > >
> > > > I think I'm gathering a good picture of what is expected here. I'll
> try
> > > to
> > > > update the Confluence page as I go.
> > > >
> > > > I'm hoping to get started tomorrow and package it early next week.
> > > >
> > > > Max
> > > >
> > > > On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand 
> > > > wrote:
> > > >
> > > > > FYI, can anyone pictorially describe the release process (and post
> it
> > > on
> > > > > the apache airflow wiki)? I think that would eliminate a lot of
> > > confusion
> > > > > in the future and avoid a rehash of this email thread on the next
> > > > release.
> > > > >
> > > > > -s
> > > > >
> > > > > On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah 
> > > wrote:
> > > > >
> > > > > > To add, the main source tarball should have instructions to
> > generate
> > > > the
> > > > > > sdist and bdist versions. Additionally, as part of the release
> > > process
> > > > if
> > > > > > the plan is to publish to pypi (after the IPMC vote succeeds),
> then
> > > the
> > > > > > appropriate bits also need to be verified/voted upon. There are
> not
> > > > > exactly
> > > > > > counted as the official release bits but they do need to be
> > verified
> > > as
> > > > > > part of the voting process to ensure that the bits do indeed map
> to
> > > the
> > > > > > source release, license/notice files are correct, etc.
> > > > > >
> > > > > > thanks
> > > > > > -- Hitesh
> > > > > >
> > > > > >
> > > > > > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
> > bdbr...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Hitesh. We discussed it with John Ament on the IPMC.
> > Python
> > > > has
> > > > > > the
> > > > > > > notion of 3 types of distributions, “source”, “sdist”, “bdist”,
> > > > > contrary
> > > > > > to
> > > > > > > Java that knows only two (source, bdist). We used to vote on
> > > “sdist”,
> > > > > > which
> > > > > > > was deemed incorrect.
> > > > > > >
> > > > > > > So, Max, indeed we need to vote on a tar.gz that contains build
> > > > > > > instructions in INSTALL to get to “sdist”. The build
> instructions
> > > > > should
> > > > > > > also contain instruction how to run the license checks by
> Apache
> > > Rat.
> > > > > > Most
> > > > > > > of the work probably goes in the build instructions and
> verifying
> > > > they
> > > > > > > work, but it should not be much.
> > > > > > >
> > > > > > > Any other clarification required?
> > > > > > >
> > > > > > > Bolke
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>



-- 
with best regards, Pavel Martynov

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[George Leslie-Waksman]

I've checked and we are no longer relying on the previous
LatestOnlyOperator behavior for any of our DAGs.

This is not a dealbreaker (though I will need to keep it in mind).

Thanks for asking,
--George

On Tue, Jul 25, 2017 at 2:50 PM Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:

> [AIRFLOW-1296] is part of 1.8.2.
>
> Is this a dealbreaker for 1.8.2?
>
> Max
>
> On Tue, Jul 25, 2017 at 2:40 PM, George Leslie-Waksman <
> geo...@cloverhealth.com.invalid> wrote:
>
> > I hope that it's not too late for me to chime in but there is a breaking
> > change in the behavior of LatestOnlyOperator.
> >
> > The change was introduced in
> > PR: https://github.com/apache/incubator-airflow/pull/2365
> > Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> >
> > Prior to this change, the LatestOnlyOperator would skip direct downstream
> > but not indirect downstream; now it skips indirect downstream.
> >
> > This breaks the use of LatestOnlyOperator with TriggerRules that do not
> > propagate skips.
> >
> > --George
> >
> > On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
> > maximebeauche...@gmail.com> wrote:
> >
> > > I think I'm gathering a good picture of what is expected here. I'll try
> > to
> > > update the Confluence page as I go.
> > >
> > > I'm hoping to get started tomorrow and package it early next week.
> > >
> > > Max
> > >
> > > On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand 
> > > wrote:
> > >
> > > > FYI, can anyone pictorially describe the release process (and post it
> > on
> > > > the apache airflow wiki)? I think that would eliminate a lot of
> > confusion
> > > > in the future and avoid a rehash of this email thread on the next
> > > release.
> > > >
> > > > -s
> > > >
> > > > On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah 
> > wrote:
> > > >
> > > > > To add, the main source tarball should have instructions to
> generate
> > > the
> > > > > sdist and bdist versions. Additionally, as part of the release
> > process
> > > if
> > > > > the plan is to publish to pypi (after the IPMC vote succeeds), then
> > the
> > > > > appropriate bits also need to be verified/voted upon. There are not
> > > > exactly
> > > > > counted as the official release bits but they do need to be
> verified
> > as
> > > > > part of the voting process to ensure that the bits do indeed map to
> > the
> > > > > source release, license/notice files are correct, etc.
> > > > >
> > > > > thanks
> > > > > -- Hitesh
> > > > >
> > > > >
> > > > > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin <
> bdbr...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Thanks Hitesh. We discussed it with John Ament on the IPMC.
> Python
> > > has
> > > > > the
> > > > > > notion of 3 types of distributions, “source”, “sdist”, “bdist”,
> > > > contrary
> > > > > to
> > > > > > Java that knows only two (source, bdist). We used to vote on
> > “sdist”,
> > > > > which
> > > > > > was deemed incorrect.
> > > > > >
> > > > > > So, Max, indeed we need to vote on a tar.gz that contains build
> > > > > > instructions in INSTALL to get to “sdist”. The build instructions
> > > > should
> > > > > > also contain instruction how to run the license checks by Apache
> > Rat.
> > > > > Most
> > > > > > of the work probably goes in the build instructions and verifying
> > > they
> > > > > > work, but it should not be much.
> > > > > >
> > > > > > Any other clarification required?
> > > > > >
> > > > > > Bolke
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[George Leslie-Waksman]

I hope that it's not too late for me to chime in but there is a breaking
change in the behavior of LatestOnlyOperator.

The change was introduced in
PR: https://github.com/apache/incubator-airflow/pull/2365
Change: 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks

Prior to this change, the LatestOnlyOperator would skip direct downstream
but not indirect downstream; now it skips indirect downstream.

This breaks the use of LatestOnlyOperator with TriggerRules that do not
propagate skips.

--George

On Thu, Jul 20, 2017 at 10:08 PM Maxime Beauchemin <
maximebeauche...@gmail.com> wrote:

> I think I'm gathering a good picture of what is expected here. I'll try to
> update the Confluence page as I go.
>
> I'm hoping to get started tomorrow and package it early next week.
>
> Max
>
> On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand 
> wrote:
>
> > FYI, can anyone pictorially describe the release process (and post it on
> > the apache airflow wiki)? I think that would eliminate a lot of confusion
> > in the future and avoid a rehash of this email thread on the next
> release.
> >
> > -s
> >
> > On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah  wrote:
> >
> > > To add, the main source tarball should have instructions to generate
> the
> > > sdist and bdist versions. Additionally, as part of the release process
> if
> > > the plan is to publish to pypi (after the IPMC vote succeeds), then the
> > > appropriate bits also need to be verified/voted upon. There are not
> > exactly
> > > counted as the official release bits but they do need to be verified as
> > > part of the voting process to ensure that the bits do indeed map to the
> > > source release, license/notice files are correct, etc.
> > >
> > > thanks
> > > -- Hitesh
> > >
> > >
> > > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin 
> > > wrote:
> > >
> > > > Thanks Hitesh. We discussed it with John Ament on the IPMC. Python
> has
> > > the
> > > > notion of 3 types of distributions, “source”, “sdist”, “bdist”,
> > contrary
> > > to
> > > > Java that knows only two (source, bdist). We used to vote on “sdist”,
> > > which
> > > > was deemed incorrect.
> > > >
> > > > So, Max, indeed we need to vote on a tar.gz that contains build
> > > > instructions in INSTALL to get to “sdist”. The build instructions
> > should
> > > > also contain instruction how to run the license checks by Apache Rat.
> > > Most
> > > > of the work probably goes in the build instructions and verifying
> they
> > > > work, but it should not be much.
> > > >
> > > > Any other clarification required?
> > > >
> > > > Bolke
> > > >
> > > >
> > >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Maxime Beauchemin]

I think I'm gathering a good picture of what is expected here. I'll try to
update the Confluence page as I go.

I'm hoping to get started tomorrow and package it early next week.

Max

On Wed, Jul 19, 2017 at 6:16 PM, siddharth anand  wrote:

> FYI, can anyone pictorially describe the release process (and post it on
> the apache airflow wiki)? I think that would eliminate a lot of confusion
> in the future and avoid a rehash of this email thread on the next release.
>
> -s
>
> On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah  wrote:
>
> > To add, the main source tarball should have instructions to generate the
> > sdist and bdist versions. Additionally, as part of the release process if
> > the plan is to publish to pypi (after the IPMC vote succeeds), then the
> > appropriate bits also need to be verified/voted upon. There are not
> exactly
> > counted as the official release bits but they do need to be verified as
> > part of the voting process to ensure that the bits do indeed map to the
> > source release, license/notice files are correct, etc.
> >
> > thanks
> > -- Hitesh
> >
> >
> > On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin 
> > wrote:
> >
> > > Thanks Hitesh. We discussed it with John Ament on the IPMC. Python has
> > the
> > > notion of 3 types of distributions, “source”, “sdist”, “bdist”,
> contrary
> > to
> > > Java that knows only two (source, bdist). We used to vote on “sdist”,
> > which
> > > was deemed incorrect.
> > >
> > > So, Max, indeed we need to vote on a tar.gz that contains build
> > > instructions in INSTALL to get to “sdist”. The build instructions
> should
> > > also contain instruction how to run the license checks by Apache Rat.
> > Most
> > > of the work probably goes in the build instructions and verifying they
> > > work, but it should not be much.
> > >
> > > Any other clarification required?
> > >
> > > Bolke
> > >
> > >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[siddharth anand]

FYI, can anyone pictorially describe the release process (and post it on
the apache airflow wiki)? I think that would eliminate a lot of confusion
in the future and avoid a rehash of this email thread on the next release.

-s

On Wed, Jul 19, 2017 at 10:48 AM, Hitesh Shah  wrote:

> To add, the main source tarball should have instructions to generate the
> sdist and bdist versions. Additionally, as part of the release process if
> the plan is to publish to pypi (after the IPMC vote succeeds), then the
> appropriate bits also need to be verified/voted upon. There are not exactly
> counted as the official release bits but they do need to be verified as
> part of the voting process to ensure that the bits do indeed map to the
> source release, license/notice files are correct, etc.
>
> thanks
> -- Hitesh
>
>
> On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin 
> wrote:
>
> > Thanks Hitesh. We discussed it with John Ament on the IPMC. Python has
> the
> > notion of 3 types of distributions, “source”, “sdist”, “bdist”, contrary
> to
> > Java that knows only two (source, bdist). We used to vote on “sdist”,
> which
> > was deemed incorrect.
> >
> > So, Max, indeed we need to vote on a tar.gz that contains build
> > instructions in INSTALL to get to “sdist”. The build instructions should
> > also contain instruction how to run the license checks by Apache Rat.
> Most
> > of the work probably goes in the build instructions and verifying they
> > work, but it should not be much.
> >
> > Any other clarification required?
> >
> > Bolke
> >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Hitesh Shah]

To add, the main source tarball should have instructions to generate the
sdist and bdist versions. Additionally, as part of the release process if
the plan is to publish to pypi (after the IPMC vote succeeds), then the
appropriate bits also need to be verified/voted upon. There are not exactly
counted as the official release bits but they do need to be verified as
part of the voting process to ensure that the bits do indeed map to the
source release, license/notice files are correct, etc.

thanks
-- Hitesh


On Tue, Jul 18, 2017 at 12:01 AM, Bolke de Bruin  wrote:

> Thanks Hitesh. We discussed it with John Ament on the IPMC. Python has the
> notion of 3 types of distributions, “source”, “sdist”, “bdist”, contrary to
> Java that knows only two (source, bdist). We used to vote on “sdist”, which
> was deemed incorrect.
>
> So, Max, indeed we need to vote on a tar.gz that contains build
> instructions in INSTALL to get to “sdist”. The build instructions should
> also contain instruction how to run the license checks by Apache Rat. Most
> of the work probably goes in the build instructions and verifying they
> work, but it should not be much.
>
> Any other clarification required?
>
> Bolke
>
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Thanks Hitesh. We discussed it with John Ament on the IPMC. Python has the 
notion of 3 types of distributions, “source”, “sdist”, “bdist”, contrary to 
Java that knows only two (source, bdist). We used to vote on “sdist”, which was 
deemed incorrect.

So, Max, indeed we need to vote on a tar.gz that contains build instructions in 
INSTALL to get to “sdist”. The build instructions should also contain 
instruction how to run the license checks by Apache Rat. Most of the work 
probably goes in the build instructions and verifying they work, but it should 
not be much.

Any other clarification required?

Bolke


> On 18 Jul 2017, at 00:36, Hitesh Shah  wrote:
> 
> The main artifact to vote upon on is a source release. This implies just
> creating a clean tarball of the source codebase with an appropriate README,
> INSTALL, LICENSE, DISCLAIMER, NOTICE (all of these should be checked in to
> the codebase as the release should map to a specific commit hash which can
> be re-used to create the source tarball again if needed). License and
> Notice files should cover all files in the codebase.
> 
> You can also publish binary build(s) as a convenience ( assuming you want
> to publish to pypi) - this would be a separate tarball and likely different
> notice/license files (would suggest having these in the src repo too) as
> the binary tarball will likely have different files.
> 
> thanks
> -- Hitesh
> 
> 
> 
> On Mon, Jul 17, 2017 at 2:18 AM, Bolke de Bruin  > wrote:
> 
>> Great!
>> 
>> It is also a bit new to me so maybe @Hitesh @Jakob can help with some
>> guidance here?
>> 
>> But my assumption indeed is:
>> 
>> 1. Make a tarball from the repo with build instructions (including a
>> working License check!) -> Vote here and IPMC. This is the “official”
>> release.
>> 2. Make sdist for redistribution on PyPi
>> 
>> Bdist isn’t required.
>> 
>> Cheers
>> Bolke.
>> 
>>> On 17 Jul 2017, at 06:27, Maxime Beauchemin 
>> wrote:
>>> 
>>> I've been slammed but skies are clearing up now I'm hoping.
>>> 
>>> Reading the general@ thread I'm unclear about the next steps, targz the
>>> whole repo and add build instructions? What should the file with the
>> build
>>> instructions be called? How to label that new tarbar? Can we skip the
>> bdist?
>>> 
>>> Max
>>> 
>>> On Sun, Jul 16, 2017 at 12:35 PM, Bolke de Bruin > >> wrote:
>>> 
 Max, Ping? Do you need help?
 
> On 9 Jul 2017, at 14:30, Bolke de Bruin  > wrote:
> 
> Hi Max,
> 
> The canonical distribution would be what we have in git right now (ie.
 before running python sdist). The rest is just convenience packages. So
 npm, would solve the issue as long as we don’t rely on any non APL
 compatible packages in core. I don’t think npm/yarn/webpack needs to be
 done for 1.8.3, but considering the messy javascript that we currently
>> have
 it would be nice to put it on the todo.
> 
> Cheers
> Bolke
> 
>> On 9 Jul 2017, at 06:46, Maxime Beauchemin <
>> maximebeauche...@gmail.com 
  
 > wrote:
>> 
>> As far as I understand npm would not solve the problem as typically
>> we'd
>> build our "entry" files and distribute that with Airflow as static
 files.
>> Those entry files would contain these other npm packages, minified.
 (from
>> my understanding that is the same issue as packaging the libs
 themselves)
>> 
>> To make them runtime deps would be atypical and more complicated.
 `airflow
>> webserver` would need to "build" (npm install/webpack) and the
>> webserver
>> would have to serve these static files out of some that temp location
>> (perhaps ~/.airflow/airflow.entry.js) as opposed to out of
 `site-packages`.
>> 
>> Also note that Airflow's javascript is in pretty bad shape (scattered
>> in
>> jinja templates files) and it would take quite a significant amount of
 work
>> to move to using npm/webpack.
>> 
>> I'm back from vacation and will have things to catch up on next week
>> but
>> I'll try to find time to look into some of this.
>> 
>> On Thu, Jul 6, 2017 at 1:10 PM, Bolke de Bruin > 
>> >
  
 > 
>>> Hi Folks,
>>> 
>>> We probably need to adjust our release process as can be observed in
 the
>>> IPMC thread. As we are packaging a “sdist” it does not pass license

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Great!

It is also a bit new to me so maybe @Hitesh @Jakob can help with some guidance 
here?

But my assumption indeed is:

1. Make a tarball from the repo with build instructions (including a working 
License check!) -> Vote here and IPMC. This is the “official” release.
2. Make sdist for redistribution on PyPi

Bdist isn’t required.

Cheers
Bolke.

> On 17 Jul 2017, at 06:27, Maxime Beauchemin  
> wrote:
> 
> I've been slammed but skies are clearing up now I'm hoping.
> 
> Reading the general@ thread I'm unclear about the next steps, targz the
> whole repo and add build instructions? What should the file with the build
> instructions be called? How to label that new tarbar? Can we skip the bdist?
> 
> Max
> 
> On Sun, Jul 16, 2017 at 12:35 PM, Bolke de Bruin  > wrote:
> 
>> Max, Ping? Do you need help?
>> 
>>> On 9 Jul 2017, at 14:30, Bolke de Bruin  wrote:
>>> 
>>> Hi Max,
>>> 
>>> The canonical distribution would be what we have in git right now (ie.
>> before running python sdist). The rest is just convenience packages. So
>> npm, would solve the issue as long as we don’t rely on any non APL
>> compatible packages in core. I don’t think npm/yarn/webpack needs to be
>> done for 1.8.3, but considering the messy javascript that we currently have
>> it would be nice to put it on the todo.
>>> 
>>> Cheers
>>> Bolke
>>> 
 On 9 Jul 2017, at 06:46, Maxime Beauchemin > >> 
>> wrote:
 
 As far as I understand npm would not solve the problem as typically we'd
 build our "entry" files and distribute that with Airflow as static
>> files.
 Those entry files would contain these other npm packages, minified.
>> (from
 my understanding that is the same issue as packaging the libs
>> themselves)
 
 To make them runtime deps would be atypical and more complicated.
>> `airflow
 webserver` would need to "build" (npm install/webpack) and the webserver
 would have to serve these static files out of some that temp location
 (perhaps ~/.airflow/airflow.entry.js) as opposed to out of
>> `site-packages`.
 
 Also note that Airflow's javascript is in pretty bad shape (scattered in
 jinja templates files) and it would take quite a significant amount of
>> work
 to move to using npm/webpack.
 
 I'm back from vacation and will have things to catch up on next week but
 I'll try to find time to look into some of this.
 
 On Thu, Jul 6, 2017 at 1:10 PM, Bolke de Bruin 
>> >> wrote:
 
> Hi Folks,
> 
> We probably need to adjust our release process as can be observed in
>> the
> IPMC thread. As we are packaging a “sdist” it does not pass license
>> checks
> and one cannot verify the validity of what we are doing. It was
>> suggested
> by one of the maintainers of another python project to create 3
>> different
> packages:
> 
> 1. A source tarball which is essentially a snapshot of the repository
> 2. A sdist
> 3. A bdist
> 
> 1 should then be the canonical Apache release. It should be
>> accompanied by
> build instructions and it should pass RAT checks. This is the package
>> we
> will vote on.
> 2 is what we have voted upon until now. It should contain (it does)
> LICENSE, NOTICE, and DISCLAIMER
> 3 bdist, wheel package. Same as 2. Not really required, but more a
> convenience package as is 2
> 
> 2 and 3 can be published to PyPi.
> 
> Max: can you take care of this? We need to vote on 1.  Build
>> instruction
> could be added to an INSTALL file or just to the README.md file? See
>> for
> inspiration the GitHub page of ariatosca: https://github.com/apache/ 
>  <
>> https://github.com/apache/ >
> incubator-ariatosca   <
>> https://github.com/apache/incubator-ariatosca 
>> >>
> 
> In the meantime I am figuring out an issue with one of the
>> dependencies of
> nvd3 which might be or have been GPL3 which is incompatible with the
>> APL,
> which we are distributing together with our source. Ideally, we should
>> move
> to a “yarn/webpack” build which will resolve those issues
>> automatically as
> these become runtime dependencies then in case of 1,2,3.
> 
> Cheers
> Bolke
> 
> 
> 
>> On 6 Jul 2017, at 05:20, Sumit Maheshwari > 
>> >>
> wrote:
>> 
>> Awesome.. thanks a lot Max 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Max, Ping? Do you need help? 

> On 9 Jul 2017, at 14:30, Bolke de Bruin  wrote:
> 
> Hi Max,
> 
> The canonical distribution would be what we have in git right now (ie. before 
> running python sdist). The rest is just convenience packages. So npm, would 
> solve the issue as long as we don’t rely on any non APL compatible packages 
> in core. I don’t think npm/yarn/webpack needs to be done for 1.8.3, but 
> considering the messy javascript that we currently have it would be nice to 
> put it on the todo.
> 
> Cheers
> Bolke
>  
>> On 9 Jul 2017, at 06:46, Maxime Beauchemin > > wrote:
>> 
>> As far as I understand npm would not solve the problem as typically we'd
>> build our "entry" files and distribute that with Airflow as static files.
>> Those entry files would contain these other npm packages, minified. (from
>> my understanding that is the same issue as packaging the libs themselves)
>> 
>> To make them runtime deps would be atypical and more complicated. `airflow
>> webserver` would need to "build" (npm install/webpack) and the webserver
>> would have to serve these static files out of some that temp location
>> (perhaps ~/.airflow/airflow.entry.js) as opposed to out of `site-packages`.
>> 
>> Also note that Airflow's javascript is in pretty bad shape (scattered in
>> jinja templates files) and it would take quite a significant amount of work
>> to move to using npm/webpack.
>> 
>> I'm back from vacation and will have things to catch up on next week but
>> I'll try to find time to look into some of this.
>> 
>> On Thu, Jul 6, 2017 at 1:10 PM, Bolke de Bruin > > wrote:
>> 
>>> Hi Folks,
>>> 
>>> We probably need to adjust our release process as can be observed in the
>>> IPMC thread. As we are packaging a “sdist” it does not pass license checks
>>> and one cannot verify the validity of what we are doing. It was suggested
>>> by one of the maintainers of another python project to create 3 different
>>> packages:
>>> 
>>> 1. A source tarball which is essentially a snapshot of the repository
>>> 2. A sdist
>>> 3. A bdist
>>> 
>>> 1 should then be the canonical Apache release. It should be accompanied by
>>> build instructions and it should pass RAT checks. This is the package we
>>> will vote on.
>>> 2 is what we have voted upon until now. It should contain (it does)
>>> LICENSE, NOTICE, and DISCLAIMER
>>> 3 bdist, wheel package. Same as 2. Not really required, but more a
>>> convenience package as is 2
>>> 
>>> 2 and 3 can be published to PyPi.
>>> 
>>> Max: can you take care of this? We need to vote on 1.  Build instruction
>>> could be added to an INSTALL file or just to the README.md file? See for
>>> inspiration the GitHub page of ariatosca: https://github.com/apache/ 
>>> 
>>> incubator-ariatosca >> >
>>> 
>>> In the meantime I am figuring out an issue with one of the dependencies of
>>> nvd3 which might be or have been GPL3 which is incompatible with the APL,
>>> which we are distributing together with our source. Ideally, we should move
>>> to a “yarn/webpack” build which will resolve those issues automatically as
>>> these become runtime dependencies then in case of 1,2,3.
>>> 
>>> Cheers
>>> Bolke
>>> 
>>> 
>>> 
 On 6 Jul 2017, at 05:20, Sumit Maheshwari >
>>> wrote:
 
 Awesome.. thanks a lot Max for taking the RM responsibility..
 
 On Jul 5, 2017 11:10 PM, "Chris Riccomini" >
>>> wrote:
 
> Nice, thanks Max!
> 
> On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin <
> maximebeauche...@gmail.com > wrote:
> 
>> Hello,
>> 
>> Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.
>> 
>> 5 “+1” votes received, 4 out of which are "binding":
>> 
>> - Maxime Beauchemin (binding)
>> - Bolke de Bruin (binding)
>> - Chris Riccomini (binding)
>> - Sumit Maheshwari  (binding)
>> 
>> - Kengo Seki  (non-binding)
>> 
>> Vote thread (start):
>> *http://mail-archives.apache.org/mod_mbox/incubator- 
>> 
>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com %3E
>> > 
>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com %3E>*
>> 
>> Next steps:
>> 1) will start the voting process at the IPMC mailinglist. I don’t
>>> expect
>> changes.
>> 2) Only 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Hi Max,

The canonical distribution would be what we have in git right now (ie. before 
running python sdist). The rest is just convenience packages. So npm, would 
solve the issue as long as we don’t rely on any non APL compatible packages in 
core. I don’t think npm/yarn/webpack needs to be done for 1.8.3, but 
considering the messy javascript that we currently have it would be nice to put 
it on the todo.

Cheers
Bolke
 
> On 9 Jul 2017, at 06:46, Maxime Beauchemin  wrote:
> 
> As far as I understand npm would not solve the problem as typically we'd
> build our "entry" files and distribute that with Airflow as static files.
> Those entry files would contain these other npm packages, minified. (from
> my understanding that is the same issue as packaging the libs themselves)
> 
> To make them runtime deps would be atypical and more complicated. `airflow
> webserver` would need to "build" (npm install/webpack) and the webserver
> would have to serve these static files out of some that temp location
> (perhaps ~/.airflow/airflow.entry.js) as opposed to out of `site-packages`.
> 
> Also note that Airflow's javascript is in pretty bad shape (scattered in
> jinja templates files) and it would take quite a significant amount of work
> to move to using npm/webpack.
> 
> I'm back from vacation and will have things to catch up on next week but
> I'll try to find time to look into some of this.
> 
> On Thu, Jul 6, 2017 at 1:10 PM, Bolke de Bruin  > wrote:
> 
>> Hi Folks,
>> 
>> We probably need to adjust our release process as can be observed in the
>> IPMC thread. As we are packaging a “sdist” it does not pass license checks
>> and one cannot verify the validity of what we are doing. It was suggested
>> by one of the maintainers of another python project to create 3 different
>> packages:
>> 
>> 1. A source tarball which is essentially a snapshot of the repository
>> 2. A sdist
>> 3. A bdist
>> 
>> 1 should then be the canonical Apache release. It should be accompanied by
>> build instructions and it should pass RAT checks. This is the package we
>> will vote on.
>> 2 is what we have voted upon until now. It should contain (it does)
>> LICENSE, NOTICE, and DISCLAIMER
>> 3 bdist, wheel package. Same as 2. Not really required, but more a
>> convenience package as is 2
>> 
>> 2 and 3 can be published to PyPi.
>> 
>> Max: can you take care of this? We need to vote on 1.  Build instruction
>> could be added to an INSTALL file or just to the README.md file? See for
>> inspiration the GitHub page of ariatosca: https://github.com/apache/
>> incubator-ariatosca > >
>> 
>> In the meantime I am figuring out an issue with one of the dependencies of
>> nvd3 which might be or have been GPL3 which is incompatible with the APL,
>> which we are distributing together with our source. Ideally, we should move
>> to a “yarn/webpack” build which will resolve those issues automatically as
>> these become runtime dependencies then in case of 1,2,3.
>> 
>> Cheers
>> Bolke
>> 
>> 
>> 
>>> On 6 Jul 2017, at 05:20, Sumit Maheshwari >> >
>> wrote:
>>> 
>>> Awesome.. thanks a lot Max for taking the RM responsibility..
>>> 
>>> On Jul 5, 2017 11:10 PM, "Chris Riccomini" >> >
>> wrote:
>>> 
 Nice, thanks Max!
 
 On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin <
 maximebeauche...@gmail.com > wrote:
 
> Hello,
> 
> Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.
> 
> 5 “+1” votes received, 4 out of which are "binding":
> 
> - Maxime Beauchemin (binding)
> - Bolke de Bruin (binding)
> - Chris Riccomini (binding)
> - Sumit Maheshwari  (binding)
> 
> - Kengo Seki  (non-binding)
> 
> Vote thread (start):
> *http://mail-archives.apache.org/mod_mbox/incubator- 
> 
> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com %3E
>  
> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com %3E>*
> 
> Next steps:
> 1) will start the voting process at the IPMC mailinglist. I don’t
>> expect
> changes.
> 2) Only after the positive voting on the IPMC and finalisation I will
> rebrand the RC to Release.
> 3) I will upload it to the incubator release page, then the tar ball
 needs
> to propagate to the mirrors.
> 4) Update the website
> 5) Finally, will upload to Pypi.
> 
> Cheers,

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Hi Folks,

We probably need to adjust our release process as can be observed in the IPMC 
thread. As we are packaging a “sdist” it does not pass license checks and one 
cannot verify the validity of what we are doing. It was suggested by one of the 
maintainers of another python project to create 3 different packages:

1. A source tarball which is essentially a snapshot of the repository
2. A sdist
3. A bdist

1 should then be the canonical Apache release. It should be accompanied by 
build instructions and it should pass RAT checks. This is the package we will 
vote on.
2 is what we have voted upon until now. It should contain (it does) LICENSE, 
NOTICE, and DISCLAIMER
3 bdist, wheel package. Same as 2. Not really required, but more a convenience 
package as is 2

2 and 3 can be published to PyPi.

Max: can you take care of this? We need to vote on 1.  Build instruction could 
be added to an INSTALL file or just to the README.md file? See for inspiration 
the GitHub page of ariatosca: https://github.com/apache/incubator-ariatosca 


In the meantime I am figuring out an issue with one of the dependencies of nvd3 
which might be or have been GPL3 which is incompatible with the APL, which we 
are distributing together with our source. Ideally, we should move to a 
“yarn/webpack” build which will resolve those issues automatically as these 
become runtime dependencies then in case of 1,2,3.

Cheers
Bolke



> On 6 Jul 2017, at 05:20, Sumit Maheshwari  wrote:
> 
> Awesome.. thanks a lot Max for taking the RM responsibility..
> 
> On Jul 5, 2017 11:10 PM, "Chris Riccomini"  wrote:
> 
>> Nice, thanks Max!
>> 
>> On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin <
>> maximebeauche...@gmail.com> wrote:
>> 
>>> Hello,
>>> 
>>> Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.
>>> 
>>> 5 “+1” votes received, 4 out of which are "binding":
>>> 
>>> - Maxime Beauchemin (binding)
>>> - Bolke de Bruin (binding)
>>> - Chris Riccomini (binding)
>>> - Sumit Maheshwari  (binding)
>>> 
>>> - Kengo Seki  (non-binding)
>>> 
>>> Vote thread (start):
>>> *http://mail-archives.apache.org/mod_mbox/incubator-
>>> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
>>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E
>>> >> airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
>>> 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E>*
>>> 
>>> Next steps:
>>> 1) will start the voting process at the IPMC mailinglist. I don’t expect
>>> changes.
>>> 2) Only after the positive voting on the IPMC and finalisation I will
>>> rebrand the RC to Release.
>>> 3) I will upload it to the incubator release page, then the tar ball
>> needs
>>> to propagate to the mirrors.
>>> 4) Update the website
>>> 5) Finally, will upload to Pypi.
>>> 
>>> Cheers,
>>> 
>>> Max
>>> 
>>> On Mon, Jul 3, 2017 at 12:11 PM, Bolke de Bruin 
>> wrote:
>>> 
 Ping?
 
 
> On 30 Jun 2017, at 14:21, Bolke de Bruin  wrote:
> 
> Max,
> 
> I think you can close the vote?
> 
> Bolke
> 
>> On 27 Jun 2017, at 02:45, Kengo Seki  wrote:
>> 
>> +1 (non-binding)
>> 
>> - verified signatures and checksums
>> - ran scheduler and webserver, confirmed they worked fine
>> - confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included
>> 
>> Kengo Seki 
>> 
>> 
>> 2017-06-27 8:53 GMT+09:00 Chris Riccomini :
>>> +1 (binding)
>>> 
>>> Been running in our dev env, and everything looks good.
>>> 
>>> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel <
>> alex.guz...@airbnb.com
>>> .
 invalid
 wrote:
>>> 
 Yeah that makes sense. It pages by default at 500 so it explains
>> why
 we saw
 it.
 
 On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini <
 criccom...@apache.org>
 wrote:
 
> In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2,
>> it
 has
> "Show <25> entries". So it looks like prior to 1.8.2, the
>>> pagination
 was
> broken in the sense that it defaulted to the whole list. We have
>>> 479
 DAGs
> in one env, and it shows them all. It looks like someone fixed
>> the
 entry
 to
> default to 25 now, which exposed the problem for our
>> environments.
> 
> On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel <
>>> alex.guz...@airbnb.com
 .
> invalid
>> wrote:
> 
>> We're running 1.8.0 + some extras, and none of us added
>> pagination
>> recently, and our homepage is paginated. Are you sure it's not
>> the
 number
>> of dags crossing the threshold? Maybe it's some Flask version
>>> thing?
>> 
>> On Mon, Jun 26, 2017 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Sumit Maheshwari]

Awesome.. thanks a lot Max for taking the RM responsibility..

On Jul 5, 2017 11:10 PM, "Chris Riccomini"  wrote:

> Nice, thanks Max!
>
> On Mon, Jul 3, 2017 at 2:36 PM, Maxime Beauchemin <
> maximebeauche...@gmail.com> wrote:
>
> > Hello,
> >
> > Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.
> >
> > 5 “+1” votes received, 4 out of which are "binding":
> >
> > - Maxime Beauchemin (binding)
> > - Bolke de Bruin (binding)
> > - Chris Riccomini (binding)
> > - Sumit Maheshwari  (binding)
> >
> > - Kengo Seki  (non-binding)
> >
> > Vote thread (start):
> > *http://mail-archives.apache.org/mod_mbox/incubator-
> > airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> > 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E
> >  > airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--
> > 78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E>*
> >
> > Next steps:
> > 1) will start the voting process at the IPMC mailinglist. I don’t expect
> > changes.
> > 2) Only after the positive voting on the IPMC and finalisation I will
> > rebrand the RC to Release.
> > 3) I will upload it to the incubator release page, then the tar ball
> needs
> > to propagate to the mirrors.
> > 4) Update the website
> > 5) Finally, will upload to Pypi.
> >
> > Cheers,
> >
> > Max
> >
> > On Mon, Jul 3, 2017 at 12:11 PM, Bolke de Bruin 
> wrote:
> >
> > > Ping?
> > >
> > >
> > > > On 30 Jun 2017, at 14:21, Bolke de Bruin  wrote:
> > > >
> > > > Max,
> > > >
> > > > I think you can close the vote?
> > > >
> > > > Bolke
> > > >
> > > >> On 27 Jun 2017, at 02:45, Kengo Seki  wrote:
> > > >>
> > > >> +1 (non-binding)
> > > >>
> > > >> - verified signatures and checksums
> > > >> - ran scheduler and webserver, confirmed they worked fine
> > > >> - confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included
> > > >>
> > > >> Kengo Seki 
> > > >>
> > > >>
> > > >> 2017-06-27 8:53 GMT+09:00 Chris Riccomini :
> > > >>> +1 (binding)
> > > >>>
> > > >>> Been running in our dev env, and everything looks good.
> > > >>>
> > > >>> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel <
> alex.guz...@airbnb.com
> > .
> > > invalid
> > >  wrote:
> > > >>>
> > >  Yeah that makes sense. It pages by default at 500 so it explains
> why
> > > we saw
> > >  it.
> > > 
> > >  On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini <
> > > criccom...@apache.org>
> > >  wrote:
> > > 
> > > > In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2,
> it
> > > has
> > > > "Show <25> entries". So it looks like prior to 1.8.2, the
> > pagination
> > > was
> > > > broken in the sense that it defaulted to the whole list. We have
> > 479
> > > DAGs
> > > > in one env, and it shows them all. It looks like someone fixed
> the
> > > entry
> > >  to
> > > > default to 25 now, which exposed the problem for our
> environments.
> > > >
> > > > On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel <
> > alex.guz...@airbnb.com
> > > .
> > > > invalid
> > > >> wrote:
> > > >
> > > >> We're running 1.8.0 + some extras, and none of us added
> pagination
> > > >> recently, and our homepage is paginated. Are you sure it's not
> the
> > >  number
> > > >> of dags crossing the threshold? Maybe it's some Flask version
> > thing?
> > > >>
> > > >> On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini <
> > >  criccom...@apache.org>
> > > >> wrote:
> > > >>
> > > >>> Yes, I did the 1.8.1 release.
> > > >>>
> > > >>> On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel <
> > > alex.guz...@airbnb.com
> > >  .
> > > >>> invalid
> > >  wrote:
> > > >>>
> > >  There's no pagination in 1.8.1? Are you sure?
> > > 
> > >  On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> > > >> criccom...@apache.org>
> > >  wrote:
> > > 
> > > > It's not happening on 1.8.1 (since there's no pagination in
> > that
> > >  version),
> > > > so I'd count this as a regression. I wouldn't say it's
> > blocking,
> > > > but
> > > >>> it's
> > > > pretty ugly.
> > > >
> > > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
> > > > alex.guz...@airbnb.com
> > > >> .
> > > > invalid
> > > >> wrote:
> > > >
> > > >> I'm not so sure this is a new issue. I think we've seen it
> on
> > >  our
> > > >> production for quite a while.
> > > >>
> > > >> On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> > >  criccom...@apache.org>
> > > >> wrote:
> > > >>
> > > >>> I am seeing a strange UI behavior on 1.8.2.RC2. I've
> opened a
> > > >> JIRA
> > > > here:
> > > >>>
> > > >>> 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Maxime Beauchemin]

Hello,

Apache Airflow (incubating) 1.8.2 (RC2) has been accepted.

5 “+1” votes received, 4 out of which are "binding":

- Maxime Beauchemin (binding)
- Bolke de Bruin (binding)
- Chris Riccomini (binding)
- Sumit Maheshwari  (binding)

- Kengo Seki  (non-binding)

Vote thread (start):
*http://mail-archives.apache.org/mod_mbox/incubator-airflow-dev/201706.mbox/%3CCAHEEp7XRRk__hGUMjOCw%2BTW0Gc--78qEewcsAsQBLrv1x6j71w%40mail.gmail.com%3E
*

Next steps:
1) will start the voting process at the IPMC mailinglist. I don’t expect
changes.
2) Only after the positive voting on the IPMC and finalisation I will
rebrand the RC to Release.
3) I will upload it to the incubator release page, then the tar ball needs
to propagate to the mirrors.
4) Update the website
5) Finally, will upload to Pypi.

Cheers,

Max

On Mon, Jul 3, 2017 at 12:11 PM, Bolke de Bruin  wrote:

> Ping?
>
>
> > On 30 Jun 2017, at 14:21, Bolke de Bruin  wrote:
> >
> > Max,
> >
> > I think you can close the vote?
> >
> > Bolke
> >
> >> On 27 Jun 2017, at 02:45, Kengo Seki  wrote:
> >>
> >> +1 (non-binding)
> >>
> >> - verified signatures and checksums
> >> - ran scheduler and webserver, confirmed they worked fine
> >> - confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included
> >>
> >> Kengo Seki 
> >>
> >>
> >> 2017-06-27 8:53 GMT+09:00 Chris Riccomini :
> >>> +1 (binding)
> >>>
> >>> Been running in our dev env, and everything looks good.
> >>>
> >>> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel  invalid
>  wrote:
> >>>
>  Yeah that makes sense. It pages by default at 500 so it explains why
> we saw
>  it.
> 
>  On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini <
> criccom...@apache.org>
>  wrote:
> 
> > In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it
> has
> > "Show <25> entries". So it looks like prior to 1.8.2, the pagination
> was
> > broken in the sense that it defaulted to the whole list. We have 479
> DAGs
> > in one env, and it shows them all. It looks like someone fixed the
> entry
>  to
> > default to 25 now, which exposed the problem for our environments.
> >
> > On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel  .
> > invalid
> >> wrote:
> >
> >> We're running 1.8.0 + some extras, and none of us added pagination
> >> recently, and our homepage is paginated. Are you sure it's not the
>  number
> >> of dags crossing the threshold? Maybe it's some Flask version thing?
> >>
> >> On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini <
>  criccom...@apache.org>
> >> wrote:
> >>
> >>> Yes, I did the 1.8.1 release.
> >>>
> >>> On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel <
> alex.guz...@airbnb.com
>  .
> >>> invalid
>  wrote:
> >>>
>  There's no pagination in 1.8.1? Are you sure?
> 
>  On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> >> criccom...@apache.org>
>  wrote:
> 
> > It's not happening on 1.8.1 (since there's no pagination in that
>  version),
> > so I'd count this as a regression. I wouldn't say it's blocking,
> > but
> >>> it's
> > pretty ugly.
> >
> > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
> > alex.guz...@airbnb.com
> >> .
> > invalid
> >> wrote:
> >
> >> I'm not so sure this is a new issue. I think we've seen it on
>  our
> >> production for quite a while.
> >>
> >> On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
>  criccom...@apache.org>
> >> wrote:
> >>
> >>> I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
> >> JIRA
> > here:
> >>>
> >>> https://issues.apache.org/jira/browse/AIRFLOW-1348
> >>>
> >>> Has anyone else seen this?
> >>>
> >>> On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> >> sumeet.ma...@gmail.com>
> >>> wrote:
> >>>
>  +1, binding.
> 
> 
>  On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> >>> bdbr...@gmail.com
> >
> >>> wrote:
> 
> > We have been running it for the last couple of days. No
> >> issues
>  and
> >>> seems
> > more responsive.
> >
> > +1, binding
> >
> > Bolke
> >
> >> On 25 Jun 2017, at 01:10, Maxime Beauchemin <
>  maximebeauche...@gmail.com>
> > wrote:
> >>
> >> Dear all,
> >>
> 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

Max,

I think you can close the vote?

Bolke

> On 27 Jun 2017, at 02:45, Kengo Seki  wrote:
> 
> +1 (non-binding)
> 
> - verified signatures and checksums
> - ran scheduler and webserver, confirmed they worked fine
> - confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included
> 
> Kengo Seki 
> 
> 
> 2017-06-27 8:53 GMT+09:00 Chris Riccomini :
>> +1 (binding)
>> 
>> Been running in our dev env, and everything looks good.
>> 
>> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel >> wrote:
>> 
>>> Yeah that makes sense. It pages by default at 500 so it explains why we saw
>>> it.
>>> 
>>> On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini 
>>> wrote:
>>> 
 In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it has
 "Show <25> entries". So it looks like prior to 1.8.2, the pagination was
 broken in the sense that it defaulted to the whole list. We have 479 DAGs
 in one env, and it shows them all. It looks like someone fixed the entry
>>> to
 default to 25 now, which exposed the problem for our environments.
 
 On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel  wrote:
 
> We're running 1.8.0 + some extras, and none of us added pagination
> recently, and our homepage is paginated. Are you sure it's not the
>>> number
> of dags crossing the threshold? Maybe it's some Flask version thing?
> 
> On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini <
>>> criccom...@apache.org>
> wrote:
> 
>> Yes, I did the 1.8.1 release.
>> 
>> On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel >> .
>> invalid
>>> wrote:
>> 
>>> There's no pagination in 1.8.1? Are you sure?
>>> 
>>> On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> criccom...@apache.org>
>>> wrote:
>>> 
 It's not happening on 1.8.1 (since there's no pagination in that
>>> version),
 so I'd count this as a regression. I wouldn't say it's blocking,
 but
>> it's
 pretty ugly.
 
 On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
 alex.guz...@airbnb.com
> .
 invalid
> wrote:
 
> I'm not so sure this is a new issue. I think we've seen it on
>>> our
> production for quite a while.
> 
> On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
>>> criccom...@apache.org>
> wrote:
> 
>> I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
> JIRA
 here:
>> 
>> https://issues.apache.org/jira/browse/AIRFLOW-1348
>> 
>> Has anyone else seen this?
>> 
>> On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> sumeet.ma...@gmail.com>
>> wrote:
>> 
>>> +1, binding.
>>> 
>>> 
>>> On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
>> bdbr...@gmail.com
 
>> wrote:
>>> 
 We have been running it for the last couple of days. No
> issues
>>> and
>> seems
 more responsive.
 
 +1, binding
 
 Bolke
 
> On 25 Jun 2017, at 01:10, Maxime Beauchemin <
>>> maximebeauche...@gmail.com>
 wrote:
> 
> Dear all,
> 
> 1.8.2 RC2 is baked and available at:
> https://dist.apache.org/repos/
>>> dist/dev/incubator/airflow
 ,
>>> public
>> keys
> are available
> at https://dist.apache.org/repos/
>>> dist/release/incubator/airflow.
> 
> Note that RC1 was the first RC (skipped RC0) and was
 never
> announced
 since
> it had issues coming out of the oven, so RC2 is the
>>> first
>>> public
> RC.
> 
> 1.8.2 RC2 is build on to of 1.8.1 with these listed
>> "cherries"
>>> on
>> top.
>>> I
> added the JIRAs that were identified blockers and
 targeted
 1.8.2. I
> attempted to bring in all of the JIRAs that targeted
 1.8.2
>> but
> bailed
>>> on
> the ones that were generating merge conflicts. I also
 added
>> all
 of
>> the
> JIRAs that we've been running in production at Airbnb.
> 
> Issues fixed:
> 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__
 ColumnOperator
>> When
>> Testing
> Booleans
> 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all
 downstream
>>> tasks
> 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop
> components
> 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific
>> 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Kengo Seki]

+1 (non-binding)

- verified signatures and checksums
- ran scheduler and webserver, confirmed they worked fine
- confirmed the latest fix on v1.8 branch (AIRFLOW-809) is included

Kengo Seki 


2017-06-27 8:53 GMT+09:00 Chris Riccomini :
> +1 (binding)
>
> Been running in our dev env, and everything looks good.
>
> On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel > wrote:
>
>> Yeah that makes sense. It pages by default at 500 so it explains why we saw
>> it.
>>
>> On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini 
>> wrote:
>>
>> > In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it has
>> > "Show <25> entries". So it looks like prior to 1.8.2, the pagination was
>> > broken in the sense that it defaulted to the whole list. We have 479 DAGs
>> > in one env, and it shows them all. It looks like someone fixed the entry
>> to
>> > default to 25 now, which exposed the problem for our environments.
>> >
>> > On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel > > invalid
>> > > wrote:
>> >
>> > > We're running 1.8.0 + some extras, and none of us added pagination
>> > > recently, and our homepage is paginated. Are you sure it's not the
>> number
>> > > of dags crossing the threshold? Maybe it's some Flask version thing?
>> > >
>> > > On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini <
>> criccom...@apache.org>
>> > > wrote:
>> > >
>> > > > Yes, I did the 1.8.1 release.
>> > > >
>> > > > On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel > .
>> > > > invalid
>> > > > > wrote:
>> > > >
>> > > > > There's no pagination in 1.8.1? Are you sure?
>> > > > >
>> > > > > On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
>> > > criccom...@apache.org>
>> > > > > wrote:
>> > > > >
>> > > > > > It's not happening on 1.8.1 (since there's no pagination in that
>> > > > > version),
>> > > > > > so I'd count this as a regression. I wouldn't say it's blocking,
>> > but
>> > > > it's
>> > > > > > pretty ugly.
>> > > > > >
>> > > > > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
>> > alex.guz...@airbnb.com
>> > > .
>> > > > > > invalid
>> > > > > > > wrote:
>> > > > > >
>> > > > > > > I'm not so sure this is a new issue. I think we've seen it on
>> our
>> > > > > > > production for quite a while.
>> > > > > > >
>> > > > > > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
>> > > > > criccom...@apache.org>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
>> > > JIRA
>> > > > > > here:
>> > > > > > > >
>> > > > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
>> > > > > > > >
>> > > > > > > > Has anyone else seen this?
>> > > > > > > >
>> > > > > > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
>> > > > > > > sumeet.ma...@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > +1, binding.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
>> > > > bdbr...@gmail.com
>> > > > > >
>> > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > > We have been running it for the last couple of days. No
>> > > issues
>> > > > > and
>> > > > > > > > seems
>> > > > > > > > > > more responsive.
>> > > > > > > > > >
>> > > > > > > > > > +1, binding
>> > > > > > > > > >
>> > > > > > > > > > Bolke
>> > > > > > > > > >
>> > > > > > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
>> > > > > > > > > maximebeauche...@gmail.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > Dear all,
>> > > > > > > > > > >
>> > > > > > > > > > > 1.8.2 RC2 is baked and available at:
>> > > > > > > > > > > https://dist.apache.org/repos/
>> dist/dev/incubator/airflow
>> > ,
>> > > > > public
>> > > > > > > > keys
>> > > > > > > > > > > are available
>> > > > > > > > > > > at https://dist.apache.org/repos/
>> > > > > dist/release/incubator/airflow.
>> > > > > > > > > > >
>> > > > > > > > > > > Note that RC1 was the first RC (skipped RC0) and was
>> > never
>> > > > > > > announced
>> > > > > > > > > > since
>> > > > > > > > > > > it had issues coming out of the oven, so RC2 is the
>> first
>> > > > > public
>> > > > > > > RC.
>> > > > > > > > > > >
>> > > > > > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed
>> > > > "cherries"
>> > > > > on
>> > > > > > > > top.
>> > > > > > > > > I
>> > > > > > > > > > > added the JIRAs that were identified blockers and
>> > targeted
>> > > > > > 1.8.2. I
>> > > > > > > > > > > attempted to bring in all of the JIRAs that targeted
>> > 1.8.2
>> > > > but
>> > > > > > > bailed
>> > > > > > > > > on
>> > > > > > > > > > > the ones that were generating merge conflicts. I also
>> > added
>> > > > all
>> > > > > > of
>> > > > > > > > the
>> > > > > > > > > > > JIRAs that we've been running in production at Airbnb.
>> > > > > > > > > > >
>> > > > > > > > > > > Issues fixed:
>> > > > > > > > > > > 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Chris Riccomini]

+1 (binding)

Been running in our dev env, and everything looks good.

On Mon, Jun 26, 2017 at 3:00 PM, Alex Guziel  wrote:

> Yeah that makes sense. It pages by default at 500 so it explains why we saw
> it.
>
> On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini 
> wrote:
>
> > In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it has
> > "Show <25> entries". So it looks like prior to 1.8.2, the pagination was
> > broken in the sense that it defaulted to the whole list. We have 479 DAGs
> > in one env, and it shows them all. It looks like someone fixed the entry
> to
> > default to 25 now, which exposed the problem for our environments.
> >
> > On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel  > invalid
> > > wrote:
> >
> > > We're running 1.8.0 + some extras, and none of us added pagination
> > > recently, and our homepage is paginated. Are you sure it's not the
> number
> > > of dags crossing the threshold? Maybe it's some Flask version thing?
> > >
> > > On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini <
> criccom...@apache.org>
> > > wrote:
> > >
> > > > Yes, I did the 1.8.1 release.
> > > >
> > > > On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  .
> > > > invalid
> > > > > wrote:
> > > >
> > > > > There's no pagination in 1.8.1? Are you sure?
> > > > >
> > > > > On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> > > criccom...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > It's not happening on 1.8.1 (since there's no pagination in that
> > > > > version),
> > > > > > so I'd count this as a regression. I wouldn't say it's blocking,
> > but
> > > > it's
> > > > > > pretty ugly.
> > > > > >
> > > > > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
> > alex.guz...@airbnb.com
> > > .
> > > > > > invalid
> > > > > > > wrote:
> > > > > >
> > > > > > > I'm not so sure this is a new issue. I think we've seen it on
> our
> > > > > > > production for quite a while.
> > > > > > >
> > > > > > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> > > > > criccom...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
> > > JIRA
> > > > > > here:
> > > > > > > >
> > > > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > > > > > > >
> > > > > > > > Has anyone else seen this?
> > > > > > > >
> > > > > > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > > > > > > sumeet.ma...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1, binding.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> > > > bdbr...@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > We have been running it for the last couple of days. No
> > > issues
> > > > > and
> > > > > > > > seems
> > > > > > > > > > more responsive.
> > > > > > > > > >
> > > > > > > > > > +1, binding
> > > > > > > > > >
> > > > > > > > > > Bolke
> > > > > > > > > >
> > > > > > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > > > > > > maximebeauche...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Dear all,
> > > > > > > > > > >
> > > > > > > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > > > > > > https://dist.apache.org/repos/
> dist/dev/incubator/airflow
> > ,
> > > > > public
> > > > > > > > keys
> > > > > > > > > > > are available
> > > > > > > > > > > at https://dist.apache.org/repos/
> > > > > dist/release/incubator/airflow.
> > > > > > > > > > >
> > > > > > > > > > > Note that RC1 was the first RC (skipped RC0) and was
> > never
> > > > > > > announced
> > > > > > > > > > since
> > > > > > > > > > > it had issues coming out of the oven, so RC2 is the
> first
> > > > > public
> > > > > > > RC.
> > > > > > > > > > >
> > > > > > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed
> > > > "cherries"
> > > > > on
> > > > > > > > top.
> > > > > > > > > I
> > > > > > > > > > > added the JIRAs that were identified blockers and
> > targeted
> > > > > > 1.8.2. I
> > > > > > > > > > > attempted to bring in all of the JIRAs that targeted
> > 1.8.2
> > > > but
> > > > > > > bailed
> > > > > > > > > on
> > > > > > > > > > > the ones that were generating merge conflicts. I also
> > added
> > > > all
> > > > > > of
> > > > > > > > the
> > > > > > > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > > > > > > >
> > > > > > > > > > > Issues fixed:
> > > > > > > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__
> > ColumnOperator
> > > > When
> > > > > > > > Testing
> > > > > > > > > > > Booleans
> > > > > > > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all
> > downstream
> > > > > tasks
> > > > > > > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop
> > > components
> > > > > > > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific
> > > > version
> > > > > > 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Alex Guziel]

Yeah that makes sense. It pages by default at 500 so it explains why we saw
it.

On Mon, Jun 26, 2017 at 2:53 PM, Chris Riccomini 
wrote:

> In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it has
> "Show <25> entries". So it looks like prior to 1.8.2, the pagination was
> broken in the sense that it defaulted to the whole list. We have 479 DAGs
> in one env, and it shows them all. It looks like someone fixed the entry to
> default to 25 now, which exposed the problem for our environments.
>
> On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel  invalid
> > wrote:
>
> > We're running 1.8.0 + some extras, and none of us added pagination
> > recently, and our homepage is paginated. Are you sure it's not the number
> > of dags crossing the threshold? Maybe it's some Flask version thing?
> >
> > On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini 
> > wrote:
> >
> > > Yes, I did the 1.8.1 release.
> > >
> > > On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  > > invalid
> > > > wrote:
> > >
> > > > There's no pagination in 1.8.1? Are you sure?
> > > >
> > > > On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> > criccom...@apache.org>
> > > > wrote:
> > > >
> > > > > It's not happening on 1.8.1 (since there's no pagination in that
> > > > version),
> > > > > so I'd count this as a regression. I wouldn't say it's blocking,
> but
> > > it's
> > > > > pretty ugly.
> > > > >
> > > > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel <
> alex.guz...@airbnb.com
> > .
> > > > > invalid
> > > > > > wrote:
> > > > >
> > > > > > I'm not so sure this is a new issue. I think we've seen it on our
> > > > > > production for quite a while.
> > > > > >
> > > > > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> > > > criccom...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
> > JIRA
> > > > > here:
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > > > > > >
> > > > > > > Has anyone else seen this?
> > > > > > >
> > > > > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > > > > > sumeet.ma...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1, binding.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> > > bdbr...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > We have been running it for the last couple of days. No
> > issues
> > > > and
> > > > > > > seems
> > > > > > > > > more responsive.
> > > > > > > > >
> > > > > > > > > +1, binding
> > > > > > > > >
> > > > > > > > > Bolke
> > > > > > > > >
> > > > > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > > > > > maximebeauche...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Dear all,
> > > > > > > > > >
> > > > > > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow
> ,
> > > > public
> > > > > > > keys
> > > > > > > > > > are available
> > > > > > > > > > at https://dist.apache.org/repos/
> > > > dist/release/incubator/airflow.
> > > > > > > > > >
> > > > > > > > > > Note that RC1 was the first RC (skipped RC0) and was
> never
> > > > > > announced
> > > > > > > > > since
> > > > > > > > > > it had issues coming out of the oven, so RC2 is the first
> > > > public
> > > > > > RC.
> > > > > > > > > >
> > > > > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed
> > > "cherries"
> > > > on
> > > > > > > top.
> > > > > > > > I
> > > > > > > > > > added the JIRAs that were identified blockers and
> targeted
> > > > > 1.8.2. I
> > > > > > > > > > attempted to bring in all of the JIRAs that targeted
> 1.8.2
> > > but
> > > > > > bailed
> > > > > > > > on
> > > > > > > > > > the ones that were generating merge conflicts. I also
> added
> > > all
> > > > > of
> > > > > > > the
> > > > > > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > > > > > >
> > > > > > > > > > Issues fixed:
> > > > > > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__
> ColumnOperator
> > > When
> > > > > > > Testing
> > > > > > > > > > Booleans
> > > > > > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all
> downstream
> > > > tasks
> > > > > > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop
> > components
> > > > > > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific
> > > version
> > > > > and
> > > > > > > > > create
> > > > > > > > > > writable warehouse dir
> > > > > > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > > > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to
> execute
> > > > > > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to
> > > match
> > > > > ASF
> > > > > > > > > > requirements
> > > > > > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > > > > > 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Chris Riccomini]

In 1.8.1, the "DAGs" page has "Show  entries". In 1.8.2, it has
"Show <25> entries". So it looks like prior to 1.8.2, the pagination was
broken in the sense that it defaulted to the whole list. We have 479 DAGs
in one env, and it shows them all. It looks like someone fixed the entry to
default to 25 now, which exposed the problem for our environments.

On Mon, Jun 26, 2017 at 2:47 PM, Alex Guziel  wrote:

> We're running 1.8.0 + some extras, and none of us added pagination
> recently, and our homepage is paginated. Are you sure it's not the number
> of dags crossing the threshold? Maybe it's some Flask version thing?
>
> On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini 
> wrote:
>
> > Yes, I did the 1.8.1 release.
> >
> > On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  > invalid
> > > wrote:
> >
> > > There's no pagination in 1.8.1? Are you sure?
> > >
> > > On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini <
> criccom...@apache.org>
> > > wrote:
> > >
> > > > It's not happening on 1.8.1 (since there's no pagination in that
> > > version),
> > > > so I'd count this as a regression. I wouldn't say it's blocking, but
> > it's
> > > > pretty ugly.
> > > >
> > > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel  .
> > > > invalid
> > > > > wrote:
> > > >
> > > > > I'm not so sure this is a new issue. I think we've seen it on our
> > > > > production for quite a while.
> > > > >
> > > > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> > > criccom...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a
> JIRA
> > > > here:
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > > > > >
> > > > > > Has anyone else seen this?
> > > > > >
> > > > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > > > > sumeet.ma...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > +1, binding.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> > bdbr...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > We have been running it for the last couple of days. No
> issues
> > > and
> > > > > > seems
> > > > > > > > more responsive.
> > > > > > > >
> > > > > > > > +1, binding
> > > > > > > >
> > > > > > > > Bolke
> > > > > > > >
> > > > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > > > > maximebeauche...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Dear all,
> > > > > > > > >
> > > > > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow,
> > > public
> > > > > > keys
> > > > > > > > > are available
> > > > > > > > > at https://dist.apache.org/repos/
> > > dist/release/incubator/airflow.
> > > > > > > > >
> > > > > > > > > Note that RC1 was the first RC (skipped RC0) and was never
> > > > > announced
> > > > > > > > since
> > > > > > > > > it had issues coming out of the oven, so RC2 is the first
> > > public
> > > > > RC.
> > > > > > > > >
> > > > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed
> > "cherries"
> > > on
> > > > > > top.
> > > > > > > I
> > > > > > > > > added the JIRAs that were identified blockers and targeted
> > > > 1.8.2. I
> > > > > > > > > attempted to bring in all of the JIRAs that targeted 1.8.2
> > but
> > > > > bailed
> > > > > > > on
> > > > > > > > > the ones that were generating merge conflicts. I also added
> > all
> > > > of
> > > > > > the
> > > > > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > > > > >
> > > > > > > > > Issues fixed:
> > > > > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator
> > When
> > > > > > Testing
> > > > > > > > > Booleans
> > > > > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> > > tasks
> > > > > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop
> components
> > > > > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific
> > version
> > > > and
> > > > > > > > create
> > > > > > > > > writable warehouse dir
> > > > > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > > > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to
> > match
> > > > ASF
> > > > > > > > > requirements
> > > > > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > > > > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom
> filters
> > > in
> > > > > > Jinja2
> > > > > > > > > templates
> > > > > > > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on
> > first
> > > > > row[]
> > > > > > > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > > > > > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old
> flask
> > > > > > versions
> > > > > > > > > 7a02841 [AIRFLOW-1074] Don't count 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Alex Guziel]

We're running 1.8.0 + some extras, and none of us added pagination
recently, and our homepage is paginated. Are you sure it's not the number
of dags crossing the threshold? Maybe it's some Flask version thing?

On Mon, Jun 26, 2017 at 2:45 PM, Chris Riccomini 
wrote:

> Yes, I did the 1.8.1 release.
>
> On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  invalid
> > wrote:
>
> > There's no pagination in 1.8.1? Are you sure?
> >
> > On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini 
> > wrote:
> >
> > > It's not happening on 1.8.1 (since there's no pagination in that
> > version),
> > > so I'd count this as a regression. I wouldn't say it's blocking, but
> it's
> > > pretty ugly.
> > >
> > > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel  > > invalid
> > > > wrote:
> > >
> > > > I'm not so sure this is a new issue. I think we've seen it on our
> > > > production for quite a while.
> > > >
> > > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> > criccom...@apache.org>
> > > > wrote:
> > > >
> > > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA
> > > here:
> > > > >
> > > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > > > >
> > > > > Has anyone else seen this?
> > > > >
> > > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > > > sumeet.ma...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1, binding.
> > > > > >
> > > > > >
> > > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin <
> bdbr...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > We have been running it for the last couple of days. No issues
> > and
> > > > > seems
> > > > > > > more responsive.
> > > > > > >
> > > > > > > +1, binding
> > > > > > >
> > > > > > > Bolke
> > > > > > >
> > > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > > > maximebeauche...@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Dear all,
> > > > > > > >
> > > > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow,
> > public
> > > > > keys
> > > > > > > > are available
> > > > > > > > at https://dist.apache.org/repos/
> > dist/release/incubator/airflow.
> > > > > > > >
> > > > > > > > Note that RC1 was the first RC (skipped RC0) and was never
> > > > announced
> > > > > > > since
> > > > > > > > it had issues coming out of the oven, so RC2 is the first
> > public
> > > > RC.
> > > > > > > >
> > > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed
> "cherries"
> > on
> > > > > top.
> > > > > > I
> > > > > > > > added the JIRAs that were identified blockers and targeted
> > > 1.8.2. I
> > > > > > > > attempted to bring in all of the JIRAs that targeted 1.8.2
> but
> > > > bailed
> > > > > > on
> > > > > > > > the ones that were generating merge conflicts. I also added
> all
> > > of
> > > > > the
> > > > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > > > >
> > > > > > > > Issues fixed:
> > > > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator
> When
> > > > > Testing
> > > > > > > > Booleans
> > > > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> > tasks
> > > > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > > > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific
> version
> > > and
> > > > > > > create
> > > > > > > > writable warehouse dir
> > > > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to
> match
> > > ASF
> > > > > > > > requirements
> > > > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > > > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters
> > in
> > > > > Jinja2
> > > > > > > > templates
> > > > > > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on
> first
> > > > row[]
> > > > > > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > > > > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
> > > > > versions
> > > > > > > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for
> concurrency
> > > > > limits
> > > > > > > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > > > > > > > TaskInstanceModelView
> > > > > > > > d1c64ab [AIRFLOW-1038] Specify celery serialization options
> > > > > explicitly
> > > > > > > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > > > > > > > 9fca409 [AIRFLOW-993] Update date inference logic
> > > > > > > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook
> > > modification
> > > > > > time
> > > > > > > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google
> BigQuery
> > > > > change
> > > > > > > > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > > > > > > > 7f33f6e [AIRFLOW-1266] Increase width of 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Alex Guziel]

Also, still see `airflow.exceptions.AirflowConfigException: section/key
[celery/celery_ssl_active] not found in config` when running with celery
executor. Likely this PR fixes,
https://github.com/apache/incubator-airflow/pull/2341, but it needs rebase
to try to pass tests.

On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  wrote:

> There's no pagination in 1.8.1? Are you sure?
>
> On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini 
> wrote:
>
>> It's not happening on 1.8.1 (since there's no pagination in that version),
>> so I'd count this as a regression. I wouldn't say it's blocking, but it's
>> pretty ugly.
>>
>> On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel > .invalid
>> > wrote:
>>
>> > I'm not so sure this is a new issue. I think we've seen it on our
>> > production for quite a while.
>> >
>> > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini > >
>> > wrote:
>> >
>> > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA
>> here:
>> > >
>> > > https://issues.apache.org/jira/browse/AIRFLOW-1348
>> > >
>> > > Has anyone else seen this?
>> > >
>> > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
>> > sumeet.ma...@gmail.com>
>> > > wrote:
>> > >
>> > > > +1, binding.
>> > > >
>> > > >
>> > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin 
>> > > wrote:
>> > > >
>> > > > > We have been running it for the last couple of days. No issues and
>> > > seems
>> > > > > more responsive.
>> > > > >
>> > > > > +1, binding
>> > > > >
>> > > > > Bolke
>> > > > >
>> > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
>> > > > maximebeauche...@gmail.com>
>> > > > > wrote:
>> > > > > >
>> > > > > > Dear all,
>> > > > > >
>> > > > > > 1.8.2 RC2 is baked and available at:
>> > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow,
>> public
>> > > keys
>> > > > > > are available
>> > > > > > at https://dist.apache.org/repos/dist/release/incubator/airflow
>> .
>> > > > > >
>> > > > > > Note that RC1 was the first RC (skipped RC0) and was never
>> > announced
>> > > > > since
>> > > > > > it had issues coming out of the oven, so RC2 is the first public
>> > RC.
>> > > > > >
>> > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries"
>> on
>> > > top.
>> > > > I
>> > > > > > added the JIRAs that were identified blockers and targeted
>> 1.8.2. I
>> > > > > > attempted to bring in all of the JIRAs that targeted 1.8.2 but
>> > bailed
>> > > > on
>> > > > > > the ones that were generating merge conflicts. I also added all
>> of
>> > > the
>> > > > > > JIRAs that we've been running in production at Airbnb.
>> > > > > >
>> > > > > > Issues fixed:
>> > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When
>> > > Testing
>> > > > > > Booleans
>> > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
>> > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
>> > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version
>> and
>> > > > > create
>> > > > > > writable warehouse dir
>> > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
>> > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
>> > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match
>> ASF
>> > > > > > requirements
>> > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
>> > > > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in
>> > > Jinja2
>> > > > > > templates
>> > > > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first
>> > row[]
>> > > > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
>> > > > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
>> > > versions
>> > > > > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency
>> > > limits
>> > > > > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
>> > > > > > TaskInstanceModelView
>> > > > > > d1c64ab [AIRFLOW-1038] Specify celery serialization options
>> > > explicitly
>> > > > > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
>> > > > > > 9fca409 [AIRFLOW-993] Update date inference logic
>> > > > > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook
>> modification
>> > > > time
>> > > > > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery
>> > > change
>> > > > > > acd0166 [AIRFLOW-1263] Dynamic height for charts
>> > > > > > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
>> > > > > > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
>> > > > > > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
>> > > > > > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_
>> > dagrun
>> > > > > > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
>> > > > > > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
>> > > > > > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Chris Riccomini]

Yes, I did the 1.8.1 release.

On Mon, Jun 26, 2017 at 2:44 PM, Alex Guziel  wrote:

> There's no pagination in 1.8.1? Are you sure?
>
> On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini 
> wrote:
>
> > It's not happening on 1.8.1 (since there's no pagination in that
> version),
> > so I'd count this as a regression. I wouldn't say it's blocking, but it's
> > pretty ugly.
> >
> > On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel  > invalid
> > > wrote:
> >
> > > I'm not so sure this is a new issue. I think we've seen it on our
> > > production for quite a while.
> > >
> > > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini <
> criccom...@apache.org>
> > > wrote:
> > >
> > > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA
> > here:
> > > >
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > > >
> > > > Has anyone else seen this?
> > > >
> > > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > > sumeet.ma...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1, binding.
> > > > >
> > > > >
> > > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin  >
> > > > wrote:
> > > > >
> > > > > > We have been running it for the last couple of days. No issues
> and
> > > > seems
> > > > > > more responsive.
> > > > > >
> > > > > > +1, binding
> > > > > >
> > > > > > Bolke
> > > > > >
> > > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > > maximebeauche...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > Dear all,
> > > > > > >
> > > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow,
> public
> > > > keys
> > > > > > > are available
> > > > > > > at https://dist.apache.org/repos/
> dist/release/incubator/airflow.
> > > > > > >
> > > > > > > Note that RC1 was the first RC (skipped RC0) and was never
> > > announced
> > > > > > since
> > > > > > > it had issues coming out of the oven, so RC2 is the first
> public
> > > RC.
> > > > > > >
> > > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries"
> on
> > > > top.
> > > > > I
> > > > > > > added the JIRAs that were identified blockers and targeted
> > 1.8.2. I
> > > > > > > attempted to bring in all of the JIRAs that targeted 1.8.2 but
> > > bailed
> > > > > on
> > > > > > > the ones that were generating merge conflicts. I also added all
> > of
> > > > the
> > > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > > >
> > > > > > > Issues fixed:
> > > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When
> > > > Testing
> > > > > > > Booleans
> > > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream
> tasks
> > > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version
> > and
> > > > > > create
> > > > > > > writable warehouse dir
> > > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match
> > ASF
> > > > > > > requirements
> > > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters
> in
> > > > Jinja2
> > > > > > > templates
> > > > > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first
> > > row[]
> > > > > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > > > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
> > > > versions
> > > > > > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency
> > > > limits
> > > > > > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > > > > > > TaskInstanceModelView
> > > > > > > d1c64ab [AIRFLOW-1038] Specify celery serialization options
> > > > explicitly
> > > > > > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > > > > > > 9fca409 [AIRFLOW-993] Update date inference logic
> > > > > > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook
> > modification
> > > > > time
> > > > > > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery
> > > > change
> > > > > > > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > > > > > > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> > > > > > > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> > > > > > > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> > > > > > > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_
> > > dagrun
> > > > > > > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> > > > > > > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> > > > > > > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty
> name
> > > > > > > 1232b6a [AIRFLOW-1243] DAGs table has no default entries to
> show
> > > > > > > b0ba3c9 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Alex Guziel]

There's no pagination in 1.8.1? Are you sure?

On Mon, Jun 26, 2017 at 2:37 PM, Chris Riccomini 
wrote:

> It's not happening on 1.8.1 (since there's no pagination in that version),
> so I'd count this as a regression. I wouldn't say it's blocking, but it's
> pretty ugly.
>
> On Mon, Jun 26, 2017 at 2:34 PM, Alex Guziel  invalid
> > wrote:
>
> > I'm not so sure this is a new issue. I think we've seen it on our
> > production for quite a while.
> >
> > On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini 
> > wrote:
> >
> > > I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA
> here:
> > >
> > > https://issues.apache.org/jira/browse/AIRFLOW-1348
> > >
> > > Has anyone else seen this?
> > >
> > > On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari <
> > sumeet.ma...@gmail.com>
> > > wrote:
> > >
> > > > +1, binding.
> > > >
> > > >
> > > > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin 
> > > wrote:
> > > >
> > > > > We have been running it for the last couple of days. No issues and
> > > seems
> > > > > more responsive.
> > > > >
> > > > > +1, binding
> > > > >
> > > > > Bolke
> > > > >
> > > > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > > > maximebeauche...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Dear all,
> > > > > >
> > > > > > 1.8.2 RC2 is baked and available at:
> > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow, public
> > > keys
> > > > > > are available
> > > > > > at https://dist.apache.org/repos/dist/release/incubator/airflow.
> > > > > >
> > > > > > Note that RC1 was the first RC (skipped RC0) and was never
> > announced
> > > > > since
> > > > > > it had issues coming out of the oven, so RC2 is the first public
> > RC.
> > > > > >
> > > > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on
> > > top.
> > > > I
> > > > > > added the JIRAs that were identified blockers and targeted
> 1.8.2. I
> > > > > > attempted to bring in all of the JIRAs that targeted 1.8.2 but
> > bailed
> > > > on
> > > > > > the ones that were generating merge conflicts. I also added all
> of
> > > the
> > > > > > JIRAs that we've been running in production at Airbnb.
> > > > > >
> > > > > > Issues fixed:
> > > > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When
> > > Testing
> > > > > > Booleans
> > > > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> > > > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > > > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version
> and
> > > > > create
> > > > > > writable warehouse dir
> > > > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match
> ASF
> > > > > > requirements
> > > > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in
> > > Jinja2
> > > > > > templates
> > > > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first
> > row[]
> > > > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
> > > versions
> > > > > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency
> > > limits
> > > > > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > > > > > TaskInstanceModelView
> > > > > > d1c64ab [AIRFLOW-1038] Specify celery serialization options
> > > explicitly
> > > > > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > > > > > 9fca409 [AIRFLOW-993] Update date inference logic
> > > > > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook
> modification
> > > > time
> > > > > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery
> > > change
> > > > > > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > > > > > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> > > > > > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> > > > > > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> > > > > > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_
> > dagrun
> > > > > > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> > > > > > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> > > > > > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> > > > > > 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> > > > > > b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> > > > > > c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> > > > > > 51a83cc [AIRFLOW-1199] Fix create modal
> > > > > > cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an
> empty
> > > key
> > > > > > 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by
> > execution_date
> > > > > > f446c08 [AIRFLOW-1145] 

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Alex Guziel]

I'm not so sure this is a new issue. I think we've seen it on our
production for quite a while.

On Mon, Jun 26, 2017 at 2:31 PM, Chris Riccomini 
wrote:

> I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA here:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1348
>
> Has anyone else seen this?
>
> On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari 
> wrote:
>
> > +1, binding.
> >
> >
> > On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin 
> wrote:
> >
> > > We have been running it for the last couple of days. No issues and
> seems
> > > more responsive.
> > >
> > > +1, binding
> > >
> > > Bolke
> > >
> > > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> > maximebeauche...@gmail.com>
> > > wrote:
> > > >
> > > > Dear all,
> > > >
> > > > 1.8.2 RC2 is baked and available at:
> > > > https://dist.apache.org/repos/dist/dev/incubator/airflow, public
> keys
> > > > are available
> > > > at https://dist.apache.org/repos/dist/release/incubator/airflow.
> > > >
> > > > Note that RC1 was the first RC (skipped RC0) and was never announced
> > > since
> > > > it had issues coming out of the oven, so RC2 is the first public RC.
> > > >
> > > > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on
> top.
> > I
> > > > added the JIRAs that were identified blockers and targeted 1.8.2. I
> > > > attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed
> > on
> > > > the ones that were generating merge conflicts. I also added all of
> the
> > > > JIRAs that we've been running in production at Airbnb.
> > > >
> > > > Issues fixed:
> > > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When
> Testing
> > > > Booleans
> > > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> > > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and
> > > create
> > > > writable warehouse dir
> > > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
> > > > requirements
> > > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in
> Jinja2
> > > > templates
> > > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
> > > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask
> versions
> > > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency
> limits
> > > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > > > TaskInstanceModelView
> > > > d1c64ab [AIRFLOW-1038] Specify celery serialization options
> explicitly
> > > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > > > 9fca409 [AIRFLOW-993] Update date inference logic
> > > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification
> > time
> > > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery
> change
> > > > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > > > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> > > > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> > > > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> > > > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
> > > > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> > > > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> > > > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> > > > 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> > > > b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> > > > c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> > > > 51a83cc [AIRFLOW-1199] Fix create modal
> > > > cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty
> key
> > > > 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
> > > > f446c08 [AIRFLOW-1145] Fix closest_date_partition function with
> before
> > > set
> > > > to True If we're looking for the closest date before, we should take
> > the
> > > > latest date in the list of date before.
> > > > 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
> > > > bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters
> > > separately
> > > > 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
> > > > 777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
> > > >
> > > > Max
> > >
> > >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Chris Riccomini]

I am seeing a strange UI behavior on 1.8.2.RC2. I've opened a JIRA here:

https://issues.apache.org/jira/browse/AIRFLOW-1348

Has anyone else seen this?

On Mon, Jun 26, 2017 at 3:27 AM, Sumit Maheshwari 
wrote:

> +1, binding.
>
>
> On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin  wrote:
>
> > We have been running it for the last couple of days. No issues and seems
> > more responsive.
> >
> > +1, binding
> >
> > Bolke
> >
> > > On 25 Jun 2017, at 01:10, Maxime Beauchemin <
> maximebeauche...@gmail.com>
> > wrote:
> > >
> > > Dear all,
> > >
> > > 1.8.2 RC2 is baked and available at:
> > > https://dist.apache.org/repos/dist/dev/incubator/airflow, public keys
> > > are available
> > > at https://dist.apache.org/repos/dist/release/incubator/airflow.
> > >
> > > Note that RC1 was the first RC (skipped RC0) and was never announced
> > since
> > > it had issues coming out of the oven, so RC2 is the first public RC.
> > >
> > > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on top.
> I
> > > added the JIRAs that were identified blockers and targeted 1.8.2. I
> > > attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed
> on
> > > the ones that were generating merge conflicts. I also added all of the
> > > JIRAs that we've been running in production at Airbnb.
> > >
> > > Issues fixed:
> > > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing
> > > Booleans
> > > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> > > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and
> > create
> > > writable warehouse dir
> > > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
> > > requirements
> > > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2
> > > templates
> > > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
> > > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
> > > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
> > > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > > TaskInstanceModelView
> > > d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
> > > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > > 9fca409 [AIRFLOW-993] Update date inference logic
> > > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification
> time
> > > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
> > > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> > > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> > > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> > > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
> > > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> > > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> > > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> > > 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> > > b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> > > c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> > > 51a83cc [AIRFLOW-1199] Fix create modal
> > > cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
> > > 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
> > > f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before
> > set
> > > to True If we're looking for the closest date before, we should take
> the
> > > latest date in the list of date before.
> > > 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
> > > bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters
> > separately
> > > 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
> > > 777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
> > >
> > > Max
> >
> >
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Sumit Maheshwari]

+1, binding.


On Mon, Jun 26, 2017 at 3:49 PM, Bolke de Bruin  wrote:

> We have been running it for the last couple of days. No issues and seems
> more responsive.
>
> +1, binding
>
> Bolke
>
> > On 25 Jun 2017, at 01:10, Maxime Beauchemin 
> wrote:
> >
> > Dear all,
> >
> > 1.8.2 RC2 is baked and available at:
> > https://dist.apache.org/repos/dist/dev/incubator/airflow, public keys
> > are available
> > at https://dist.apache.org/repos/dist/release/incubator/airflow.
> >
> > Note that RC1 was the first RC (skipped RC0) and was never announced
> since
> > it had issues coming out of the oven, so RC2 is the first public RC.
> >
> > 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on top. I
> > added the JIRAs that were identified blockers and targeted 1.8.2. I
> > attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed on
> > the ones that were generating merge conflicts. I also added all of the
> > JIRAs that we've been running in production at Airbnb.
> >
> > Issues fixed:
> > 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing
> > Booleans
> > 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> > 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> > 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and
> create
> > writable warehouse dir
> > 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> > 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> > 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
> > requirements
> > 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> > 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2
> > templates
> > 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
> > b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> > a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
> > 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
> > a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> > TaskInstanceModelView
> > d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
> > b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> > 9fca409 [AIRFLOW-993] Update date inference logic
> > 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
> > c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
> > acd0166 [AIRFLOW-1263] Dynamic height for charts
> > 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> > fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> > 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> > 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
> > bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> > 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> > 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> > 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> > b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> > c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> > 51a83cc [AIRFLOW-1199] Fix create modal
> > cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
> > 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
> > f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before
> set
> > to True If we're looking for the closest date before, we should take the
> > latest date in the list of date before.
> > 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
> > bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters
> separately
> > 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
> > 777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
> >
> > Max
>
>

Re: [VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Bolke de Bruin]

We have been running it for the last couple of days. No issues and seems more 
responsive.

+1, binding

Bolke

> On 25 Jun 2017, at 01:10, Maxime Beauchemin  
> wrote:
> 
> Dear all,
> 
> 1.8.2 RC2 is baked and available at:
> https://dist.apache.org/repos/dist/dev/incubator/airflow, public keys
> are available
> at https://dist.apache.org/repos/dist/release/incubator/airflow.
> 
> Note that RC1 was the first RC (skipped RC0) and was never announced since
> it had issues coming out of the oven, so RC2 is the first public RC.
> 
> 1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on top. I
> added the JIRAs that were identified blockers and targeted 1.8.2. I
> attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed on
> the ones that were generating merge conflicts. I also added all of the
> JIRAs that we've been running in production at Airbnb.
> 
> Issues fixed:
> 9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing
> Booleans
> 333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
> 93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
> 33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and create
> writable warehouse dir
> 7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
> 570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
> 3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
> requirements
> 69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
> 9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2
> templates
> 6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
> b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
> a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
> 7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
> a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
> TaskInstanceModelView
> d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
> b4ee88a [AIRFLOW-1036] Randomize exponential backoff
> 9fca409 [AIRFLOW-993] Update date inference logic
> 272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
> c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
> acd0166 [AIRFLOW-1263] Dynamic height for charts
> 7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
> fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
> 2e9eee3 [AIRFLOW-1282] Fix known event column sorting
> 2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
> bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
> 57d5bcd [AIRFLOW-1281] Sort variables by key field by default
> 802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
> 1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
> b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
> c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
> 51a83cc [AIRFLOW-1199] Fix create modal
> cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
> 5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
> f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set
> to True If we're looking for the closest date before, we should take the
> latest date in the list of date before.
> 93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
> bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
> 093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
> 777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
> 
> Max

[VOTE] Release Airflow 1.8.2 based on Airflow 1.8.2 RC2

[Maxime Beauchemin]

Dear all,

1.8.2 RC2 is baked and available at:
https://dist.apache.org/repos/dist/dev/incubator/airflow, public keys
are available
at https://dist.apache.org/repos/dist/release/incubator/airflow.

Note that RC1 was the first RC (skipped RC0) and was never announced since
it had issues coming out of the oven, so RC2 is the first public RC.

1.8.2 RC2 is build on to of 1.8.1 with these listed "cherries" on top. I
added the JIRAs that were identified blockers and targeted 1.8.2. I
attempted to bring in all of the JIRAs that targeted 1.8.2 but bailed on
the ones that were generating merge conflicts. I also added all of the
JIRAs that we've been running in production at Airbnb.

Issues fixed:
9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing
Booleans
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
93825d5 [AIRFLOW-XXX] Re-enable caching for hadoop components
33a9dcb [AIRFLOW-XXX] Pin Hive and Hadoop to a specific version and create
writable warehouse dir
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF
requirements
69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2
templates
6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
a2c18a5 [AIRFLOW-1064] Change default sort to job_id for
TaskInstanceModelView
d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
b4ee88a [AIRFLOW-1036] Randomize exponential backoff
9fca409 [AIRFLOW-993] Update date inference logic
272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
acd0166 [AIRFLOW-1263] Dynamic height for charts
7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
2e9eee3 [AIRFLOW-1282] Fix known event column sorting
2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
57d5bcd [AIRFLOW-1281] Sort variables by key field by default
802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
51a83cc [AIRFLOW-1199] Fix create modal
cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set
to True If we're looking for the closest date before, we should take the
latest date in the list of date before.
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
777f181 [AIRFLOW-1168] Add closing() to all connections and cursors

Max