Re: 1.8.1 release update

2017-04-07 Thread Chris Riccomini 
> Also some jiras still need to be cherry-picked into 1.8.1 and some will
create conflicts.

Any idea how to determine which ones? Going JIRA by JIRA every few days
really sucks.

On Fri, Apr 7, 2017 at 1:29 PM, Bolke de Bruin <bdbr...@gmail.com> wrote:

> Agree.
>
> Airflow-1000 can be merged. I think you want to put a big notice somewhere.
>
> Also some jiras still need to be cherry-picked into 1.8.1 and some will
> create conflicts.
>
> Bolke
>
> Sent from my iPhone
>
> > On 7 Apr 2017, at 20:16, Chris Riccomini <criccom...@apache.org> wrote:
> >
> > Hey all,
> >
> > I'm still targeting to cut 1.8.1 RC0 for April 17. Doing a checkup to see
> > where we stand right now. There's about one week left.
> >
> > We look in pretty good shape. There are two blocker bugs left:
> >
> > AIRFLOW-1055
> > https://issues.apache.org/jira/browse/AIRFLOW-1055
> > (no PR)
> > I'm going to drop this to critical instead of blocker.
> >
> > AIRFLOW-1000
> > https://issues.apache.org/jira/browse/AIRFLOW-1000
> > https://github.com/apache/incubator-airflow/pull/2172
> > @bolke, Can we merge AIRFLOW-1000? Dan gave a LGTM on it.
> >
> > Cheers,
> > Chris
> >
> > On Mon, Apr 3, 2017 at 3:12 PM, Chris Riccomini <criccom...@apache.org>
> > wrote:
> >
> >> Another blocker was added:
> >>
> >>  AIRFLOW-1011 https://github.com/apache/incubator-airflow/pull/2179
> >>
> >> On Mon, Apr 3, 2017 at 1:29 PM, Chris Riccomini <criccom...@apache.org>
> >> wrote:
> >>
> >>> Hey all,
> >>>
> >>> Here's the status of the 1.8.1 release:
> >>>
> >>> * Targeting April 17 RC1 vote
> >>> * There are five blocker bugs
> >>> * We will probably remove AIRFLOW-1055 and AIRFLOW-1019 from the
> blocker
> >>> list for 1.8.1
> >>>
> >>> The list of open 1.8.1 issues is here:
> >>>
> >>> https://issues.apache.org/jira/issues/?jql=project%20%3D%
> >>> 20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%20Progress%
> >>> 22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
> >>>
> >>> The following blocker bugs have PRs up, and need reviewing:
> >>>
> >>>  AIRFLOW-1013 https://github.com/apache/incubator-airflow/pull/2203
> >>>  AIRFLOW-1000 https://github.com/apache/incubator-airflow/pull/2172
> >>>  AIRFLOW-1001 https://github.com/apache/incubator-airflow/pull/2213
> >>>
> >>> PLEASE have a look at this, so we can get them merged.
> >>>
> >>> Cheers,
> >>> Chris
> >>>
> >>
> >>
>

Re: 1.8.1 release update

2017-04-07 Thread Bolke de Bruin 
Agree. 

Airflow-1000 can be merged. I think you want to put a big notice somewhere. 

Also some jiras still need to be cherry-picked into 1.8.1 and some will create 
conflicts. 

Bolke

Sent from my iPhone

> On 7 Apr 2017, at 20:16, Chris Riccomini <criccom...@apache.org> wrote:
> 
> Hey all,
> 
> I'm still targeting to cut 1.8.1 RC0 for April 17. Doing a checkup to see
> where we stand right now. There's about one week left.
> 
> We look in pretty good shape. There are two blocker bugs left:
> 
> AIRFLOW-1055
> https://issues.apache.org/jira/browse/AIRFLOW-1055
> (no PR)
> I'm going to drop this to critical instead of blocker.
> 
> AIRFLOW-1000
> https://issues.apache.org/jira/browse/AIRFLOW-1000
> https://github.com/apache/incubator-airflow/pull/2172
> @bolke, Can we merge AIRFLOW-1000? Dan gave a LGTM on it.
> 
> Cheers,
> Chris
> 
> On Mon, Apr 3, 2017 at 3:12 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
> 
>> Another blocker was added:
>> 
>>  AIRFLOW-1011 https://github.com/apache/incubator-airflow/pull/2179
>> 
>> On Mon, Apr 3, 2017 at 1:29 PM, Chris Riccomini <criccom...@apache.org>
>> wrote:
>> 
>>> Hey all,
>>> 
>>> Here's the status of the 1.8.1 release:
>>> 
>>> * Targeting April 17 RC1 vote
>>> * There are five blocker bugs
>>> * We will probably remove AIRFLOW-1055 and AIRFLOW-1019 from the blocker
>>> list for 1.8.1
>>> 
>>> The list of open 1.8.1 issues is here:
>>> 
>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%
>>> 20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%20Progress%
>>> 22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>> 
>>> The following blocker bugs have PRs up, and need reviewing:
>>> 
>>>  AIRFLOW-1013 https://github.com/apache/incubator-airflow/pull/2203
>>>  AIRFLOW-1000 https://github.com/apache/incubator-airflow/pull/2172
>>>  AIRFLOW-1001 https://github.com/apache/incubator-airflow/pull/2213
>>> 
>>> PLEASE have a look at this, so we can get them merged.
>>> 
>>> Cheers,
>>> Chris
>>> 
>> 
>>

Re: 1.8.1 release update

2017-04-07 Thread Chris Riccomini 
Hey all,

I'm still targeting to cut 1.8.1 RC0 for April 17. Doing a checkup to see
where we stand right now. There's about one week left.

We look in pretty good shape. There are two blocker bugs left:

AIRFLOW-1055
https://issues.apache.org/jira/browse/AIRFLOW-1055
(no PR)
I'm going to drop this to critical instead of blocker.

AIRFLOW-1000
https://issues.apache.org/jira/browse/AIRFLOW-1000
https://github.com/apache/incubator-airflow/pull/2172
@bolke, Can we merge AIRFLOW-1000? Dan gave a LGTM on it.

Cheers,
Chris

On Mon, Apr 3, 2017 at 3:12 PM, Chris Riccomini <criccom...@apache.org>
wrote:

> Another blocker was added:
>
>   AIRFLOW-1011 https://github.com/apache/incubator-airflow/pull/2179
>
> On Mon, Apr 3, 2017 at 1:29 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
>> Hey all,
>>
>> Here's the status of the 1.8.1 release:
>>
>> * Targeting April 17 RC1 vote
>> * There are five blocker bugs
>> * We will probably remove AIRFLOW-1055 and AIRFLOW-1019 from the blocker
>> list for 1.8.1
>>
>> The list of open 1.8.1 issues is here:
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%
>> 20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%20Progress%
>> 22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>
>> The following blocker bugs have PRs up, and need reviewing:
>>
>>   AIRFLOW-1013 https://github.com/apache/incubator-airflow/pull/2203
>>   AIRFLOW-1000 https://github.com/apache/incubator-airflow/pull/2172
>>   AIRFLOW-1001 https://github.com/apache/incubator-airflow/pull/2213
>>
>> PLEASE have a look at this, so we can get them merged.
>>
>> Cheers,
>> Chris
>>
>
>

Re: 1.8.1 release

2017-04-05 Thread Ruslan Dautkhanov 
https://issues.apache.org/jira/browse/AIRFLOW-1033

Is also 1.8 and not 1.8.0 blocker ?

Thanks.



-- 
Ruslan Dautkhanov

On Mon, Apr 3, 2017 at 3:20 PM, Bolke de Bruin <bdbr...@gmail.com> wrote:

> Hi Chris,
>
> Please add:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1011 <
> https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR:
> https://github.com/apache/incubator-airflow/pull/2179 <
> https://github.com/apache/incubator-airflow/pull/2179>)
>
> To the blocker list. It was missed due to the “Airflow 1.8” affected
> version tag instead of “1.8.0”. I fixed that for you.
>
> Bolke.
>
> > On 3 Apr 2017, at 22:06, Bolke de Bruin <bdbr...@gmail.com> wrote:
> >
> > I have a PR out for
> >
> > * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <
> https://github.com/apache/incubator-airflow/pull/2213>
> > * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <
> https://github.com/apache/incubator-airflow/pull/2212>
> >
> > Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I
> don’t think it is a blocker: it happens only with @once dags that have a
> SLA, hardly very common. Nevertheless a fix would be nice obviously.
> >
> > Bolke
> >
> >> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbr...@gmail.com  bdbr...@gmail.com>> wrote:
> >>
> >> Hey Chris,
> >>
> >> AIRFLOW-1000 has a PR out. You might want to discuss it on the list
> what the impact is though.
> >> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for
> the scheduler log files. I did downgrade from blocker to critical.
> >> AIRFLOW-719 Has a PR, including much needed increased test coverage,
> that I am pretty sure is working, but needs verification (plz @Alex)
> >>
> >> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
> around at the moment.
> >>
> >> Furthermore, do you have any timelines for getting to the release? Can
> I help in any way? You might want to chase a couple of times ;-).
> >>
> >> Bolke
> >>
> >>> On 30 Mar 2017, at 19:48, siddharth anand <san...@apache.org  san...@apache.org>> wrote:
> >>>
> >>> Chris,
> >>> I've submitted PRs for :
> >>>
> >>>  - PR [AIRFLOW-1013] :
> >>>  https://github.com/apache/incubator-airflow/pull/2203 <
> https://github.com/apache/incubator-airflow/pull/2203>
> >>>  - PR [AIRFLOW-1054]:
> >>>  https://github.com/apache/incubator-airflow/pull/2201 <
> https://github.com/apache/incubator-airflow/pull/2201>
> >>>
> >>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> >>> created if catchup=False :
> >>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <
> https://issues.apache.org/jira/browse/AIRFLOW-1055>
> >>>
> >>> I have a PR that works for this, but will need to add unit tests for
> it as
> >>> well as for AIRFLOW-1013.
> >>>
> >>> -s
> >>>
> >>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org>
> wrote:
> >>>
> >>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013
> was a
> >>>> blocker. I will have a PR shortly.
> >>>> -s
> >>>>
> >>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <
> criccom...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> The following three JIRAs were not merged into the v1-8-test branch,
> but
> >>>>> are listed as part of the 1.8.1 release:
> >>>>>
> >>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> >>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> >>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> >>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
> >>>>>
> >>>>> I'm going to merge them in now.
> >>>>>
> >>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <
> criccom...@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey Bolke,
> >>>>>>
> >>>>>> Great. Assuming your PR is committed, that leaves five blockers:
> >>>>>>
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
> >>>>>> https://issues.apache.

Re: 1.8.1 release update

2017-04-03 Thread Chris Riccomini 
Another blocker was added:

  AIRFLOW-1011 https://github.com/apache/incubator-airflow/pull/2179

On Mon, Apr 3, 2017 at 1:29 PM, Chris Riccomini <criccom...@apache.org>
wrote:

> Hey all,
>
> Here's the status of the 1.8.1 release:
>
> * Targeting April 17 RC1 vote
> * There are five blocker bugs
> * We will probably remove AIRFLOW-1055 and AIRFLOW-1019 from the blocker
> list for 1.8.1
>
> The list of open 1.8.1 issues is here:
>
> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>
> The following blocker bugs have PRs up, and need reviewing:
>
>   AIRFLOW-1013 https://github.com/apache/incubator-airflow/pull/2203
>   AIRFLOW-1000 https://github.com/apache/incubator-airflow/pull/2172
>   AIRFLOW-1001 https://github.com/apache/incubator-airflow/pull/2213
>
> PLEASE have a look at this, so we can get them merged.
>
> Cheers,
> Chris
>

Re: 1.8.1 release

2017-04-03 Thread Bolke de Bruin 
Done.


> On 3 Apr 2017, at 22:22, Chris Riccomini <criccom...@apache.org> wrote:
> 
> Hey Bolke,
> 
> > Furthermore, do you have any timelines for getting to the release
> 
> Let's target April 17 to begin a vote for RC.
> 
> > Can I help in any way
> 
> I need people to review things. The more reviewers, the better.
> 
> Also, can you please reach a conclusion on whether the issues you raised 
> (AIRFLOW-1019 and AIRFLOW-1013) are blockers or not (or just change the 
> status unilaterally if people aren't responding)?
> 
> Cheers,
> Chris
> 
> On Mon, Apr 3, 2017 at 1:06 PM, Bolke de Bruin <bdbr...@gmail.com 
> <mailto:bdbr...@gmail.com>> wrote:
> I have a PR out for 
> 
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 
> <https://github.com/apache/incubator-airflow/pull/2213>
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 
> <https://github.com/apache/incubator-airflow/pull/2212>
> 
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t 
> think it is a blocker: it happens only with @once dags that have a SLA, 
> hardly very common. Nevertheless a fix would be nice obviously.
> 
> Bolke
> 
>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbr...@gmail.com 
>> <mailto:bdbr...@gmail.com>> wrote:
>> 
>> Hey Chris,
>> 
>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the 
>> impact is though.
>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the 
>> scheduler log files. I did downgrade from blocker to critical.
>> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I 
>> am pretty sure is working, but needs verification (plz @Alex)
>> 
>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around 
>> at the moment.
>> 
>> Furthermore, do you have any timelines for getting to the release? Can I 
>> help in any way? You might want to chase a couple of times ;-).
>> 
>> Bolke
>> 
>>> On 30 Mar 2017, at 19:48, siddharth anand <san...@apache.org 
>>> <mailto:san...@apache.org>> wrote:
>>> 
>>> Chris,
>>> I've submitted PRs for :
>>> 
>>>  - PR [AIRFLOW-1013] :
>>>  https://github.com/apache/incubator-airflow/pull/2203 
>>> <https://github.com/apache/incubator-airflow/pull/2203>
>>>  - PR [AIRFLOW-1054]:
>>>  https://github.com/apache/incubator-airflow/pull/2201 
>>> <https://github.com/apache/incubator-airflow/pull/2201>
>>> 
>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>>> created if catchup=False :
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 
>>> <https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>> 
>>> I have a PR that works for this, but will need to add unit tests for it as
>>> well as for AIRFLOW-1013.
>>> 
>>> -s
>>> 
>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org 
>>> <mailto:san...@apache.org>> wrote:
>>> 
>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 
>>>> <https://issues.apache.org/jira/browse/AIRFLOW-1013> was a
>>>> blocker. I will have a PR shortly.
>>>> -s
>>>> 
>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org 
>>>> <mailto:criccom...@apache.org>>
>>>> wrote:
>>>> 
>>>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>>>> are listed as part of the 1.8.1 release:
>>>>> 
>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>> 
>>>>> I'm going to merge them in now.
>>>>> 
>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org 
>>>>> <mailto:criccom...@apache.org>>
>>>>> wrote:
>>>>> 
>>>>>> Hey Bolke,
>>>>>> 
>>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>> 
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000 
>>>>>> <https://issues.apache.org/jira/browse/AIRFLOW-1

Re: 1.8.1 release

2017-04-03 Thread Chris Riccomini 
Hey Bolke,

> Furthermore, do you have any timelines for getting to the release

Let's target April 17 to begin a vote for RC.

> Can I help in any way

I need people to review things. The more reviewers, the better.

Also, can you please reach a conclusion on whether the issues you raised
(AIRFLOW-1019 and AIRFLOW-1013) are blockers or not (or just change the
status unilaterally if people aren't responding)?

Cheers,
Chris

On Mon, Apr 3, 2017 at 1:06 PM, Bolke de Bruin <bdbr...@gmail.com> wrote:

> I have a PR out for
>
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212
>
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t
> think it is a blocker: it happens only with @once dags that have a SLA,
> hardly very common. Nevertheless a fix would be nice obviously.
>
> Bolke
>
> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbr...@gmail.com> wrote:
>
> Hey Chris,
>
> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what
> the impact is though.
> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the
> scheduler log files. I did downgrade from blocker to critical.
> AIRFLOW-719 Has a PR, including much needed increased test coverage, that
> I am pretty sure is working, but needs verification (plz @Alex)
>
> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
> around at the moment.
>
> Furthermore, do you have any timelines for getting to the release? Can I
> help in any way? You might want to chase a couple of times ;-).
>
> Bolke
>
> On 30 Mar 2017, at 19:48, siddharth anand <san...@apache.org> wrote:
>
> Chris,
> I've submitted PRs for :
>
>  - PR [AIRFLOW-1013] :
>  https://github.com/apache/incubator-airflow/pull/2203
>  - PR [AIRFLOW-1054]:
>  https://github.com/apache/incubator-airflow/pull/2201
>
> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> created if catchup=False :
> https://issues.apache.org/jira/browse/AIRFLOW-1055
>
> I have a PR that works for this, but will need to add unit tests for it as
> well as for AIRFLOW-1013.
>
> -s
>
> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org>
> wrote:
>
> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
> blocker. I will have a PR shortly.
> -s
>
> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
> The following three JIRAs were not merged into the v1-8-test branch, but
> are listed as part of the 1.8.1 release:
>
> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>
> I'm going to merge them in now.
>
> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
> Hey Bolke,
>
> Great. Assuming your PR is committed, that leaves five blockers:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1000
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1018
> https://issues.apache.org/jira/browse/AIRFLOW-1019
>
> I've also got a list of all open 1.8.1 JIRAs [1].
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>
> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com>
>
> wrote:
>
>
> Hi Chris,
>
> I have a PR out for
>
> * Revert of 719, which makes 982 obsolete and removes 983 from the
> blockers list and just a new feature.
>
> See: https://github.com/apache/incubator-airflow/pull/2195 <
> https://github.com/apache/incubator-airflow/pull/2195>
>
> Cc: @alexvanboxel
>
> Bolke
>
> On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
>
> wrote:
>
>
> Hey all,
>
> I've let this thread sit for a while. Here are a list of the issues
>
> that
>
> were raised:
>
> BLOCKERS:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019
> https://issues.apache.org/jira/browse/AIRFLOW-1017
>
> NICE TO HAVE:
> https://issues.apache.org/jira/browse/AIRFLOW-1015
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org

Re: 1.8.1 release

2017-04-03 Thread Bolke de Bruin 
I have a PR out for 

* AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 
<https://github.com/apache/incubator-airflow/pull/2213>
* AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 
<https://github.com/apache/incubator-airflow/pull/2212>

Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t think 
it is a blocker: it happens only with @once dags that have a SLA, hardly very 
common. Nevertheless a fix would be nice obviously.

Bolke

> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbr...@gmail.com> wrote:
> 
> Hey Chris,
> 
> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the 
> impact is though.
> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the 
> scheduler log files. I did downgrade from blocker to critical.
> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I 
> am pretty sure is working, but needs verification (plz @Alex)
> 
> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around 
> at the moment.
> 
> Furthermore, do you have any timelines for getting to the release? Can I help 
> in any way? You might want to chase a couple of times ;-).
> 
> Bolke
> 
>> On 30 Mar 2017, at 19:48, siddharth anand <san...@apache.org> wrote:
>> 
>> Chris,
>> I've submitted PRs for :
>> 
>>  - PR [AIRFLOW-1013] :
>>  https://github.com/apache/incubator-airflow/pull/2203
>>  - PR [AIRFLOW-1054]:
>>  https://github.com/apache/incubator-airflow/pull/2201
>> 
>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>> created if catchup=False :
>> https://issues.apache.org/jira/browse/AIRFLOW-1055
>> 
>> I have a PR that works for this, but will need to add unit tests for it as
>> well as for AIRFLOW-1013.
>> 
>> -s
>> 
>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org> wrote:
>> 
>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
>>> blocker. I will have a PR shortly.
>>> -s
>>> 
>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org>
>>> wrote:
>>> 
>>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>>> are listed as part of the 1.8.1 release:
>>>> 
>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>> 
>>>> I'm going to merge them in now.
>>>> 
>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
>>>> wrote:
>>>> 
>>>>> Hey Bolke,
>>>>> 
>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>> 
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>> 
>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>> 
>>>>> Cheers,
>>>>> Chris
>>>>> 
>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>> 
>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> Hi Chris,
>>>>>> 
>>>>>> I have a PR out for
>>>>>> 
>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>>> blockers list and just a new feature.
>>>>>> 
>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>> 
>>>>>> Cc: @alexvanboxel
>>>>>> 
>>>>>> Bolke
>>>>>> 
>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
>>>>>> wrote:
>>>>>>> 
>>>>>>&

Re: 1.8.1 release

2017-04-03 Thread Bolke de Bruin 
Hey Chris,

AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the 
impact is though.
AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the 
scheduler log files. I did downgrade from blocker to critical.
AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am 
pretty sure is working, but needs verification (plz @Alex)

I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at 
the moment.

Furthermore, do you have any timelines for getting to the release? Can I help 
in any way? You might want to chase a couple of times ;-).

Bolke

> On 30 Mar 2017, at 19:48, siddharth anand <san...@apache.org> wrote:
> 
> Chris,
> I've submitted PRs for :
> 
>   - PR [AIRFLOW-1013] :
>   https://github.com/apache/incubator-airflow/pull/2203
>   - PR [AIRFLOW-1054]:
>   https://github.com/apache/incubator-airflow/pull/2201
> 
> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> created if catchup=False :
> https://issues.apache.org/jira/browse/AIRFLOW-1055
> 
> I have a PR that works for this, but will need to add unit tests for it as
> well as for AIRFLOW-1013.
> 
> -s
> 
> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org> wrote:
> 
>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
>> blocker. I will have a PR shortly.
>> -s
>> 
>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org>
>> wrote:
>> 
>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>> are listed as part of the 1.8.1 release:
>>> 
>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>> 
>>> I'm going to merge them in now.
>>> 
>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
>>> wrote:
>>> 
>>>> Hey Bolke,
>>>> 
>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>> 
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>> 
>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>> 
>>>> Cheers,
>>>> Chris
>>>> 
>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>> 
>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com>
>>> wrote:
>>>> 
>>>>> Hi Chris,
>>>>> 
>>>>> I have a PR out for
>>>>> 
>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>> blockers list and just a new feature.
>>>>> 
>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>> 
>>>>> Cc: @alexvanboxel
>>>>> 
>>>>> Bolke
>>>>> 
>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
>>>>> wrote:
>>>>>> 
>>>>>> Hey all,
>>>>>> 
>>>>>> I've let this thread sit for a while. Here are a list of the issues
>>> that
>>>>>> were raised:
>>>>>> 
>>>>>> BLOCKERS:
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>> 
>>>>>> NICE TO HAVE:
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>

Re: 1.8.1 release

2017-03-30 Thread siddharth anand 
Chris,
I've submitted PRs for :

   - PR [AIRFLOW-1013] :
   https://github.com/apache/incubator-airflow/pull/2203
   - PR [AIRFLOW-1054]:
   https://github.com/apache/incubator-airflow/pull/2201

And filed a blocker for a new issue. Essentially, @once DAGs cannot be
created if catchup=False :
https://issues.apache.org/jira/browse/AIRFLOW-1055

I have a PR that works for this, but will need to add unit tests for it as
well as for AIRFLOW-1013.

-s

On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <san...@apache.org> wrote:

> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
> blocker. I will have a PR shortly.
> -s
>
> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
>> The following three JIRAs were not merged into the v1-8-test branch, but
>> are listed as part of the 1.8.1 release:
>>
>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>
>> I'm going to merge them in now.
>>
>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
>> wrote:
>>
>> > Hey Bolke,
>> >
>> > Great. Assuming your PR is committed, that leaves five blockers:
>> >
>> > https://issues.apache.org/jira/browse/AIRFLOW-1000
>> > https://issues.apache.org/jira/browse/AIRFLOW-1001
>> > https://issues.apache.org/jira/browse/AIRFLOW-1013
>> > https://issues.apache.org/jira/browse/AIRFLOW-1018
>> > https://issues.apache.org/jira/browse/AIRFLOW-1019
>> >
>> > I've also got a list of all open 1.8.1 JIRAs [1].
>> >
>> > Cheers,
>> > Chris
>> >
>> > [1] https://issues.apache.org/jira/issues/?jql=project%20%
>> > 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>> > 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>> >
>> > On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com>
>> wrote:
>> >
>> >> Hi Chris,
>> >>
>> >> I have a PR out for
>> >>
>> >> * Revert of 719, which makes 982 obsolete and removes 983 from the
>> >> blockers list and just a new feature.
>> >>
>> >> See: https://github.com/apache/incubator-airflow/pull/2195 <
>> >> https://github.com/apache/incubator-airflow/pull/2195>
>> >>
>> >> Cc: @alexvanboxel
>> >>
>> >> Bolke
>> >>
>> >> > On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
>> >> wrote:
>> >> >
>> >> > Hey all,
>> >> >
>> >> > I've let this thread sit for a while. Here are a list of the issues
>> that
>> >> > were raised:
>> >> >
>> >> > BLOCKERS:
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-982
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-983
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1019
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1017
>> >> >
>> >> > NICE TO HAVE:
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1015
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1013
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1004
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1003
>> >> > https://issues.apache.org/jira/browse/AIRFLOW-1001
>> >> >
>> >> > It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>> >> >
>> >> > The rest remain open. I will wait on the release until the remaining
>> >> > blockers are finished. Dan/Daniel, can you comment on status?
>> >> >
>> >> > Ruslan, if you want to work on your nice to haves, and submit
>> patches,
>> >> > that's great, otherwise I don't believe they'll get fixed as part of
>> >> 1.8.1.
>> >> >
>> >> > Cheers,
>> >> > Chris
>> >> >
>> >> > On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>> >> dautkha...@gmail.com>
>> >> > wrote:
>> >> >
>> >> >> Thank you Sid!
>> >> >>
>> >> >>
>> >> >> Best regards,
>> >> >> Ruslan
>> >> >>
>> >> >

Re: 1.8.1 release

2017-03-29 Thread siddharth anand 
Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
blocker. I will have a PR shortly.
-s

On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccom...@apache.org>
wrote:

> The following three JIRAs were not merged into the v1-8-test branch, but
> are listed as part of the 1.8.1 release:
>
> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>
> I'm going to merge them in now.
>
> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
> > Hey Bolke,
> >
> > Great. Assuming your PR is committed, that leaves five blockers:
> >
> > https://issues.apache.org/jira/browse/AIRFLOW-1000
> > https://issues.apache.org/jira/browse/AIRFLOW-1001
> > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > https://issues.apache.org/jira/browse/AIRFLOW-1018
> > https://issues.apache.org/jira/browse/AIRFLOW-1019
> >
> > I've also got a list of all open 1.8.1 JIRAs [1].
> >
> > Cheers,
> > Chris
> >
> > [1] https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> > 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
> >
> > On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com>
> wrote:
> >
> >> Hi Chris,
> >>
> >> I have a PR out for
> >>
> >> * Revert of 719, which makes 982 obsolete and removes 983 from the
> >> blockers list and just a new feature.
> >>
> >> See: https://github.com/apache/incubator-airflow/pull/2195 <
> >> https://github.com/apache/incubator-airflow/pull/2195>
> >>
> >> Cc: @alexvanboxel
> >>
> >> Bolke
> >>
> >> > On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
> >> wrote:
> >> >
> >> > Hey all,
> >> >
> >> > I've let this thread sit for a while. Here are a list of the issues
> that
> >> > were raised:
> >> >
> >> > BLOCKERS:
> >> > https://issues.apache.org/jira/browse/AIRFLOW-982
> >> > https://issues.apache.org/jira/browse/AIRFLOW-983
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1019
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1017
> >> >
> >> > NICE TO HAVE:
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1015
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1013
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1004
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1003
> >> > https://issues.apache.org/jira/browse/AIRFLOW-1001
> >> >
> >> > It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
> >> >
> >> > The rest remain open. I will wait on the release until the remaining
> >> > blockers are finished. Dan/Daniel, can you comment on status?
> >> >
> >> > Ruslan, if you want to work on your nice to haves, and submit patches,
> >> > that's great, otherwise I don't believe they'll get fixed as part of
> >> 1.8.1.
> >> >
> >> > Cheers,
> >> > Chris
> >> >
> >> > On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
> >> dautkha...@gmail.com>
> >> > wrote:
> >> >
> >> >> Thank you Sid!
> >> >>
> >> >>
> >> >> Best regards,
> >> >> Ruslan
> >> >>
> >> >> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <san...@apache.org
> >
> >> >> wrote:
> >> >>
> >> >>> Ruslan,
> >> >>> Thanks for sharing this list. I can pick a few up. I agree we should
> >> aim
> >> >> to
> >> >>> get some of them into 1.8.1.
> >> >>>
> >> >>> -s
> >> >>>
> >> >>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
> >> dautkha...@gmail.com
> >> >>>
> >> >>> wrote:
> >> >>>
> >> >>>> Some of the issues I ran into while testing 1.8rc5 :
> >> >>>>
> >> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> >> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> >> &

Re: 1.8.1 release

2017-03-29 Thread Chris Riccomini 
The following three JIRAs were not merged into the v1-8-test branch, but
are listed as part of the 1.8.1 release:

AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b

I'm going to merge them in now.

On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccom...@apache.org>
wrote:

> Hey Bolke,
>
> Great. Assuming your PR is committed, that leaves five blockers:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1000
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1018
> https://issues.apache.org/jira/browse/AIRFLOW-1019
>
> I've also got a list of all open 1.8.1 JIRAs [1].
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>
> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbr...@gmail.com> wrote:
>
>> Hi Chris,
>>
>> I have a PR out for
>>
>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>> blockers list and just a new feature.
>>
>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>> https://github.com/apache/incubator-airflow/pull/2195>
>>
>> Cc: @alexvanboxel
>>
>> Bolke
>>
>> > On 24 Mar 2017, at 10:21, Chris Riccomini <criccom...@apache.org>
>> wrote:
>> >
>> > Hey all,
>> >
>> > I've let this thread sit for a while. Here are a list of the issues that
>> > were raised:
>> >
>> > BLOCKERS:
>> > https://issues.apache.org/jira/browse/AIRFLOW-982
>> > https://issues.apache.org/jira/browse/AIRFLOW-983
>> > https://issues.apache.org/jira/browse/AIRFLOW-1019
>> > https://issues.apache.org/jira/browse/AIRFLOW-1017
>> >
>> > NICE TO HAVE:
>> > https://issues.apache.org/jira/browse/AIRFLOW-1015
>> > https://issues.apache.org/jira/browse/AIRFLOW-1013
>> > https://issues.apache.org/jira/browse/AIRFLOW-1004
>> > https://issues.apache.org/jira/browse/AIRFLOW-1003
>> > https://issues.apache.org/jira/browse/AIRFLOW-1001
>> >
>> > It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>> >
>> > The rest remain open. I will wait on the release until the remaining
>> > blockers are finished. Dan/Daniel, can you comment on status?
>> >
>> > Ruslan, if you want to work on your nice to haves, and submit patches,
>> > that's great, otherwise I don't believe they'll get fixed as part of
>> 1.8.1.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>> dautkha...@gmail.com>
>> > wrote:
>> >
>> >> Thank you Sid!
>> >>
>> >>
>> >> Best regards,
>> >> Ruslan
>> >>
>> >> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <san...@apache.org>
>> >> wrote:
>> >>
>> >>> Ruslan,
>> >>> Thanks for sharing this list. I can pick a few up. I agree we should
>> aim
>> >> to
>> >>> get some of them into 1.8.1.
>> >>>
>> >>> -s
>> >>>
>> >>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>> dautkha...@gmail.com
>> >>>
>> >>> wrote:
>> >>>
>> >>>> Some of the issues I ran into while testing 1.8rc5 :
>> >>>>
>> >>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>> >>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>> >>>>
>> >>>>
>> >>>> It would be great to have at least some of them fixed in 1.8.1.
>> >>>>
>> >>>> Thank you.
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Ruslan Dautkhanov
>> >>>>
>> >>>> On Tue, Mar 21, 2017

Re: 1.8.1 release

2017-03-25 Thread Bolke de Bruin 
I have set it to blocker.

> On 25 Mar 2017, at 17:56, Vincent Poulain  
> wrote:
> 
> Hello,
> 
> For some people who are running airflow on prod with docker, this one is
> quite important : https://issues.apache.org/jira/browse/AIRFLOW-1018. I
> don't have log anymore :/
> 
> cheers,
> 
> On Fri, Mar 24, 2017 at 6:59 PM, Bolke de Bruin  wrote:
> 
>> Hi Chris
>> 
>> I think some jira are missing from the blocker list, I'll supply them
>> soon. Also some fixes are already in the v1-8-test branch, that are not
>> part of your list yet and some need to be (check jira on fixes for 1.8.1).
>> 
>> 982 and 983 might be fixed by reverting a change that we did as part of
>> 1.8.0 and including the "wait for all tasks" patch, that is already in
>> master. Let me pick this up.
>> 
>> To help you out I already did some work on the jira classifications (e.g.
>> try filtering on blocking issues) which should make it easier to find out
>> what needs to go into 1.8.1.
>> 
>> Bolke
>> 
>> Sent from my iPhone
>> 
>>> On 24 Mar 2017, at 10:21, Chris Riccomini  wrote:
>>> 
>>> Hey all,
>>> 
>>> I've let this thread sit for a while. Here are a list of the issues that
>>> were raised:
>>> 
>>> BLOCKERS:
>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>> 
>>> NICE TO HAVE:
>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>> 
>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>>> 
>>> The rest remain open. I will wait on the release until the remaining
>>> blockers are finished. Dan/Daniel, can you comment on status?
>>> 
>>> Ruslan, if you want to work on your nice to haves, and submit patches,
>>> that's great, otherwise I don't believe they'll get fixed as part of
>> 1.8.1.
>>> 
>>> Cheers,
>>> Chris
>>> 
>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov >> 
>>> wrote:
>>> 
 Thank you Sid!
 
 
 Best regards,
 Ruslan
 
 On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand 
 wrote:
 
> Ruslan,
> Thanks for sharing this list. I can pick a few up. I agree we should
>> aim
 to
> get some of them into 1.8.1.
> 
> -s
> 
> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>> dautkha...@gmail.com
> 
> wrote:
> 
>> Some of the issues I ran into while testing 1.8rc5 :
>> 
>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>> 
>> 
>> It would be great to have at least some of them fixed in 1.8.1.
>> 
>> Thank you.
>> 
>> 
>> 
>> 
>> --
>> Ruslan Dautkhanov
>> 
>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov > invalid
>>> wrote:
>> 
>>> Here is my list for targeted 1.8.1 fixes:
>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general
 the
>>> slow
>>> startup time from this new logic of orphaned/reset task)
>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>> hopefully
>>> have a fix out for soon just finishing up tests)
>>> 
>>> We are also hitting a new issue with subdags with rc5 that we weren't
>>> hitting with rc4 where subdags will occasionally just hang (had to
 roll
>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
>> should
>>> be on the list too.
>>> 
>>> 
>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> criccom...@apache.org>
>>> wrote:
>>> 
 Agreed. I'm looking for a list of checksums/JIRAs that we want in
 the
 bugfix release.
 
 On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
 bdbr...@gmail.com>
 wrote:
 
> 
> 
>> On 21 Mar 2017, at 12:51, Bolke de Bruin 
>> wrote:
>> 
>> My suggestion, as we are using semantic versioning is:
>> 
>> 1) no new features in the 1.8 branch
>> 2) only bug fixes in the 1.8 branch
>> 3) new features to land in 1.9
>> 
>> This allows companies to
> 
> Have a

Re: 1.8.1 release

2017-03-25 Thread Vincent Poulain 
Hello,

For some people who are running airflow on prod with docker, this one is
quite important : https://issues.apache.org/jira/browse/AIRFLOW-1018. I
don't have log anymore :/

cheers,

On Fri, Mar 24, 2017 at 6:59 PM, Bolke de Bruin  wrote:

> Hi Chris
>
> I think some jira are missing from the blocker list, I'll supply them
> soon. Also some fixes are already in the v1-8-test branch, that are not
> part of your list yet and some need to be (check jira on fixes for 1.8.1).
>
> 982 and 983 might be fixed by reverting a change that we did as part of
> 1.8.0 and including the "wait for all tasks" patch, that is already in
> master. Let me pick this up.
>
> To help you out I already did some work on the jira classifications (e.g.
> try filtering on blocking issues) which should make it easier to find out
> what needs to go into 1.8.1.
>
> Bolke
>
> Sent from my iPhone
>
> > On 24 Mar 2017, at 10:21, Chris Riccomini  wrote:
> >
> > Hey all,
> >
> > I've let this thread sit for a while. Here are a list of the issues that
> > were raised:
> >
> > BLOCKERS:
> > https://issues.apache.org/jira/browse/AIRFLOW-982
> > https://issues.apache.org/jira/browse/AIRFLOW-983
> > https://issues.apache.org/jira/browse/AIRFLOW-1019
> > https://issues.apache.org/jira/browse/AIRFLOW-1017
> >
> > NICE TO HAVE:
> > https://issues.apache.org/jira/browse/AIRFLOW-1015
> > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > https://issues.apache.org/jira/browse/AIRFLOW-1004
> > https://issues.apache.org/jira/browse/AIRFLOW-1003
> > https://issues.apache.org/jira/browse/AIRFLOW-1001
> >
> > It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
> >
> > The rest remain open. I will wait on the release until the remaining
> > blockers are finished. Dan/Daniel, can you comment on status?
> >
> > Ruslan, if you want to work on your nice to haves, and submit patches,
> > that's great, otherwise I don't believe they'll get fixed as part of
> 1.8.1.
> >
> > Cheers,
> > Chris
> >
> > On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov  >
> > wrote:
> >
> >> Thank you Sid!
> >>
> >>
> >> Best regards,
> >> Ruslan
> >>
> >> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand 
> >> wrote:
> >>
> >>> Ruslan,
> >>> Thanks for sharing this list. I can pick a few up. I agree we should
> aim
> >> to
> >>> get some of them into 1.8.1.
> >>>
> >>> -s
> >>>
> >>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
> dautkha...@gmail.com
> >>>
> >>> wrote:
> >>>
>  Some of the issues I ran into while testing 1.8rc5 :
> 
>  https://issues.apache.org/jira/browse/AIRFLOW-1015
> > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > https://issues.apache.org/jira/browse/AIRFLOW-1004
> > https://issues.apache.org/jira/browse/AIRFLOW-1003
> > https://issues.apache.org/jira/browse/AIRFLOW-1001
> > https://issues.apache.org/jira/browse/AIRFLOW-1015
> 
> 
>  It would be great to have at least some of them fixed in 1.8.1.
> 
>  Thank you.
> 
> 
> 
> 
>  --
>  Ruslan Dautkhanov
> 
>  On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov   invalid
> > wrote:
> 
> > Here is my list for targeted 1.8.1 fixes:
> > https://issues.apache.org/jira/browse/AIRFLOW-982
> > https://issues.apache.org/jira/browse/AIRFLOW-983
> > https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general
> >> the
> > slow
> > startup time from this new logic of orphaned/reset task)
> > https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>  hopefully
> > have a fix out for soon just finishing up tests)
> >
> > We are also hitting a new issue with subdags with rc5 that we weren't
> > hitting with rc4 where subdags will occasionally just hang (had to
> >> roll
> > back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
>  should
> > be on the list too.
> >
> >
> > On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> >>> criccom...@apache.org>
> > wrote:
> >
> >> Agreed. I'm looking for a list of checksums/JIRAs that we want in
> >> the
> >> bugfix release.
> >>
> >> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
> >> bdbr...@gmail.com>
> >> wrote:
> >>
> >>>
> >>>
>  On 21 Mar 2017, at 12:51, Bolke de Bruin 
>  wrote:
> 
>  My suggestion, as we are using semantic versioning is:
> 
>  1) no new features in the 1.8 branch
>  2) only bug fixes in the 1.8 branch
>  3) new features to land in 1.9
> 
>  This allows companies to
> >>>
> >>> Have a "known" version and can move to the new branch when they
> >>> want
>  to
> >>> get new features. Obviously we only support N-1, so when 1.10
> >> comes
>  out
> >> we
> >>> stop

Re: 1.8.1 release

2017-03-24 Thread Bolke de Bruin 
Hi Chris

I think some jira are missing from the blocker list, I'll supply them soon. 
Also some fixes are already in the v1-8-test branch, that are not part of your 
list yet and some need to be (check jira on fixes for 1.8.1). 

982 and 983 might be fixed by reverting a change that we did as part of 1.8.0 
and including the "wait for all tasks" patch, that is already in master. Let me 
pick this up. 

To help you out I already did some work on the jira classifications (e.g. try 
filtering on blocking issues) which should make it easier to find out what 
needs to go into 1.8.1. 

Bolke

Sent from my iPhone

> On 24 Mar 2017, at 10:21, Chris Riccomini  wrote:
> 
> Hey all,
> 
> I've let this thread sit for a while. Here are a list of the issues that
> were raised:
> 
> BLOCKERS:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019
> https://issues.apache.org/jira/browse/AIRFLOW-1017
> 
> NICE TO HAVE:
> https://issues.apache.org/jira/browse/AIRFLOW-1015
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org/jira/browse/AIRFLOW-1003
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> 
> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
> 
> The rest remain open. I will wait on the release until the remaining
> blockers are finished. Dan/Daniel, can you comment on status?
> 
> Ruslan, if you want to work on your nice to haves, and submit patches,
> that's great, otherwise I don't believe they'll get fixed as part of 1.8.1.
> 
> Cheers,
> Chris
> 
> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov 
> wrote:
> 
>> Thank you Sid!
>> 
>> 
>> Best regards,
>> Ruslan
>> 
>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand 
>> wrote:
>> 
>>> Ruslan,
>>> Thanks for sharing this list. I can pick a few up. I agree we should aim
>> to
>>> get some of them into 1.8.1.
>>> 
>>> -s
>>> 
>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov >> 
>>> wrote:
>>> 
 Some of the issues I ran into while testing 1.8rc5 :
 
 https://issues.apache.org/jira/browse/AIRFLOW-1015
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org/jira/browse/AIRFLOW-1003
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1015
 
 
 It would be great to have at least some of them fixed in 1.8.1.
 
 Thank you.
 
 
 
 
 --
 Ruslan Dautkhanov
 
 On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  wrote:
 
> Here is my list for targeted 1.8.1 fixes:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general
>> the
> slow
> startup time from this new logic of orphaned/reset task)
> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
 hopefully
> have a fix out for soon just finishing up tests)
> 
> We are also hitting a new issue with subdags with rc5 that we weren't
> hitting with rc4 where subdags will occasionally just hang (had to
>> roll
> back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
 should
> be on the list too.
> 
> 
> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>> criccom...@apache.org>
> wrote:
> 
>> Agreed. I'm looking for a list of checksums/JIRAs that we want in
>> the
>> bugfix release.
>> 
>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>> bdbr...@gmail.com>
>> wrote:
>> 
>>> 
>>> 
 On 21 Mar 2017, at 12:51, Bolke de Bruin 
 wrote:
 
 My suggestion, as we are using semantic versioning is:
 
 1) no new features in the 1.8 branch
 2) only bug fixes in the 1.8 branch
 3) new features to land in 1.9
 
 This allows companies to
>>> 
>>> Have a "known" version and can move to the new branch when they
>>> want
 to
>>> get new features. Obviously we only support N-1, so when 1.10
>> comes
 out
>> we
>>> stop supporting 1.8.X.
>>> 
 
 Sent from my iPhone
 
> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>> criccom...@apache.org>
>>> wrote:
> 
> Hey all,
> 
> I suggest that we start a 1.8.1 Airflow release now. The goal
 would
>> be:
> 
> 1) get a second release under our belt
> 2) patch known issues with the 1.8.0 release
> 
> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>>> might
>>

Re: 1.8.1 release

2017-03-24 Thread Chris Riccomini 
Hey all,

I've let this thread sit for a while. Here are a list of the issues that
were raised:

BLOCKERS:
https://issues.apache.org/jira/browse/AIRFLOW-982
https://issues.apache.org/jira/browse/AIRFLOW-983
https://issues.apache.org/jira/browse/AIRFLOW-1019
https://issues.apache.org/jira/browse/AIRFLOW-1017

NICE TO HAVE:
https://issues.apache.org/jira/browse/AIRFLOW-1015
https://issues.apache.org/jira/browse/AIRFLOW-1013
https://issues.apache.org/jira/browse/AIRFLOW-1004
https://issues.apache.org/jira/browse/AIRFLOW-1003
https://issues.apache.org/jira/browse/AIRFLOW-1001

It looks like AIRFLOW-1017 is done, though the JIRA is not closed.

The rest remain open. I will wait on the release until the remaining
blockers are finished. Dan/Daniel, can you comment on status?

Ruslan, if you want to work on your nice to haves, and submit patches,
that's great, otherwise I don't believe they'll get fixed as part of 1.8.1.

Cheers,
Chris

On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov 
wrote:

> Thank you Sid!
>
>
> Best regards,
> Ruslan
>
> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand 
> wrote:
>
> > Ruslan,
> > Thanks for sharing this list. I can pick a few up. I agree we should aim
> to
> > get some of them into 1.8.1.
> >
> > -s
> >
> > On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov  >
> > wrote:
> >
> > > Some of the issues I ran into while testing 1.8rc5 :
> > >
> > > https://issues.apache.org/jira/browse/AIRFLOW-1015
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1004
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1003
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1001
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1015
> > >
> > >
> > > It would be great to have at least some of them fixed in 1.8.1.
> > >
> > > Thank you.
> > >
> > >
> > >
> > >
> > > --
> > > Ruslan Dautkhanov
> > >
> > > On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  > > invalid
> > > > wrote:
> > >
> > > > Here is my list for targeted 1.8.1 fixes:
> > > > https://issues.apache.org/jira/browse/AIRFLOW-982
> > > > https://issues.apache.org/jira/browse/AIRFLOW-983
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general
> the
> > > > slow
> > > > startup time from this new logic of orphaned/reset task)
> > > > https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
> > > hopefully
> > > > have a fix out for soon just finishing up tests)
> > > >
> > > > We are also hitting a new issue with subdags with rc5 that we weren't
> > > > hitting with rc4 where subdags will occasionally just hang (had to
> roll
> > > > back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
> > > should
> > > > be on the list too.
> > > >
> > > >
> > > > On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> > criccom...@apache.org>
> > > > wrote:
> > > >
> > > > > Agreed. I'm looking for a list of checksums/JIRAs that we want in
> the
> > > > > bugfix release.
> > > > >
> > > > > On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
> bdbr...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > >
> > > > > > > On 21 Mar 2017, at 12:51, Bolke de Bruin 
> > > wrote:
> > > > > > >
> > > > > > > My suggestion, as we are using semantic versioning is:
> > > > > > >
> > > > > > > 1) no new features in the 1.8 branch
> > > > > > > 2) only bug fixes in the 1.8 branch
> > > > > > > 3) new features to land in 1.9
> > > > > > >
> > > > > > > This allows companies to
> > > > > >
> > > > > > Have a "known" version and can move to the new branch when they
> > want
> > > to
> > > > > > get new features. Obviously we only support N-1, so when 1.10
> comes
> > > out
> > > > > we
> > > > > > stop supporting 1.8.X.
> > > > > >
> > > > > > >
> > > > > > > Sent from my iPhone
> > > > > > >
> > > > > > >> On 21 Mar 2017, at 11:22, Chris Riccomini <
> > criccom...@apache.org>
> > > > > > wrote:
> > > > > > >>
> > > > > > >> Hey all,
> > > > > > >>
> > > > > > >> I suggest that we start a 1.8.1 Airflow release now. The goal
> > > would
> > > > > be:
> > > > > > >>
> > > > > > >> 1) get a second release under our belt
> > > > > > >> 2) patch known issues with the 1.8.0 release
> > > > > > >>
> > > > > > >> I'm happy to run it, but I saw Maxime mentioning that Airbnb
> > might
> > > > > want
> > > > > > to.
> > > > > > >> @Max et al, can you comment?
> > > > > > >>
> > > > > > >> Also, can folks supply JIRAs for stuff that think needs to be
> in
> > > the
> > > > > > 1.8.1
> > > > > > >> bugfix release?
> > > > > > >>
> > > > > > >> Cheers,
> > > > > > >> Chris
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: 1.8.1 release

2017-03-22 Thread Ruslan Dautkhanov 
Thank you Sid!


Best regards,
Ruslan

On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand  wrote:

> Ruslan,
> Thanks for sharing this list. I can pick a few up. I agree we should aim to
> get some of them into 1.8.1.
>
> -s
>
> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov 
> wrote:
>
> > Some of the issues I ran into while testing 1.8rc5 :
> >
> > https://issues.apache.org/jira/browse/AIRFLOW-1015
> > > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > > https://issues.apache.org/jira/browse/AIRFLOW-1004
> > > https://issues.apache.org/jira/browse/AIRFLOW-1003
> > > https://issues.apache.org/jira/browse/AIRFLOW-1001
> > > https://issues.apache.org/jira/browse/AIRFLOW-1015
> >
> >
> > It would be great to have at least some of them fixed in 1.8.1.
> >
> > Thank you.
> >
> >
> >
> >
> > --
> > Ruslan Dautkhanov
> >
> > On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  > invalid
> > > wrote:
> >
> > > Here is my list for targeted 1.8.1 fixes:
> > > https://issues.apache.org/jira/browse/AIRFLOW-982
> > > https://issues.apache.org/jira/browse/AIRFLOW-983
> > > https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the
> > > slow
> > > startup time from this new logic of orphaned/reset task)
> > > https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
> > hopefully
> > > have a fix out for soon just finishing up tests)
> > >
> > > We are also hitting a new issue with subdags with rc5 that we weren't
> > > hitting with rc4 where subdags will occasionally just hang (had to roll
> > > back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
> > should
> > > be on the list too.
> > >
> > >
> > > On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> criccom...@apache.org>
> > > wrote:
> > >
> > > > Agreed. I'm looking for a list of checksums/JIRAs that we want in the
> > > > bugfix release.
> > > >
> > > > On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
> > > > wrote:
> > > >
> > > > >
> > > > >
> > > > > > On 21 Mar 2017, at 12:51, Bolke de Bruin 
> > wrote:
> > > > > >
> > > > > > My suggestion, as we are using semantic versioning is:
> > > > > >
> > > > > > 1) no new features in the 1.8 branch
> > > > > > 2) only bug fixes in the 1.8 branch
> > > > > > 3) new features to land in 1.9
> > > > > >
> > > > > > This allows companies to
> > > > >
> > > > > Have a "known" version and can move to the new branch when they
> want
> > to
> > > > > get new features. Obviously we only support N-1, so when 1.10 comes
> > out
> > > > we
> > > > > stop supporting 1.8.X.
> > > > >
> > > > > >
> > > > > > Sent from my iPhone
> > > > > >
> > > > > >> On 21 Mar 2017, at 11:22, Chris Riccomini <
> criccom...@apache.org>
> > > > > wrote:
> > > > > >>
> > > > > >> Hey all,
> > > > > >>
> > > > > >> I suggest that we start a 1.8.1 Airflow release now. The goal
> > would
> > > > be:
> > > > > >>
> > > > > >> 1) get a second release under our belt
> > > > > >> 2) patch known issues with the 1.8.0 release
> > > > > >>
> > > > > >> I'm happy to run it, but I saw Maxime mentioning that Airbnb
> might
> > > > want
> > > > > to.
> > > > > >> @Max et al, can you comment?
> > > > > >>
> > > > > >> Also, can folks supply JIRAs for stuff that think needs to be in
> > the
> > > > > 1.8.1
> > > > > >> bugfix release?
> > > > > >>
> > > > > >> Cheers,
> > > > > >> Chris
> > > > >
> > > >
> > >
> >
>

Re: 1.8.1 release

2017-03-22 Thread siddharth anand 
Ruslan,
Thanks for sharing this list. I can pick a few up. I agree we should aim to
get some of them into 1.8.1.

-s

On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov 
wrote:

> Some of the issues I ran into while testing 1.8rc5 :
>
> https://issues.apache.org/jira/browse/AIRFLOW-1015
> > https://issues.apache.org/jira/browse/AIRFLOW-1013
> > https://issues.apache.org/jira/browse/AIRFLOW-1004
> > https://issues.apache.org/jira/browse/AIRFLOW-1003
> > https://issues.apache.org/jira/browse/AIRFLOW-1001
> > https://issues.apache.org/jira/browse/AIRFLOW-1015
>
>
> It would be great to have at least some of them fixed in 1.8.1.
>
> Thank you.
>
>
>
>
> --
> Ruslan Dautkhanov
>
> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  invalid
> > wrote:
>
> > Here is my list for targeted 1.8.1 fixes:
> > https://issues.apache.org/jira/browse/AIRFLOW-982
> > https://issues.apache.org/jira/browse/AIRFLOW-983
> > https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the
> > slow
> > startup time from this new logic of orphaned/reset task)
> > https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
> hopefully
> > have a fix out for soon just finishing up tests)
> >
> > We are also hitting a new issue with subdags with rc5 that we weren't
> > hitting with rc4 where subdags will occasionally just hang (had to roll
> > back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
> should
> > be on the list too.
> >
> >
> > On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini 
> > wrote:
> >
> > > Agreed. I'm looking for a list of checksums/JIRAs that we want in the
> > > bugfix release.
> > >
> > > On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
> > > wrote:
> > >
> > > >
> > > >
> > > > > On 21 Mar 2017, at 12:51, Bolke de Bruin 
> wrote:
> > > > >
> > > > > My suggestion, as we are using semantic versioning is:
> > > > >
> > > > > 1) no new features in the 1.8 branch
> > > > > 2) only bug fixes in the 1.8 branch
> > > > > 3) new features to land in 1.9
> > > > >
> > > > > This allows companies to
> > > >
> > > > Have a "known" version and can move to the new branch when they want
> to
> > > > get new features. Obviously we only support N-1, so when 1.10 comes
> out
> > > we
> > > > stop supporting 1.8.X.
> > > >
> > > > >
> > > > > Sent from my iPhone
> > > > >
> > > > >> On 21 Mar 2017, at 11:22, Chris Riccomini 
> > > > wrote:
> > > > >>
> > > > >> Hey all,
> > > > >>
> > > > >> I suggest that we start a 1.8.1 Airflow release now. The goal
> would
> > > be:
> > > > >>
> > > > >> 1) get a second release under our belt
> > > > >> 2) patch known issues with the 1.8.0 release
> > > > >>
> > > > >> I'm happy to run it, but I saw Maxime mentioning that Airbnb might
> > > want
> > > > to.
> > > > >> @Max et al, can you comment?
> > > > >>
> > > > >> Also, can folks supply JIRAs for stuff that think needs to be in
> the
> > > > 1.8.1
> > > > >> bugfix release?
> > > > >>
> > > > >> Cheers,
> > > > >> Chris
> > > >
> > >
> >
>

Re: 1.8.1 release

2017-03-21 Thread Dan Davydov 
It seemed to only affect some subdags (but I think even when restarted they
were still affected), it seemed like a race condition.

For the second question, we did not yet (but checking this is part of the
ticket).

On Tue, Mar 21, 2017 at 3:01 PM, Bolke de Bruin  wrote:

> @dan
>
> I'm obviously interested in the subdag issue as it is executed by the
> backfill logic. Do you have anything to reproduce it with? Can also talk
> about it tomorrow.
>
> Secondly, did you verify the all success / skipped 'fix' against 'wait for
> all tasks to finish'?
>
> @chris I also suggest using/embracing jira more (as you are doing), as it
> helps with cleaner changelogs, tracking and targeting releases.
>
> Also note that I already included some fixes in v1-8-test.
>
> Bolke
>
> Sent from my iPhone
>
> > On 21 Mar 2017, at 14:29, Ruslan Dautkhanov 
> wrote:
> >
> > Some of the issues I ran into while testing 1.8rc5 :
> >
> > https://issues.apache.org/jira/browse/AIRFLOW-1015
> >> https://issues.apache.org/jira/browse/AIRFLOW-1013
> >> https://issues.apache.org/jira/browse/AIRFLOW-1004
> >> https://issues.apache.org/jira/browse/AIRFLOW-1003
> >> https://issues.apache.org/jira/browse/AIRFLOW-1001
> >> https://issues.apache.org/jira/browse/AIRFLOW-1015
> >
> >
> > It would be great to have at least some of them fixed in 1.8.1.
> >
> > Thank you.
> >
> >
> >
> >
> > --
> > Ruslan Dautkhanov
> >
> > On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  invalid
> >> wrote:
> >
> >> Here is my list for targeted 1.8.1 fixes:
> >> https://issues.apache.org/jira/browse/AIRFLOW-982
> >> https://issues.apache.org/jira/browse/AIRFLOW-983
> >> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the
> >> slow
> >> startup time from this new logic of orphaned/reset task)
> >> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
> hopefully
> >> have a fix out for soon just finishing up tests)
> >>
> >> We are also hitting a new issue with subdags with rc5 that we weren't
> >> hitting with rc4 where subdags will occasionally just hang (had to roll
> >> back from rc5 to rc4), I'll try to spin up a JIRA for it soon which
> should
> >> be on the list too.
> >>
> >>
> >> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini  >
> >> wrote:
> >>
> >>> Agreed. I'm looking for a list of checksums/JIRAs that we want in the
> >>> bugfix release.
> >>>
> >>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
> >>> wrote:
> >>>
> 
> 
> > On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> >
> > My suggestion, as we are using semantic versioning is:
> >
> > 1) no new features in the 1.8 branch
> > 2) only bug fixes in the 1.8 branch
> > 3) new features to land in 1.9
> >
> > This allows companies to
> 
>  Have a "known" version and can move to the new branch when they want
> to
>  get new features. Obviously we only support N-1, so when 1.10 comes
> out
> >>> we
>  stop supporting 1.8.X.
> 
> >
> > Sent from my iPhone
> >
> >> On 21 Mar 2017, at 11:22, Chris Riccomini 
>  wrote:
> >>
> >> Hey all,
> >>
> >> I suggest that we start a 1.8.1 Airflow release now. The goal would
> >>> be:
> >>
> >> 1) get a second release under our belt
> >> 2) patch known issues with the 1.8.0 release
> >>
> >> I'm happy to run it, but I saw Maxime mentioning that Airbnb might
> >>> want
>  to.
> >> @Max et al, can you comment?
> >>
> >> Also, can folks supply JIRAs for stuff that think needs to be in the
>  1.8.1
> >> bugfix release?
> >>
> >> Cheers,
> >> Chris
> 
> >>>
> >>
>

Re: 1.8.1 release

2017-03-21 Thread Bolke de Bruin 
@dan

I'm obviously interested in the subdag issue as it is executed by the backfill 
logic. Do you have anything to reproduce it with? Can also talk about it 
tomorrow. 

Secondly, did you verify the all success / skipped 'fix' against 'wait for all 
tasks to finish'?

@chris I also suggest using/embracing jira more (as you are doing), as it helps 
with cleaner changelogs, tracking and targeting releases. 

Also note that I already included some fixes in v1-8-test. 

Bolke

Sent from my iPhone

> On 21 Mar 2017, at 14:29, Ruslan Dautkhanov  wrote:
> 
> Some of the issues I ran into while testing 1.8rc5 :
> 
> https://issues.apache.org/jira/browse/AIRFLOW-1015
>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> 
> 
> It would be great to have at least some of them fixed in 1.8.1.
> 
> Thank you.
> 
> 
> 
> 
> -- 
> Ruslan Dautkhanov
> 
> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov > wrote:
> 
>> Here is my list for targeted 1.8.1 fixes:
>> https://issues.apache.org/jira/browse/AIRFLOW-982
>> https://issues.apache.org/jira/browse/AIRFLOW-983
>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the
>> slow
>> startup time from this new logic of orphaned/reset task)
>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will hopefully
>> have a fix out for soon just finishing up tests)
>> 
>> We are also hitting a new issue with subdags with rc5 that we weren't
>> hitting with rc4 where subdags will occasionally just hang (had to roll
>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon which should
>> be on the list too.
>> 
>> 
>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini 
>> wrote:
>> 
>>> Agreed. I'm looking for a list of checksums/JIRAs that we want in the
>>> bugfix release.
>>> 
>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
>>> wrote:
>>> 
 
 
> On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> 
> My suggestion, as we are using semantic versioning is:
> 
> 1) no new features in the 1.8 branch
> 2) only bug fixes in the 1.8 branch
> 3) new features to land in 1.9
> 
> This allows companies to
 
 Have a "known" version and can move to the new branch when they want to
 get new features. Obviously we only support N-1, so when 1.10 comes out
>>> we
 stop supporting 1.8.X.
 
> 
> Sent from my iPhone
> 
>> On 21 Mar 2017, at 11:22, Chris Riccomini 
 wrote:
>> 
>> Hey all,
>> 
>> I suggest that we start a 1.8.1 Airflow release now. The goal would
>>> be:
>> 
>> 1) get a second release under our belt
>> 2) patch known issues with the 1.8.0 release
>> 
>> I'm happy to run it, but I saw Maxime mentioning that Airbnb might
>>> want
 to.
>> @Max et al, can you comment?
>> 
>> Also, can folks supply JIRAs for stuff that think needs to be in the
 1.8.1
>> bugfix release?
>> 
>> Cheers,
>> Chris
 
>>> 
>>

Re: 1.8.1 release

2017-03-21 Thread Ruslan Dautkhanov 
Some of the issues I ran into while testing 1.8rc5 :

https://issues.apache.org/jira/browse/AIRFLOW-1015
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org/jira/browse/AIRFLOW-1003
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1015


It would be great to have at least some of them fixed in 1.8.1.

Thank you.




-- 
Ruslan Dautkhanov

On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov  wrote:

> Here is my list for targeted 1.8.1 fixes:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the
> slow
> startup time from this new logic of orphaned/reset task)
> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will hopefully
> have a fix out for soon just finishing up tests)
>
> We are also hitting a new issue with subdags with rc5 that we weren't
> hitting with rc4 where subdags will occasionally just hang (had to roll
> back from rc5 to rc4), I'll try to spin up a JIRA for it soon which should
> be on the list too.
>
>
> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini 
> wrote:
>
> > Agreed. I'm looking for a list of checksums/JIRAs that we want in the
> > bugfix release.
> >
> > On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
> > wrote:
> >
> > >
> > >
> > > > On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> > > >
> > > > My suggestion, as we are using semantic versioning is:
> > > >
> > > > 1) no new features in the 1.8 branch
> > > > 2) only bug fixes in the 1.8 branch
> > > > 3) new features to land in 1.9
> > > >
> > > > This allows companies to
> > >
> > > Have a "known" version and can move to the new branch when they want to
> > > get new features. Obviously we only support N-1, so when 1.10 comes out
> > we
> > > stop supporting 1.8.X.
> > >
> > > >
> > > > Sent from my iPhone
> > > >
> > > >> On 21 Mar 2017, at 11:22, Chris Riccomini 
> > > wrote:
> > > >>
> > > >> Hey all,
> > > >>
> > > >> I suggest that we start a 1.8.1 Airflow release now. The goal would
> > be:
> > > >>
> > > >> 1) get a second release under our belt
> > > >> 2) patch known issues with the 1.8.0 release
> > > >>
> > > >> I'm happy to run it, but I saw Maxime mentioning that Airbnb might
> > want
> > > to.
> > > >> @Max et al, can you comment?
> > > >>
> > > >> Also, can folks supply JIRAs for stuff that think needs to be in the
> > > 1.8.1
> > > >> bugfix release?
> > > >>
> > > >> Cheers,
> > > >> Chris
> > >
> >
>

Re: 1.8.1 release

2017-03-21 Thread Dan Davydov 
Here is my list for targeted 1.8.1 fixes:
https://issues.apache.org/jira/browse/AIRFLOW-982
https://issues.apache.org/jira/browse/AIRFLOW-983
https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in general the slow
startup time from this new logic of orphaned/reset task)
https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will hopefully
have a fix out for soon just finishing up tests)

We are also hitting a new issue with subdags with rc5 that we weren't
hitting with rc4 where subdags will occasionally just hang (had to roll
back from rc5 to rc4), I'll try to spin up a JIRA for it soon which should
be on the list too.


On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini 
wrote:

> Agreed. I'm looking for a list of checksums/JIRAs that we want in the
> bugfix release.
>
> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin 
> wrote:
>
> >
> >
> > > On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> > >
> > > My suggestion, as we are using semantic versioning is:
> > >
> > > 1) no new features in the 1.8 branch
> > > 2) only bug fixes in the 1.8 branch
> > > 3) new features to land in 1.9
> > >
> > > This allows companies to
> >
> > Have a "known" version and can move to the new branch when they want to
> > get new features. Obviously we only support N-1, so when 1.10 comes out
> we
> > stop supporting 1.8.X.
> >
> > >
> > > Sent from my iPhone
> > >
> > >> On 21 Mar 2017, at 11:22, Chris Riccomini 
> > wrote:
> > >>
> > >> Hey all,
> > >>
> > >> I suggest that we start a 1.8.1 Airflow release now. The goal would
> be:
> > >>
> > >> 1) get a second release under our belt
> > >> 2) patch known issues with the 1.8.0 release
> > >>
> > >> I'm happy to run it, but I saw Maxime mentioning that Airbnb might
> want
> > to.
> > >> @Max et al, can you comment?
> > >>
> > >> Also, can folks supply JIRAs for stuff that think needs to be in the
> > 1.8.1
> > >> bugfix release?
> > >>
> > >> Cheers,
> > >> Chris
> >
>

Re: 1.8.1 release

2017-03-21 Thread Chris Riccomini 
Agreed. I'm looking for a list of checksums/JIRAs that we want in the
bugfix release.

On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin  wrote:

>
>
> > On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> >
> > My suggestion, as we are using semantic versioning is:
> >
> > 1) no new features in the 1.8 branch
> > 2) only bug fixes in the 1.8 branch
> > 3) new features to land in 1.9
> >
> > This allows companies to
>
> Have a "known" version and can move to the new branch when they want to
> get new features. Obviously we only support N-1, so when 1.10 comes out we
> stop supporting 1.8.X.
>
> >
> > Sent from my iPhone
> >
> >> On 21 Mar 2017, at 11:22, Chris Riccomini 
> wrote:
> >>
> >> Hey all,
> >>
> >> I suggest that we start a 1.8.1 Airflow release now. The goal would be:
> >>
> >> 1) get a second release under our belt
> >> 2) patch known issues with the 1.8.0 release
> >>
> >> I'm happy to run it, but I saw Maxime mentioning that Airbnb might want
> to.
> >> @Max et al, can you comment?
> >>
> >> Also, can folks supply JIRAs for stuff that think needs to be in the
> 1.8.1
> >> bugfix release?
> >>
> >> Cheers,
> >> Chris
>

Re: 1.8.1 release

2017-03-21 Thread Bolke de Bruin 


> On 21 Mar 2017, at 12:51, Bolke de Bruin  wrote:
> 
> My suggestion, as we are using semantic versioning is:
> 
> 1) no new features in the 1.8 branch
> 2) only bug fixes in the 1.8 branch
> 3) new features to land in 1.9
> 
> This allows companies to 

Have a "known" version and can move to the new branch when they want to get new 
features. Obviously we only support N-1, so when 1.10 comes out we stop 
supporting 1.8.X. 

> 
> Sent from my iPhone
> 
>> On 21 Mar 2017, at 11:22, Chris Riccomini  wrote:
>> 
>> Hey all,
>> 
>> I suggest that we start a 1.8.1 Airflow release now. The goal would be:
>> 
>> 1) get a second release under our belt
>> 2) patch known issues with the 1.8.0 release
>> 
>> I'm happy to run it, but I saw Maxime mentioning that Airbnb might want to.
>> @Max et al, can you comment?
>> 
>> Also, can folks supply JIRAs for stuff that think needs to be in the 1.8.1
>> bugfix release?
>> 
>> Cheers,
>> Chris

Re: 1.8.1 release

2017-03-21 Thread Bolke de Bruin 
My suggestion, as we are using semantic versioning is:

1) no new features in the 1.8 branch
2) only bug fixes in the 1.8 branch
3) new features to land in 1.9

This allows companies to fon

Sent from my iPhone

> On 21 Mar 2017, at 11:22, Chris Riccomini  wrote:
> 
> Hey all,
> 
> I suggest that we start a 1.8.1 Airflow release now. The goal would be:
> 
> 1) get a second release under our belt
> 2) patch known issues with the 1.8.0 release
> 
> I'm happy to run it, but I saw Maxime mentioning that Airbnb might want to.
> @Max et al, can you comment?
> 
> Also, can folks supply JIRAs for stuff that think needs to be in the 1.8.1
> bugfix release?
> 
> Cheers,
> Chris

1.8.1 release

2017-03-21 Thread Chris Riccomini 
Hey all,

I suggest that we start a 1.8.1 Airflow release now. The goal would be:

1) get a second release under our belt
2) patch known issues with the 1.8.0 release

I'm happy to run it, but I saw Maxime mentioning that Airbnb might want to.
@Max et al, can you comment?

Also, can folks supply JIRAs for stuff that think needs to be in the 1.8.1
bugfix release?

Cheers,
Chris