Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Thanks to Kenn, I'm able to move forward on RC2. I will start the RC2 release process in the coming hour. Regards JB On 02/06/2018 11:17 PM, Kenneth Knowles wrote: > I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621 > > On master it seems that HBase tests are failing

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Changes are cherrypicked to the release-2.3.0 branch, or applied directly there. On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov wrote: > Possibly stupid question: are new RCs created from master, or by > cherrypicks of desired changes on top of previous RCs? > (it

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Possibly stupid question: are new RCs created from master, or by cherrypicks of desired changes on top of previous RCs? (it affects whether or not my recently merged change on "master" will be in 2.3.0; I'd like it to be :) ) On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik wrote: >

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

+1 for rollback as well. On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax wrote: > +1 for the rollback. > > On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles wrote: > >> +1 for option 1. It is almost certainly repeated proto parsing. That is >> fixed for ParDo via

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

+1 for the rollback. On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles wrote: > +1 for option 1. It is almost certainly repeated proto parsing. That is > fixed for ParDo via generalized caching but not for other transforms. Flink > also has a proto round trip, but the design

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

+1 for option 1. It is almost certainly repeated proto parsing. That is fixed for ParDo via generalized caching but not for other transforms. Flink also has a proto round trip, but the design might avoid the problem anyhow. We should investigate before releasing, or just preemptively roll it back

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi all, As you might have seen, I found the cause of performance degradation on the direct runner (BEAM-3617). We have basically three options for RC2: 1. We revert the change for 2.3.0. I think it's the fastest and more secure way. I created a PR for that

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi Reuven, it's what I'm suspecting. git bisect should give us more information (still in progress, 3 more steps to complete ;)). I keep you posted. Regards JB On 02/06/2018 08:36 AM, Reuven Lax wrote: > Could this be related to any of the portability changes? > > On Mon, Feb 5, 2018 at 7:51

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Could this be related to any of the portability changes? On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré wrote: > Created: > > https://issues.apache.org/jira/browse/BEAM-3617 > > Regards > JB > > On 02/05/2018 04:42 PM, Kenneth Knowles wrote: > > What is the Jira for

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi, quick update about RC2: BEAM-3617 is the only Jira pending for the release. I'm doing a git bisect to identify the commit who caused the performance degradation. Depending of the result, if it's a easy fix than we will try to do it for RC2, else I will start the RC2 as now. I will keep you

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Created: https://issues.apache.org/jira/browse/BEAM-3617 Regards JB On 02/05/2018 04:42 PM, Kenneth Knowles wrote: > What is the Jira for direct runner perf? > > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré > wrote: > > Thanks ! > >

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi Kenn, my bad, I didn't create one yet (I was busy on the TextIO with flink runner, now identify \o/ ;)). Let me create it right now. Thanks ! Regards JB On 02/05/2018 04:42 PM, Kenneth Knowles wrote: > What is the Jira for direct runner perf? > > On Mon, Feb 5, 2018 at 4:35 AM,

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

What is the Jira for direct runner perf? On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré wrote: > Thanks ! > > I cherry-pick on release-2.3.0 branch. > > I'm on the direct runner perf test in the mean time. > > Thanks again ! > > Regards > JB > > On 02/05/2018 12:06 PM,

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Thanks ! I cherry-pick on release-2.3.0 branch. I'm on the direct runner perf test in the mean time. Thanks again ! Regards JB On 02/05/2018 12:06 PM, Aljoscha Krettek wrote: > I merged fixes for: >  - https://issues.apache.org/jira/browse/BEAM-3186 >  - 

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

I merged fixes for: - https://issues.apache.org/jira/browse/BEAM-3186 - https://issues.apache.org/jira/browse/BEAM-3589 @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi guys, Quick update on the RC2 preparation: * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not using artifacts created by Maven). Anyway, I will take a look today. * BEAM-3186 has a PR. Aljoscha will do the review pretty soon. * I'm also taking a look on the

[CANCEL][VOTE] Release 2.3.0, release candidate #1

Hi guys, Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1. We will cherry-pick fixes on release-2.3.0 branch. I'm updating Jira right now. When the fixes will be cherry-picked, I will submit a RC2 to vote. Thanks ! Regards JB On 01/30/2018 09:04 AM, Jean-Baptiste Onofré

Re: [VOTE] Release 2.3.0, release candidate #1

Fully agree. Good catch. I cancel RC1 to prepare a RC2 including at least this fix. Thanks ! Regards JB On 02/01/2018 04:11 PM, Aljoscha Krettek wrote: > -1 > > I think the issue discovered with unbounded sources on Flink Streaming Runner > is a serious regression. Good news is that there is

Re: [VOTE] Release 2.3.0, release candidate #1

-1 I think the issue discovered with unbounded sources on Flink Streaming Runner is a serious regression. Good news is that there is already a fix for that: https://github.com/apache/beam/pull/4558/files And BEAM-3587 also seems serious enough, IMHO. Btw, BEAM-3186, which seems quite serious,

Re: [VOTE] Release 2.3.0, release candidate #1

So, are you casting -1 vote ? I guess so. Regards JB On 02/01/2018 03:53 PM, Ismaël Mejía wrote: > Hi, > > I started to test the release with Nexmark and found three issues > (from minor to more important): > > 1. Small issues to run Nexmark with the release (BEAM-3531 fixed, > BEAM-3592 in

Re: [VOTE] Release 2.3.0, release candidate #1

Hi, I started to test the release with Nexmark and found three issues (from minor to more important): 1. Small issues to run Nexmark with the release (BEAM-3531 fixed, BEAM-3592 in PR): BEAM-3531 Nexmark failed with NPE with DEFAULT suite BEAM-3592 Spark-runner profile is broken on Nexmark

Re: [VOTE] Release 2.3.0, release candidate #1

Hi all, just a quick reminder about the vote process: 1. Any vote can be changed during the vote period. A -1 vote has to be argued (especially if there's not change to do in the project codebase). 2. For convenience to the release manager, please inform if your vote is binding or non-binding

Re: [VOTE] Release 2.3.0, release candidate #1

@ismael: any vote can be changes from -1 to +1 (or +-0) without additional delay Le 1 févr. 2018 03:15, "Lukasz Cwik" a écrit : > Note that a user reported TextIO being broken on Flink. > Thread is here: https://lists.apache.org/thread.html/ >

Re: [VOTE] Release 2.3.0, release candidate #1

Note that a user reported TextIO being broken on Flink. Thread is here: https://lists.apache.org/thread.html/47b16c94032392782505415e010970fd2a9480891c55c2f7b5de92bd@%3Cuser.beam.apache.org%3E Can someone confirm/refute? On Wed, Jan 31, 2018 at 3:36 PM, Konstantinos Katsiapis <

Re: [VOTE] Release 2.3.0, release candidate #1

This will require a change in the Beam code, because image names are hardcoded in to code (python) and configuration (java). RC1 as it is will not work correctly with Cloud Dataflow. On Wed, Jan 31, 2018 at 2:08 PM, Reuven Lax wrote: > Hopefully we can validate soon. I believe

Re: [VOTE] Release 2.3.0, release candidate #1

Hopefully we can validate soon. I believe some of the delays are because of integrating major changes done over the last week (e.g. Java 8 migration). On Wed, Jan 31, 2018 at 2:04 PM, Ismaël Mejía wrote: > What is the common procedure in cases like this ? Because it doesn't >

Re: [VOTE] Release 2.3.0, release candidate #1

What is the common procedure in cases like this ? Because it doesn't seems that it needs a re-vote, just an extra day or two for validation, any ideas JB ? On Wed, Jan 31, 2018 at 10:41 PM, Alan Myrvold wrote: > Yes, it is a dataflow step. Happy to test this again when they

Re: [VOTE] Release 2.3.0, release candidate #1

Yes, it is a dataflow step. Happy to test this again when they are available. On Wed, Jan 31, 2018 at 1:39 PM, Jean-Baptiste Onofré wrote: > OK, I think I understood ;) > > So it's not "directly" related to Beam itself (it's more a Dataflow step > to perform). > > @Alan, I

Re: [VOTE] Release 2.3.0, release candidate #1

OK, I think I understood ;) So it's not "directly" related to Beam itself (it's more a Dataflow step to perform). @Alan, I think it's better to test first and then cast the vote. This kind of tests are valuable to validate the release and make sense. But vote should represent the state of

Re: [VOTE] Release 2.3.0, release candidate #1

It's just a step that needs to be peformed before the new release works on Dataflow. Alan is saying that we've been unable to validate Dataflow so far, as worker images are not yet built. Hopefully they'll be built soon, and we'll be able to validate. On Wed, Jan 31, 2018 at 1:31 PM,

Re: [VOTE] Release 2.3.0, release candidate #1

Hi Alan does it related to change in the codebase or in a dependency/related project ? I mean: is it something we have to fix/change in Beam ? Just curious as I'm not sure what you mean by "worker images" ;) Thanks ! Regards JB On 31/01/2018 22:18, Alan Myrvold wrote: -1 (for now, hope to

Re: [VOTE] Release 2.3.0, release candidate #1

-1 (for now, hope to change this) Dataflow runner jobs are failing for me with 2.3.0 RC1, for both Java and Python. This is not an issues with the 2.3.0 RC1 SDK, we (google) need to release worker images. I have assigned these bugs to myself, and will be working to help get these workers

Re: [VOTE] Release 2.3.0, release candidate #1

Thanks Kenn, I prepared the list of tasks I did. I will complete where release is out. Regards JB On 01/31/2018 03:07 PM, Kenneth Knowles wrote: > I've cloned the release validation spreadsheet: > >     https://s.apache.org/beam-2.3.0-release-validation > > If you plan to perform a manual

Re: [VOTE] Release 2.3.0, release candidate #1

I've cloned the release validation spreadsheet: https://s.apache.org/beam-2.3.0-release-validation If you plan to perform a manual validation task, please sign up so multiple people don't waste effort. Alan & JB, as far as your pairing up to automate more, anything manual on this sheet

Re: [VOTE] Release 2.3.0, release candidate #1

+1 (binding) Casting my own +1 ;) Regards JB On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 2.3.0, as > follows: > > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide