I mistakenly though that Java PostCommit would run these tests, and I merged based on PostCommit passing. That's how the bug got into master.
On Tue, Apr 23, 2019 at 3:21 PM Kenneth Knowles <k...@apache.org> wrote: > What can we do to make this part of day-to-day workflow instead of finding > out during release validation? Was this just a failing test that was missed? > > Kenn > > On Tue, Apr 23, 2019 at 3:02 PM Andrew Pilloud <apill...@google.com> > wrote: > >> It looks like Java Nexmark tests are on the validation sheet but we've >> missed it the last few releases. Thanks for checking it Etienne! Does the >> current release process require everything to be tested before making the >> release final? >> >> I fully agree with you on point 2. All of these issues were in RC1 and >> could have been fixed for RC2. >> >> Andrew >> >> On Tue, Apr 23, 2019 at 2:58 PM Ahmet Altay <al...@google.com> wrote: >> >>> Thank you Andrew. I will suggest two improvements to the release process: >>> 1. We can include benchmarks in the validation sheet ("Apache Beam >>> Release Acceptance Criteria"). They are used part of the validation process >>> and we can ensure that we check those for each release. >>> 2. For RC validation, we can continue to exhaustively validate each RC >>> even after the first -1 vote. Otherwise we end up with not discovering all >>> issues in a given RC and find them a successive RC, increasing the number >>> of iterations required. >>> >>> >>> On Tue, Apr 23, 2019 at 2:11 PM Andrew Pilloud <apill...@google.com> >>> wrote: >>> >>>> Please consider the vote for RC4 canceled. I'll quickly follow up with >>>> a new RC. >>>> >>>> Thanks for the complete testing everyone! >>>> >>>> Andrew >>>> >>>> On Tue, Apr 23, 2019 at 2:06 PM Reuven Lax <re...@google.com> wrote: >>>> >>>>> -1 >>>>> >>>>> we need to cherry pick pr/8325 and pr/8385 to fix the above issue >>>>> >>>>> On Tue, Apr 23, 2019 at 1:48 PM Andrew Pilloud <apill...@google.com> >>>>> wrote: >>>>> >>>>>> I believe the breakage of Nexmark on Dataflow is >>>>>> https://issues.apache.org/jira/browse/BEAM-7002, which went in >>>>>> before the release was cut. It looks like this might be a release blocker >>>>>> based on the fix: https://github.com/apache/beam/pull/8325. >>>>>> >>>>>> The degraded performance is after the release is cut, so we should be >>>>>> good there. >>>>>> >>>>>> Andrew >>>>>> >>>>>> On Tue, Apr 23, 2019 at 8:44 AM Ismaël Mejía <ieme...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Etienne RC1 vote happened in 04/03 and there have not been any cherry >>>>>>> picks on the spark runner afterwards so if there is a commit that >>>>>>> degraded performance around 04/10 it is not part of the release we >>>>>>> are >>>>>>> voting, so please consider reverting your -1. >>>>>>> >>>>>>> However the issue you are reporting looks important, from a quick >>>>>>> look >>>>>>> I am guessing it could be related to BEAM-5775 that was merged on >>>>>>> 12/04 however the performance regressions started happening since >>>>>>> 09/04 so it could be unrelated. Maybe it could be due to changes in >>>>>>> our infrastructure. Maybe the change in the workers to be tracked, >>>>>>> but >>>>>>> definitely not a release blocker at least for the Spark runner. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Apr 23, 2019 at 5:12 PM Etienne Chauchot < >>>>>>> echauc...@apache.org> wrote: >>>>>>> > >>>>>>> > Hi guys , >>>>>>> > I will vote -1 (binding) on this RC (although degradation is >>>>>>> before RC4 cut date). I took a look at Nexmark graphs for the 3 major >>>>>>> runners : >>>>>>> > - there seem to have functional regressions on Dataflow: >>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5647201107705856 >>>>>>> . 13 queries fail in batch mode starting on 04/17 >>>>>>> > - there is a perf degradation (+200%) in spark runner starting on >>>>>>> 04/10 for all the queries: >>>>>>> https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712 >>>>>>> > >>>>>>> > Sorry Andrew for the added work >>>>>>> > >>>>>>> > Etienne >>>>>>> > >>>>>>> > Le lundi 22 avril 2019 à 12:21 -0700, Andrew Pilloud a écrit : >>>>>>> > >>>>>>> > I signed the wheels files and updated the build process to not >>>>>>> require giving travis apache credentials. (You should probably change >>>>>>> your >>>>>>> password if you haven't already.) >>>>>>> > >>>>>>> > Andrew >>>>>>> > >>>>>>> > On Mon, Apr 22, 2019 at 12:18 PM Ahmet Altay <al...@google.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > +1 (binding) >>>>>>> > >>>>>>> > Verified the python 2 wheel files with quick start examples. >>>>>>> > >>>>>>> > On Mon, Apr 22, 2019 at 11:26 AM Ahmet Altay <al...@google.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > I built the wheel files. They are in the usual place along with >>>>>>> other python artifacts. I will test them a bit and update here. Could >>>>>>> someone else please try the wheel files as well? >>>>>>> > >>>>>>> > Andrew, could you sign and hash the wheel files? >>>>>>> > >>>>>>> > On Mon, Apr 22, 2019 at 10:11 AM Ahmet Altay <al...@google.com> >>>>>>> wrote: >>>>>>> > >>>>>>> > I verified >>>>>>> > - signatures and hashes. >>>>>>> > - python streaming quickstart guide >>>>>>> > >>>>>>> > I would like to verify the wheel files before voting. Please let >>>>>>> us know when they are ready. Also, if you need help with building wheel >>>>>>> files I can help/build. >>>>>>> > >>>>>>> > Ahmet >>>>>>> > >>>>>>> > On Mon, Apr 22, 2019 at 3:33 AM Maximilian Michels <m...@apache.org> >>>>>>> wrote: >>>>>>> > >>>>>>> > +1 (binding) >>>>>>> > >>>>>>> > Found a minor bug while testing, but not a blocker: >>>>>>> > https://jira.apache.org/jira/browse/BEAM-7128 >>>>>>> > >>>>>>> > Thanks, >>>>>>> > Max >>>>>>> > >>>>>>> > On 20.04.19 23:02, Pablo Estrada wrote: >>>>>>> > > +1 >>>>>>> > > Ran SQL postcommit, and Dataflow Portability Java >>>>>>> validatesrunner tests. >>>>>>> > > >>>>>>> > > -P. >>>>>>> > > >>>>>>> > > On Wed, Apr 17, 2019 at 1:38 AM Jean-Baptiste Onofré < >>>>>>> j...@nanthrax.net >>>>>>> > > <mailto:j...@nanthrax.net>> wrote: >>>>>>> > > >>>>>>> > > +1 (binding) >>>>>>> > > >>>>>>> > > Quickly checked with beam-samples. >>>>>>> > > >>>>>>> > > Regards >>>>>>> > > JB >>>>>>> > > >>>>>>> > > On 16/04/2019 00:50, Andrew Pilloud wrote: >>>>>>> > > > Hi everyone, >>>>>>> > > > >>>>>>> > > > Please review and vote on the release candidate #4 for >>>>>>> the version >>>>>>> > > > 2.12.0, as follows: >>>>>>> > > > >>>>>>> > > > [ ] +1, Approve the release >>>>>>> > > > [ ] -1, Do not approve the release (please provide >>>>>>> specific comments) >>>>>>> > > > >>>>>>> > > > The complete staging area is available for your review, >>>>>>> which >>>>>>> > > includes: >>>>>>> > > > * JIRA release notes [1], >>>>>>> > > > * the official Apache source release to be deployed to >>>>>>> > > dist.apache.org <http://dist.apache.org> >>>>>>> > > > <http://dist.apache.org> [2], which is signed with the >>>>>>> key with >>>>>>> > > > fingerprint 9E7CEC0661EFD610B632C610AE8FE17F9F8AE3D4 [3], >>>>>>> > > > * all artifacts to be deployed to the Maven Central >>>>>>> Repository [4], >>>>>>> > > > * source code tag "v2.12.0-RC4" [5], >>>>>>> > > > * website pull request listing the release [6], >>>>>>> publishing the API >>>>>>> > > > reference manual [7], and the blog post [8]. >>>>>>> > > > * Java artifacts were built with Gradle/5.2.1 and >>>>>>> OpenJDK/Oracle JDK >>>>>>> > > > 1.8.0_181. >>>>>>> > > > * Python artifacts are deployed along with the source >>>>>>> release to the >>>>>>> > > > dist.apache.org <http://dist.apache.org> < >>>>>>> http://dist.apache.org> >>>>>>> > > [2]. >>>>>>> > > > * Validation sheet with a tab for 2.12.0 release to help >>>>>>> with >>>>>>> > > validation >>>>>>> > > > [9]. >>>>>>> > > > >>>>>>> > > > The vote will be open for at least 72 hours. It is >>>>>>> adopted by >>>>>>> > > majority >>>>>>> > > > approval, with at least 3 PMC affirmative votes. >>>>>>> > > > >>>>>>> > > > Thanks, >>>>>>> > > > Andrew >>>>>>> > > > >>>>>>> > > > 1] >>>>>>> > > >>>>>>> https://jira.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12344944 >>>>>>> > > > [2] https://dist.apache.org/repos/dist/dev/beam/2.12.0/ >>>>>>> > > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS >>>>>>> > > > [4] >>>>>>> > > >>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1068/ >>>>>>> > > > [5] https://github.com/apache/beam/tree/v2.12.0-RC4 >>>>>>> > > > [6] https://github.com/apache/beam/pull/8215 >>>>>>> > > > [7] https://github.com/apache/beam-site/pull/588 >>>>>>> > > > [8] https://github.com/apache/beam/pull/8314 >>>>>>> > > > [9] >>>>>>> > > >>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1007316984 >>>>>>> > > >>>>>>> > > -- >>>>>>> > > Jean-Baptiste Onofré >>>>>>> > > jbono...@apache.org <mailto:jbono...@apache.org> >>>>>>> > > http://blog.nanthrax.net >>>>>>> > > Talend - http://www.talend.com >>>>>>> > > >>>>>>> >>>>>>