Hi, no problem, I can cut RC2 as soon as the cherry pick is done.
Thanks for catching up ! Please let me know when the cherry pick is done, or you can do the PR and I will do it, up to you. Regards JB On 12/06/2018 04:02, Udi Meiri wrote: > Another bug: reading from PubSub with_attributes=True is broken on > Python with Dataflow. > https://issues.apache.org/jira/browse/BEAM-4536 > > JB, I'm making a PR that removes this keyword and I'd like to propose it > as a cherrypick to 2.5.0. > (feature should be fixed in the next release) > > On Mon, Jun 11, 2018 at 6:19 PM Chamikara Jayalath <chamik...@google.com > <mailto:chamik...@google.com>> wrote: > > FYI: looks like Python tests are failing for Windows. JIRA > is https://issues.apache.org/jira/browse/BEAM-4535. > > I don't think this is a release blocker but this should probably go > in release notes (for any user that tries to run tests on Python > source build). And we should try to incorporate a fix if we happen > to cut another release candidate for some reason. > > Thanks, > Cham > > On Mon, Jun 11, 2018 at 5:46 PM Pablo Estrada <pabl...@google.com > <mailto:pabl...@google.com>> wrote: > > Thanks everyone who has pitched in to validate the release! > > Boyuan Zhang and I have also run a few pipelines, and verified > that they work properly (see release validation spreadsheet[1]). > > We have also found that the Game Stats pipeline is failing in > Python Streaming Dataflow. I have filed BEAM-4534[2]. This is > not a blocker, since Python streaming is not yet fully supported. > > It seems that the uploaded artifacts look good. > > We have noticed that the Python artifacts are still missing > Python wheel files (compare [3] and [4]). JB, could you please > add the wheel files? Boyuan and I can try to help you prepare > them / upload them if necessary. Please let us know. > > Thanks again! > -P. > > [1] > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807 > [2] https://issues.apache.org/jira/browse/BEAM-4534 > [3] https://dist.apache.org/repos/dist/dev/beam/2.4.0/ > [4] https://dist.apache.org/repos/dist/dev/beam/2.5.0/ > > On Mon, Jun 11, 2018 at 12:37 PM Alan Myrvold > <amyrv...@google.com <mailto:amyrv...@google.com>> wrote: > > +1 (non-binding) > > tested some of the quickstarts > > On Sun, Jun 10, 2018 at 1:39 AM Tim > <timrobertson...@gmail.com > <mailto:timrobertson...@gmail.com>> wrote: > > Tested by our team: > - mvn inclusion > - Avro, ES, Hadoop IF IO > - Pipelines run on Spark (Cloudera 5.12.0 YARN cluster) > - Reviewed release notes > > +1 > > Thanks also to everyone who helped get over the gradle > hurdle and in particular to JB. > > Tim > > > On 9 Jun 2018, at 05:56, Jean-Baptiste Onofré > <j...@nanthrax.net <mailto:j...@nanthrax.net>> wrote: > > > > No problem Pablo. > > > > The vote period is a minimum, it can be extended as > requested or if we > > don't have the minimum of 3 binding votes. > > > > Regards > > JB > > > >> On 09/06/2018 01:54, Pablo Estrada wrote: > >> Hello all, > >> I'd like to request an extension of the voting period > until Monday > >> evening (US time, so later in other geographical > regions). This is > >> because we were only now able to publish Dataflow > Workers, and have not > >> had the chance to run release validation tests on > them. The extension > >> will allow us to validate and vote by Monday. > >> Is this acceptable to the community? > >> > >> Best > >> -P. > >> > >> On Fri, Jun 8, 2018 at 6:20 AM Alexey Romanenko > >> <aromanenko....@gmail.com > <mailto:aromanenko....@gmail.com> > <mailto:aromanenko....@gmail.com > <mailto:aromanenko....@gmail.com>>> wrote: > >> > >> Thank you JB for your work! > >> > >> I tested running simple streaming (/KafkaIO/) and > batch (/TextIO / > >> HDFS/) pipelines with SparkRunner on YARN cluster > - it works fine. > >> > >> WBR, > >> Alexey > >> > >> > >>> On 8 Jun 2018, at 10:00, Etienne Chauchot > <echauc...@apache.org <mailto:echauc...@apache.org> > >>> <mailto:echauc...@apache.org > <mailto:echauc...@apache.org>>> wrote: > >>> > >>> I forgot to vote: > >>> +1 (non binding). > >>> What I tested: > >>> - no functional or performance regression > comparing to v2.4 > >>> - dependencies in the poms are ok > >>> > >>> Etienne > >>>> Le vendredi 08 juin 2018 à 08:27 +0200, Romain > Manni-Bucau a écrit : > >>>> +1 (non-binding), mainstream usage is not broken > by the pom > >>>> changes and runtime has no known regression > compared to the 2.4.0 > >>>> > >>>> (side note: kudo to JB for this build tool > change release, I know > >>>> how it can hurt ;)) > >>>> > >>>> Romain Manni-Bucau > >>>> @rmannibucau <https://twitter.com/rmannibucau> > | Blog > >>>> <https://rmannibucau.metawerx.net/> | Old Blog > >>>> <http://rmannibucau.wordpress.com/> | Github > >>>> <https://github.com/rmannibucau> | LinkedIn > >>>> <https://www.linkedin.com/in/rmannibucau> | Book > >>>> > > <https://www.packtpub.com/application-development/java-ee-8-high-performance> > >>>> > >>>> > >>>> Le jeu. 7 juin 2018 à 16:17, Jean-Baptiste Onofré > >>>> <j...@nanthrax.net <mailto:j...@nanthrax.net> > <mailto:j...@nanthrax.net <mailto:j...@nanthrax.net>>> a > écrit : > >>>>> Thanks for the details Etienne ! > >>>>> > >>>>> The good news is that the artifacts seem OK and > the overall Nexmark > >>>>> results are consistent with the 2.4.0 release ones. > >>>>> > >>>>> I'm starting a complete review using the > beam-samples as well. > >>>>> > >>>>> Regards > >>>>> JB > >>>>> > >>>>>> On 07/06/2018 16:14, Etienne Chauchot wrote: > >>>>>> Hi, > >>>>>> I've just run the nexmark queries on v2.5.0-RC1 tag > >>>>>> What we can notice: > >>>>>> - query 3 (exercises CoGroupByKey, state and > timer) shows > >>>>> different > >>>>>> output with DR between batch and streaming and > with the other > >>>>> runners => > >>>>>> I compared with v2.4 there were still these > differences but with > >>>>>> different output size numbers > >>>>>> > >>>>>> - query 6 (exercises specialized combiner) shows > different output > >>>>>> between the runners => the correct output is 401. > strange that > >>>>> in batch > >>>>>> mode some runners output les Sellers. I compared > with v2.4 > >>>>> same output > >>>>>> > >>>>>> - response time of query 7 (exercices Max > transform, fanout > >>>>> and side > >>>>>> input) is very slow on DR => I compared with v2.4 > , comparable > >>>>> execution > >>>>>> times > >>>>>> > >>>>>> I'm not comparing q10 because it is a write to > GCS so it is > >>>>> very specific. > >>>>>> > >>>>>> => Basically no regression comparing to v2.4 > >>>>>> > >>>>>> For the record here is the output (waiting for > ongoing perfkit > >>>>> integration): > >>>>>> > >>>>>> > >>>>>> 1. DR batch > >>>>>> > >>>>>> Performance: > >>>>>> > >>>>>> > >>>>> Conf Runtime(sec) (Baseline) Events(/sec) > (Baseline) Results (Baseline) > >>>>>> > >>>>>> > >>>>> 0000 5,8 17283,1 > 100000 > >>>>>> > >>>>>> > >>>>> 0001 3,2 31104,2 > 92000 > >>>>>> > >>>>>> > >>>>> 0002 1,2 82918,7 > 351 > >>>>>> > >>>>>> > >>>>> 0003 2,2 46210,7 > 458 > >>>>>> > >>>>>> > >>>>> 0004 1,2 8503,4 > 40 > >>>>>> > >>>>>> > >>>>> 0005 4,0 25220,7 > 12 > >>>>>> > >>>>>> > >>>>> 0006 0,9 11148,3 > 401 > >>>>>> > >>>>>> > >>>>> 0007 13,2 7580,9 > 1 > >>>>>> > >>>>>> > >>>>> 0008 1,5 67340,1 > 6000 > >>>>>> > >>>>>> > >>>>> 0009 0,7 14025,2 > 298 > >>>>>> > >>>>>> > >>>>> 0010 12,8 7793,0 > 1 > >>>>>> > >>>>>> > >>>>> 0011 2,4 42319,1 > 1919 > >>>>>> > >>>>>> > >>>>> 0012 1,6 61462,8 > 1919 > >>>>>> > >>>>> > > ========================================================================================== > >>>>>> > >>>>>> 2. DR streaming > >>>>>> > >>>>>> Performance: > >>>>>> > >>>>>> > >>>>> Conf Runtime(sec) (Baseline) Events(/sec) > (Baseline) Results (Baseline) > >>>>>> > >>>>>> > >>>>> 0000 6,5 15285,8 > 100000 > >>>>>> > >>>>>> > >>>>> 0001 3,7 27397,3 > 92000 > >>>>>> > >>>>>> > >>>>> 0002 1,4 69108,5 > 351 > >>>>>> > >>>>>> > >>>>> 0003 3,2 31181,8 > 447 > >>>>>> > >>>>>> > >>>>> 0004 1,2 8361,2 > 40 > >>>>>> > >>>>>> > >>>>> 0005 5,3 18903,6 > 12 > >>>>>> > >>>>>> > >>>>> 0006 0,9 11111,1 > 401 > >>>>>> > >>>>>> > >>>>> 0007 82,5 1212,2 > 1 > >>>>>> > >>>>>> > >>>>> 0008 2,0 51072,5 > 6000 > >>>>>> > >>>>>> > >>>>> 0009 0,8 12903,2 > 298 > >>>>>> > >>>>>> > >>>>> 0010 49,5 2021,8 > 1 > >>>>>> > >>>>>> > >>>>> 0011 3,9 25667,4 > 1919 > >>>>>> > >>>>>> > >>>>> 0012 2,4 41067,8 > 1919 > >>>>>> > >>>>> > > ========================================================================================== > >>>>>> > >>>>>> 3. Flink batch > >>>>>> Performance: > >>>>>> > >>>>>> > >>>>> Conf Runtime(sec) (Baseline) Events(/sec) > (Baseline) Results (Baseline) > >>>>>> > >>>>>> > >>>>> 0000 1,0 97656,3 > 100000 > >>>>>> > >>>>>> > >>>>> 0001 0,7 141643,1 > 92000 > >>>>>> > >>>>>> > >>>>> 0002 0,4 228310,5 > 351 > >>>>>> > >>>>>> > >>>>> 0003 1,6 64020,5 > 580 > >>>>>> > >>>>>> > >>>>> 0004 0,7 13831,3 > 40 > >>>>>> > >>>>>> > >>>>> 0005 1,4 72939,5 > 12 > >>>>>> > >>>>>> > >>>>> 0006 0,5 20491,8 > 103 > >>>>>> > >>>>>> > >>>>> 0007 1,3 74239,0 > 1 > >>>>>> > >>>>>> > >>>>> 0008 0,8 121506,7 > 6000 > >>>>>> > >>>>>> > >>>>> 0009 0,6 17953,3 > 298 > >>>>>> > >>>>>> > >>>>> 0010 1,3 74682,6 > 1 > >>>>>> > >>>>>> > >>>>> 0011 1,1 92936,8 > 1919 > >>>>>> > >>>>>> > >>>>> 0012 0,8 123001,2 > 1919 > >>>>>> > >>>>> > > ========================================================================================== > >>>>>> > >>>>>> 4. Flink streaming > >>>>>> Performance: > >>>>>> > >>>>>> > >>>>> Conf Runtime(sec) (Baseline) Events(/sec) > (Baseline) Results (Baseline) > >>>>>> > >>>>>> > >>>>> 0000 5,4 18677,6 > 100000 > >>>>>> > >>>>>> > >>>>> 0001 2,8 35511,4 > 92000 > >>>>>> > >>>>>> > >>>>> 0002 1,8 54318,3 > 351 > >>>>>> > >>>>>> > >>>>> 0003 2,4 41614,6 > 580 > >>>>>> > >>>>>> > >>>>> 0004 1,0 10341,3 > 40 > >>>>>> > >>>>>> > >>>>> 0005 3,4 29568,3 > 12 > >>>>>> > >>>>>> > >>>>> 0006 0,7 13369,0 > 401 > >>>>>> > >>>>>> > >>>>> 0007 2,8 36192,5 > 1 > >>>>>> > >>>>>> > >>>>> 0008 1,8 54854,6 > 6000 > >>>>>> > >>>>>> > >>>>> 0009 0,7 13369,0 > 298 > >>>>>> > >>>>>> > >>>>> 0010 3,4 29841,8 > 2 > >>>>>> > >>>>>> > >>>>> 0011 5,0 19932,2 > 1919 > >>>>>> > >>>>>> > >>>>> 0012 2,6 38835,0 > 1919 > >>>>>> > >>>>> > > ========================================================================================== > >>>>>> > >>>>>> 5. Spark batch > >>>>>> Performance: > >>>>>> > >>>>>> > >>>>> Conf Runtime(sec) (Baseline) Events(/sec) > (Baseline) Results (Baseline) > >>>>>> > >>>>>> > >>>>> 0000 1,5 65445,0 > 100000 > >>>>>> > >>>>>> > >>>>> 0001 1,3 79491,3 > 92000 > >>>>>> > >>>>>> > >>>>> 0002 0,9 112107,6 > 351 > >>>>>> > >>>>>> > >>>>> 0003 2,0 48804,3 > 580 > >>>>>> > >>>>>> > >>>>> 0004 1,2 8382,2 > 40 > >>>>>> > >>>>>> > >>>>> 0005 2,0 50838,8 > 12 > >>>>>> > >>>>>> > >>>>> 0006 1,0 9699,3 > 103 > >>>>>> > >>>>>> > >>>>> 0007 2,3 43308,8 > 1 > >>>>>> > >>>>>> > >>>>> 0008 2,1 46794,6 > 6000 > >>>>>> > >>>>>> > >>>>> 0009 1,1 8976,7 > 298 > >>>>>> > >>>>>> > >>>>> 0010 1,6 62111,8 > 1 > >>>>>> > >>>>>> > >>>>> 0011 2,1 46598,3 > 1919 > >>>>>> > >>>>>> > >>>>> 0012 2,3 43687,2 > 1919 > >>>>>> > >>>>> > > ========================================================================================== > >>>>>> > >>>>>>> Le mercredi 06 juin 2018 à 10:50 +0200, Etienne > Chauchot a écrit : > >>>>>>> Thanks JB for all your work ! I believe doing > the first > >>>>> gradle release > >>>>>>> must have been hard. > >>>>>>> I'll run Nexmark on the release and keep you posted. > >>>>>>> > >>>>>>> Best > >>>>>>> Etienne > >>>>>>> > >>>>>>> > >>>>>>> Le mercredi 06 juin 2018 à 10:44 +0200, > Jean-Baptiste Onofré > >>>>> a écrit : > >>>>>>>> Hi everyone, > >>>>>>>> > >>>>>>>> Please review and vote on the release candidate > #1 for the > >>>>> version > >>>>>>>> 2.5.0, as follows: > >>>>>>>> > >>>>>>>> [ ] +1, Approve the release > >>>>>>>> [ ] -1, Do not approve the release (please > provide specific > >>>>> comments) > >>>>>>>> > >>>>>>>> NB: this is the first release using Gradle, so > don't be too > >>>>> harsh ;) A > >>>>>>>> PR about the release guide will follow thanks > to this release. > >>>>>>>> > >>>>>>>> The complete staging area is available for your > review, > >>>>> which includes: > >>>>>>>> * JIRA release notes [1], > >>>>>>>> * the official Apache source release to be > deployed to > >>>>> dist.apache.org <http://dist.apache.org> > <http://dist.apache.org/> > >>>>>>>> [2], which is signed with the key with > fingerprint C8282E76 [3], > >>>>>>>> * all artifacts to be deployed to the Maven Central > >>>>> Repository [4], > >>>>>>>> * source code tag "v2.5.0-RC1" [5], > >>>>>>>> * website pull request listing the release and > publishing > >>>>> the API > >>>>>>>> reference manual [6]. > >>>>>>>> * Java artifacts were built with Gradle 4.7 > (wrapper) and > >>>>> OpenJDK/Oracle > >>>>>>>> JDK 1.8.0_172 (Oracle Corporation 25.172-b11). > >>>>>>>> * Python artifacts are deployed along with the > source > >>>>> release to the > >>>>>>>> dist.apache.org <http://dist.apache.org> > <http://dist.apache.org/> [2]. > >>>>>>>> > >>>>>>>> The vote will be open for at least 72 hours. It > is adopted > >>>>> by majority > >>>>>>>> approval, with at least 3 PMC affirmative votes. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> JB > >>>>>>>> > >>>>>>>> [1] > >>>>>>>> > >>>>> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12342847 > >>>>>>>> [2] > https://dist.apache.org/repos/dist/dev/beam/2.5.0/ > >>>>>>>> [3] > https://dist.apache.org/repos/dist/release/beam/KEYS > >>>>>>>> [4] > >>>>> > > https://repository.apache.org/content/repositories/orgapachebeam-1041/ > >>>>>>>> [5] https://github.com/apache/beam/tree/v2.5.0-RC1 > >>>>>>>> [6] https://github.com/apache/beam-site/pull/463 > >>>>>>>> > >>>>> > >> > >> -- > >> Got feedback? go/pabloem-feedback > <https://goto.google.com/pabloem-feedback> > > > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org <mailto:jbono...@apache.org> > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > -- > Got feedback? go/pabloem-feedback > <https://goto.google.com/pabloem-feedback> > -- Jean-Baptiste Onofré jbono...@apache.org http://blog.nanthrax.net Talend - http://www.talend.com
