Re: [VOTE] Release 2.27.0, release candidate #1
+Boyuan Zhang helped me get to the bottom of the sql_taxi issue. The problem is with the WriteStringsToPubSub API, which is deprecated since 2.7.0, but used in the example. Boyuan has [1] out to fix WriteStringsToPubSub and I just sent [2] to replace WriteStringsToPubSub with WriteToPubSub in example code. Issue is tracked in [3]. [1] https://github.com/apache/beam/pull/13614 [2] https://github.com/apache/beam/pull/13615 [3] https://issues.apache.org/jira/browse/BEAM-11524 On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada wrote: > Alright! Thanks everyone for your validations. I'm cancelling this RC, and > I'll perform cherry picks to prepare the next one. > > Please update this thread with any other cherry pick requests! > -P. > > On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía wrote: > >> It might be a good idea to include also: >> >> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in >> SDF Wrapper DoFn >> https://github.com/apache/beam/pull/13592 >> >> So Java development experience is less affected (as with 2.26.0) (There >> is a flag to exclude but defaults matter). >> >> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev >> wrote: >> > >> > We discovered a regression on CombineFn.from_callable() started in >> 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we >> fix it in 2.27.0 as it leads to buggy behavior, so I vote -1. >> > >> > The fix to release branch is in flight: >> https://github.com/apache/beam/pull/13613. >> > >> > >> > >> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette >> wrote: >> >> >> >> -1 (non-binding) >> >> Good news: I validated a dataframe pipeline on Dataflow which looked >> good (with expected performance improvements!) >> >> Bad news: I also tried to run the sql_taxi example pipeline (streaming >> SQL in python) on Dataflow and ran into PubSub IO related issues. The >> example fails in the same way with 2.26.0, but it works in 2.25.0. It's >> possible this is a Dataflow bug and not a Beam one, but I'd like to >> investigate further to make sure. >> >> >> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver >> wrote: >> >>> >> >>> +1 (non-binding) Validated wordcount with Python source + Flink and >> Spark job server jars. Also checked that the ...:sql:udf jar was added and >> includes our cherry-picks. Thanks Pablo :) >> >>> >> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay >> wrote: >> >> +1 (binding). >> >> I validated python quickstarts. Thank you Pablo. >> >> On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre < >> j...@nanthrax.net> wrote: >> > >> > +1 (binding) >> > >> > Regards >> > JB >> > >> > Le 23 déc. 2020 à 06:46, Pablo Estrada a >> écrit : >> > >> > Hi everyone, >> > Please review and vote on the release candidate #1 for the version >> 2.27.0, as follows: >> > [ ] +1, Approve the release >> > [ ] -1, Do not approve the release (please provide specific >> comments) >> > >> > >> > Reviewers are encouraged to test their own use cases with the >> release candidate, and vote +1 >> > if no issues are found. >> > >> > The complete staging area is available for your review, which >> includes: >> > * JIRA release notes [1], >> > * the official Apache source release to be deployed to >> dist.apache.org [2], which is signed with the key with fingerprint >> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3], >> > * all artifacts to be deployed to the Maven Central Repository [4], >> > * source code tag "v2.27.0-RC1" [5], >> > * website pull request listing the release [6], publishing the API >> reference manual [7], and the blog post [8]. >> > * Python artifacts are deployed along with the source release to >> the dist.apache.org [2]. >> > * Validation sheet with a tab for 2.27.0 release to help with >> validation [9]. >> > * Docker images published to Docker Hub [10]. >> > >> > The vote will be open for at least 72 hours, but given the >> holidays, we will likely extend for a few more days. The release will be >> adopted by majority approval, with at least 3 PMC affirmative votes. >> > >> > Thanks, >> > -P. >> > >> > [1] >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380 >> > [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/ >> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS >> > [4] >> https://repository.apache.org/content/repositories/orgapachebeam-1145/ >> > [5] https://github.com/apache/beam/tree/v2.27.0-RC1 >> > [6] https://github.com/apache/beam/pull/13602 >> > [7] https://github.com/apache/beam-site/pull/610 >> > [8] https://github.com/apache/beam/pull/13603 >> > [9] >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106 >> > [10] https://hub.docker.com/search?q=apache%2Fbeam=image >> > >> > >> >
Re: [VOTE] Release 2.27.0, release candidate #1
Alright! Thanks everyone for your validations. I'm cancelling this RC, and I'll perform cherry picks to prepare the next one. Please update this thread with any other cherry pick requests! -P. On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía wrote: > It might be a good idea to include also: > > [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in > SDF Wrapper DoFn > https://github.com/apache/beam/pull/13592 > > So Java development experience is less affected (as with 2.26.0) (There > is a flag to exclude but defaults matter). > > On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev > wrote: > > > > We discovered a regression on CombineFn.from_callable() started in > 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we > fix it in 2.27.0 as it leads to buggy behavior, so I vote -1. > > > > The fix to release branch is in flight: > https://github.com/apache/beam/pull/13613. > > > > > > > > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette > wrote: > >> > >> -1 (non-binding) > >> Good news: I validated a dataframe pipeline on Dataflow which looked > good (with expected performance improvements!) > >> Bad news: I also tried to run the sql_taxi example pipeline (streaming > SQL in python) on Dataflow and ran into PubSub IO related issues. The > example fails in the same way with 2.26.0, but it works in 2.25.0. It's > possible this is a Dataflow bug and not a Beam one, but I'd like to > investigate further to make sure. > >> > >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver > wrote: > >>> > >>> +1 (non-binding) Validated wordcount with Python source + Flink and > Spark job server jars. Also checked that the ...:sql:udf jar was added and > includes our cherry-picks. Thanks Pablo :) > >>> > >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay wrote: > > +1 (binding). > > I validated python quickstarts. Thank you Pablo. > > On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre < > j...@nanthrax.net> wrote: > > > > +1 (binding) > > > > Regards > > JB > > > > Le 23 déc. 2020 à 06:46, Pablo Estrada a écrit > : > > > > Hi everyone, > > Please review and vote on the release candidate #1 for the version > 2.27.0, as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > > > Reviewers are encouraged to test their own use cases with the > release candidate, and vote +1 > > if no issues are found. > > > > The complete staging area is available for your review, which > includes: > > * JIRA release notes [1], > > * the official Apache source release to be deployed to > dist.apache.org [2], which is signed with the key with fingerprint > C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3], > > * all artifacts to be deployed to the Maven Central Repository [4], > > * source code tag "v2.27.0-RC1" [5], > > * website pull request listing the release [6], publishing the API > reference manual [7], and the blog post [8]. > > * Python artifacts are deployed along with the source release to the > dist.apache.org [2]. > > * Validation sheet with a tab for 2.27.0 release to help with > validation [9]. > > * Docker images published to Docker Hub [10]. > > > > The vote will be open for at least 72 hours, but given the holidays, > we will likely extend for a few more days. The release will be adopted by > majority approval, with at least 3 PMC affirmative votes. > > > > Thanks, > > -P. > > > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380 > > [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/ > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS > > [4] > https://repository.apache.org/content/repositories/orgapachebeam-1145/ > > [5] https://github.com/apache/beam/tree/v2.27.0-RC1 > > [6] https://github.com/apache/beam/pull/13602 > > [7] https://github.com/apache/beam-site/pull/610 > > [8] https://github.com/apache/beam/pull/13603 > > [9] > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106 > > [10] https://hub.docker.com/search?q=apache%2Fbeam=image > > > > >
Re: [VOTE] Release 2.27.0, release candidate #1
It might be a good idea to include also: [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in SDF Wrapper DoFn https://github.com/apache/beam/pull/13592 So Java development experience is less affected (as with 2.26.0) (There is a flag to exclude but defaults matter). On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev wrote: > > We discovered a regression on CombineFn.from_callable() started in 2.26.0. > Even though it's not a regression in 2.27.0, I strongly prefer we fix it in > 2.27.0 as it leads to buggy behavior, so I vote -1. > > The fix to release branch is in flight: > https://github.com/apache/beam/pull/13613. > > > > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette wrote: >> >> -1 (non-binding) >> Good news: I validated a dataframe pipeline on Dataflow which looked good >> (with expected performance improvements!) >> Bad news: I also tried to run the sql_taxi example pipeline (streaming SQL >> in python) on Dataflow and ran into PubSub IO related issues. The example >> fails in the same way with 2.26.0, but it works in 2.25.0. It's possible >> this is a Dataflow bug and not a Beam one, but I'd like to investigate >> further to make sure. >> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver wrote: >>> >>> +1 (non-binding) Validated wordcount with Python source + Flink and Spark >>> job server jars. Also checked that the ...:sql:udf jar was added and >>> includes our cherry-picks. Thanks Pablo :) >>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay wrote: +1 (binding). I validated python quickstarts. Thank you Pablo. On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre wrote: > > +1 (binding) > > Regards > JB > > Le 23 déc. 2020 à 06:46, Pablo Estrada a écrit : > > Hi everyone, > Please review and vote on the release candidate #1 for the version > 2.27.0, as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > > Reviewers are encouraged to test their own use cases with the release > candidate, and vote +1 > if no issues are found. > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], which is signed with the key with fingerprint > C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag "v2.27.0-RC1" [5], > * website pull request listing the release [6], publishing the API > reference manual [7], and the blog post [8]. > * Python artifacts are deployed along with the source release to the > dist.apache.org [2]. > * Validation sheet with a tab for 2.27.0 release to help with validation > [9]. > * Docker images published to Docker Hub [10]. > > The vote will be open for at least 72 hours, but given the holidays, we > will likely extend for a few more days. The release will be adopted by > majority approval, with at least 3 PMC affirmative votes. > > Thanks, > -P. > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380 > [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/ > [3] https://dist.apache.org/repos/dist/release/beam/KEYS > [4] https://repository.apache.org/content/repositories/orgapachebeam-1145/ > [5] https://github.com/apache/beam/tree/v2.27.0-RC1 > [6] https://github.com/apache/beam/pull/13602 > [7] https://github.com/apache/beam-site/pull/610 > [8] https://github.com/apache/beam/pull/13603 > [9] > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106 > [10] https://hub.docker.com/search?q=apache%2Fbeam=image > >
