Re: [VOTE] Release 2.27.0, release candidate #1

[Kyle Weaver]

> I am not sure what's the status of this work - should this block the
release until we can get those containers out?

We can release Flink 1.11 and Flink 1.12 containers and I'm *pretty sure*
they would work correctly, but to be totally sure we'd have to upgrade our
portable Flink tests to cover the new containers. Rather than rush new
portable Flink versions into the release, I think it may be better to
document that they aren't officially supported for now (users can always
build the job server containers manually if they need).

The larger problem here is that we need to improve the process of adding
new Flink versions. Specifically, we need a single source (or at least
fewer sources) of truth for supported Flink versions so that we don't have
things getting out of sync again.
https://issues.apache.org/jira/browse/BEAM-9541

On Mon, Dec 28, 2020 at 10:31 AM Pablo Estrada  wrote:

> As an update on the release, I've been made aware of, and merged all
> cherry picks. I was in the process of building rc2, but I'm pausing to
> resolve the issue with these containers.
> Best
> -P.
>
> On Mon, Dec 28, 2020, 1:11 PM Pablo Estrada  wrote:
>
>> Good catch Ismael. Thanks!
>>
>> I've created https://issues.apache.org/jira/browse/INFRA-21238 to
>> request the repositories to be created.
>>
>> I am not sure what's the status of this work - should this block the
>> release until we can get those containers out?
>> cc: @Kyle Weaver 
>> Best
>> -P.
>>
>> On Mon, Dec 28, 2020 at 9:48 AM Ismaël Mejía  wrote:
>>
>>> It seems we are not publishing the latest versions of the Flink Job
>>> Server (Flink 1.11 and 1.12) docker images
>>>
>>> These do not exist:
>>>
>>> https://hub.docker.com/r/apache/beam_flink1.11_job_server
>>> https://hub.docker.com/r/apache/beam_flink1.12_job_server
>>>
>>> but this does and has the good 2.27.0_rc1 tag:
>>> https://hub.docker.com/r/apache/beam_flink1.10_job_server
>>>
>>> I wonder if the issue might be related to the fact that we need to
>>> request the repo to be created or if it is something different.
>>>
>>> On Thu, Dec 24, 2020 at 5:33 PM Brian Hulette 
>>> wrote:
>>> >
>>> > +Boyuan Zhang helped me get to the bottom of the sql_taxi issue. The
>>> problem is with the WriteStringsToPubSub API, which is deprecated since
>>> 2.7.0, but used in the example. Boyuan has [1] out to fix
>>> WriteStringsToPubSub and I just sent [2] to replace WriteStringsToPubSub
>>> with WriteToPubSub in example code. Issue is tracked in [3].
>>> >
>>> > [1] https://github.com/apache/beam/pull/13614
>>> > [2] https://github.com/apache/beam/pull/13615
>>> > [3] https://issues.apache.org/jira/browse/BEAM-11524
>>> >
>>> > On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada 
>>> wrote:
>>> >>
>>> >> Alright! Thanks everyone for your validations. I'm cancelling this
>>> RC, and I'll perform cherry picks to prepare the next one.
>>> >>
>>> >> Please update this thread with any other cherry pick requests!
>>> >> -P.
>>> >>
>>> >> On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía  wrote:
>>> >>>
>>> >>> It might be a good idea to include also:
>>> >>>
>>> >>> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
>>> >>> SDF Wrapper DoFn
>>> >>> https://github.com/apache/beam/pull/13592
>>> >>>
>>> >>> So Java development experience is less affected (as with 2.26.0)
>>> (There
>>> >>> is a flag to exclude but defaults matter).
>>> >>>
>>> >>> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev <
>>> valen...@google.com> wrote:
>>> >>> >
>>> >>> > We discovered a regression on CombineFn.from_callable() started in
>>> 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we
>>> fix it in 2.27.0 as it leads to buggy behavior, so I vote -1.
>>> >>> >
>>> >>> > The fix to release branch is in flight:
>>> https://github.com/apache/beam/pull/13613.
>>> >>> >
>>> >>> >
>>> >>> >
>>> >>> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette 
>>> wrote:
>>> >>> >>
>>> >>> >> -1 (non-binding)
>>> >>> >> Good news: I validated a dataframe pipeline on Dataflow which
>>> looked good (with expected performance improvements!)
>>> >>> >> Bad news: I also tried to run the sql_taxi example pipeline
>>> (streaming SQL in python) on Dataflow and ran into PubSub IO related
>>> issues. The example fails in the same way with 2.26.0, but it works in
>>> 2.25.0. It's possible this is a Dataflow bug and not a Beam one, but I'd
>>> like to investigate further to make sure.
>>> >>> >>
>>> >>> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver 
>>> wrote:
>>> >>> >>>
>>> >>> >>> +1 (non-binding) Validated wordcount with Python source + Flink
>>> and Spark job server jars. Also checked that the ...:sql:udf jar was added
>>> and includes our cherry-picks. Thanks Pablo :)
>>> >>> >>>
>>> >>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay 
>>> wrote:
>>> >>> 
>>> >>>  +1 (binding).
>>> >>> 
>>> >>>  I validated python quickstarts. Thank you Pablo.
>>> >>> 
>>> >>>  On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste 

Re: [VOTE] Release 2.27.0, release candidate #1

[Pablo Estrada]

Good catch Ismael. Thanks!

I've created https://issues.apache.org/jira/browse/INFRA-21238 to request
the repositories to be created.

I am not sure what's the status of this work - should this block the
release until we can get those containers out?
cc: @Kyle Weaver 
Best
-P.

On Mon, Dec 28, 2020 at 9:48 AM Ismaël Mejía  wrote:

> It seems we are not publishing the latest versions of the Flink Job
> Server (Flink 1.11 and 1.12) docker images
>
> These do not exist:
>
> https://hub.docker.com/r/apache/beam_flink1.11_job_server
> https://hub.docker.com/r/apache/beam_flink1.12_job_server
>
> but this does and has the good 2.27.0_rc1 tag:
> https://hub.docker.com/r/apache/beam_flink1.10_job_server
>
> I wonder if the issue might be related to the fact that we need to
> request the repo to be created or if it is something different.
>
> On Thu, Dec 24, 2020 at 5:33 PM Brian Hulette  wrote:
> >
> > +Boyuan Zhang helped me get to the bottom of the sql_taxi issue. The
> problem is with the WriteStringsToPubSub API, which is deprecated since
> 2.7.0, but used in the example. Boyuan has [1] out to fix
> WriteStringsToPubSub and I just sent [2] to replace WriteStringsToPubSub
> with WriteToPubSub in example code. Issue is tracked in [3].
> >
> > [1] https://github.com/apache/beam/pull/13614
> > [2] https://github.com/apache/beam/pull/13615
> > [3] https://issues.apache.org/jira/browse/BEAM-11524
> >
> > On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada 
> wrote:
> >>
> >> Alright! Thanks everyone for your validations. I'm cancelling this RC,
> and I'll perform cherry picks to prepare the next one.
> >>
> >> Please update this thread with any other cherry pick requests!
> >> -P.
> >>
> >> On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía  wrote:
> >>>
> >>> It might be a good idea to include also:
> >>>
> >>> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
> >>> SDF Wrapper DoFn
> >>> https://github.com/apache/beam/pull/13592
> >>>
> >>> So Java development experience is less affected (as with 2.26.0) (There
> >>> is a flag to exclude but defaults matter).
> >>>
> >>> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev <
> valen...@google.com> wrote:
> >>> >
> >>> > We discovered a regression on CombineFn.from_callable() started in
> 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we
> fix it in 2.27.0 as it leads to buggy behavior, so I vote -1.
> >>> >
> >>> > The fix to release branch is in flight:
> https://github.com/apache/beam/pull/13613.
> >>> >
> >>> >
> >>> >
> >>> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette 
> wrote:
> >>> >>
> >>> >> -1 (non-binding)
> >>> >> Good news: I validated a dataframe pipeline on Dataflow which
> looked good (with expected performance improvements!)
> >>> >> Bad news: I also tried to run the sql_taxi example pipeline
> (streaming SQL in python) on Dataflow and ran into PubSub IO related
> issues. The example fails in the same way with 2.26.0, but it works in
> 2.25.0. It's possible this is a Dataflow bug and not a Beam one, but I'd
> like to investigate further to make sure.
> >>> >>
> >>> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver 
> wrote:
> >>> >>>
> >>> >>> +1 (non-binding) Validated wordcount with Python source + Flink
> and Spark job server jars. Also checked that the ...:sql:udf jar was added
> and includes our cherry-picks. Thanks Pablo :)
> >>> >>>
> >>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay 
> wrote:
> >>> 
> >>>  +1 (binding).
> >>> 
> >>>  I validated python quickstarts. Thank you Pablo.
> >>> 
> >>>  On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre <
> j...@nanthrax.net> wrote:
> >>> >
> >>> > +1 (binding)
> >>> >
> >>> > Regards
> >>> > JB
> >>> >
> >>> > Le 23 déc. 2020 à 06:46, Pablo Estrada  a
> écrit :
> >>> >
> >>> > Hi everyone,
> >>> > Please review and vote on the release candidate #1 for the
> version 2.27.0, as follows:
> >>> > [ ] +1, Approve the release
> >>> > [ ] -1, Do not approve the release (please provide specific
> comments)
> >>> >
> >>> >
> >>> > Reviewers are encouraged to test their own use cases with the
> release candidate, and vote +1
> >>> >  if no issues are found.
> >>> >
> >>> > The complete staging area is available for your review, which
> includes:
> >>> > * JIRA release notes [1],
> >>> > * the official Apache source release to be deployed to
> dist.apache.org [2], which is signed with the key with fingerprint
> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
> >>> > * all artifacts to be deployed to the Maven Central Repository
> [4],
> >>> > * source code tag "v2.27.0-RC1" [5],
> >>> > * website pull request listing the release [6], publishing the
> API reference manual [7], and the blog post [8].
> >>> > * Python artifacts are deployed along with the source release to
> the dist.apache.org [2].
> >>> > * Validation sheet with a tab for 2.27.0 release 

Re: [VOTE] Release 2.27.0, release candidate #1

[Ismaël Mejía]

It seems we are not publishing the latest versions of the Flink Job
Server (Flink 1.11 and 1.12) docker images

These do not exist:

https://hub.docker.com/r/apache/beam_flink1.11_job_server
https://hub.docker.com/r/apache/beam_flink1.12_job_server

but this does and has the good 2.27.0_rc1 tag:
https://hub.docker.com/r/apache/beam_flink1.10_job_server

I wonder if the issue might be related to the fact that we need to
request the repo to be created or if it is something different.

On Thu, Dec 24, 2020 at 5:33 PM Brian Hulette  wrote:
>
> +Boyuan Zhang helped me get to the bottom of the sql_taxi issue. The problem 
> is with the WriteStringsToPubSub API, which is deprecated since 2.7.0, but 
> used in the example. Boyuan has [1] out to fix WriteStringsToPubSub and I 
> just sent [2] to replace WriteStringsToPubSub with WriteToPubSub in example 
> code. Issue is tracked in [3].
>
> [1] https://github.com/apache/beam/pull/13614
> [2] https://github.com/apache/beam/pull/13615
> [3] https://issues.apache.org/jira/browse/BEAM-11524
>
> On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada  wrote:
>>
>> Alright! Thanks everyone for your validations. I'm cancelling this RC, and 
>> I'll perform cherry picks to prepare the next one.
>>
>> Please update this thread with any other cherry pick requests!
>> -P.
>>
>> On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía  wrote:
>>>
>>> It might be a good idea to include also:
>>>
>>> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
>>> SDF Wrapper DoFn
>>> https://github.com/apache/beam/pull/13592
>>>
>>> So Java development experience is less affected (as with 2.26.0) (There
>>> is a flag to exclude but defaults matter).
>>>
>>> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev  
>>> wrote:
>>> >
>>> > We discovered a regression on CombineFn.from_callable() started in 
>>> > 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we 
>>> > fix it in 2.27.0 as it leads to buggy behavior, so I vote -1.
>>> >
>>> > The fix to release branch is in flight: 
>>> > https://github.com/apache/beam/pull/13613.
>>> >
>>> >
>>> >
>>> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette  wrote:
>>> >>
>>> >> -1 (non-binding)
>>> >> Good news: I validated a dataframe pipeline on Dataflow which looked 
>>> >> good (with expected performance improvements!)
>>> >> Bad news: I also tried to run the sql_taxi example pipeline (streaming 
>>> >> SQL in python) on Dataflow and ran into PubSub IO related issues. The 
>>> >> example fails in the same way with 2.26.0, but it works in 2.25.0. It's 
>>> >> possible this is a Dataflow bug and not a Beam one, but I'd like to 
>>> >> investigate further to make sure.
>>> >>
>>> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver  wrote:
>>> >>>
>>> >>> +1 (non-binding) Validated wordcount with Python source + Flink and 
>>> >>> Spark job server jars. Also checked that the ...:sql:udf jar was added 
>>> >>> and includes our cherry-picks. Thanks Pablo :)
>>> >>>
>>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:
>>> 
>>>  +1 (binding).
>>> 
>>>  I validated python quickstarts. Thank you Pablo.
>>> 
>>>  On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre 
>>>   wrote:
>>> >
>>> > +1 (binding)
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
>>> >
>>> > Hi everyone,
>>> > Please review and vote on the release candidate #1 for the version 
>>> > 2.27.0, as follows:
>>> > [ ] +1, Approve the release
>>> > [ ] -1, Do not approve the release (please provide specific comments)
>>> >
>>> >
>>> > Reviewers are encouraged to test their own use cases with the release 
>>> > candidate, and vote +1
>>> >  if no issues are found.
>>> >
>>> > The complete staging area is available for your review, which 
>>> > includes:
>>> > * JIRA release notes [1],
>>> > * the official Apache source release to be deployed to 
>>> > dist.apache.org [2], which is signed with the key with fingerprint 
>>> > C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
>>> > * all artifacts to be deployed to the Maven Central Repository [4],
>>> > * source code tag "v2.27.0-RC1" [5],
>>> > * website pull request listing the release [6], publishing the API 
>>> > reference manual [7], and the blog post [8].
>>> > * Python artifacts are deployed along with the source release to the 
>>> > dist.apache.org [2].
>>> > * Validation sheet with a tab for 2.27.0 release to help with 
>>> > validation [9].
>>> > * Docker images published to Docker Hub [10].
>>> >
>>> > The vote will be open for at least 72 hours, but given the holidays, 
>>> > we will likely extend for a few more days. The release will be 
>>> > adopted by majority approval, with at least 3 PMC affirmative votes.
>>> >
>>> > Thanks,
>>> > -P.
>>> >
>>> > [1] 
>>> > 

Re: [VOTE] Release 2.27.0, release candidate #1

[Brian Hulette]

+Boyuan Zhang  helped me get to the bottom of the
sql_taxi issue. The problem is with the WriteStringsToPubSub API, which is
deprecated since 2.7.0, but used in the example. Boyuan has [1] out to fix
WriteStringsToPubSub and I just sent [2] to replace WriteStringsToPubSub
with WriteToPubSub in example code. Issue is tracked in [3].

[1] https://github.com/apache/beam/pull/13614
[2] https://github.com/apache/beam/pull/13615
[3] https://issues.apache.org/jira/browse/BEAM-11524

On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada  wrote:

> Alright! Thanks everyone for your validations. I'm cancelling this RC, and
> I'll perform cherry picks to prepare the next one.
>
> Please update this thread with any other cherry pick requests!
> -P.
>
> On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía  wrote:
>
>> It might be a good idea to include also:
>>
>> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
>> SDF Wrapper DoFn
>> https://github.com/apache/beam/pull/13592
>>
>> So Java development experience is less affected (as with 2.26.0) (There
>> is a flag to exclude but defaults matter).
>>
>> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev 
>> wrote:
>> >
>> > We discovered a regression on CombineFn.from_callable() started in
>> 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we
>> fix it in 2.27.0 as it leads to buggy behavior, so I vote -1.
>> >
>> > The fix to release branch is in flight:
>> https://github.com/apache/beam/pull/13613.
>> >
>> >
>> >
>> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette 
>> wrote:
>> >>
>> >> -1 (non-binding)
>> >> Good news: I validated a dataframe pipeline on Dataflow which looked
>> good (with expected performance improvements!)
>> >> Bad news: I also tried to run the sql_taxi example pipeline (streaming
>> SQL in python) on Dataflow and ran into PubSub IO related issues. The
>> example fails in the same way with 2.26.0, but it works in 2.25.0. It's
>> possible this is a Dataflow bug and not a Beam one, but I'd like to
>> investigate further to make sure.
>> >>
>> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver 
>> wrote:
>> >>>
>> >>> +1 (non-binding) Validated wordcount with Python source + Flink and
>> Spark job server jars. Also checked that the ...:sql:udf jar was added and
>> includes our cherry-picks. Thanks Pablo :)
>> >>>
>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay 
>> wrote:
>> 
>>  +1 (binding).
>> 
>>  I validated python quickstarts. Thank you Pablo.
>> 
>>  On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre <
>> j...@nanthrax.net> wrote:
>> >
>> > +1 (binding)
>> >
>> > Regards
>> > JB
>> >
>> > Le 23 déc. 2020 à 06:46, Pablo Estrada  a
>> écrit :
>> >
>> > Hi everyone,
>> > Please review and vote on the release candidate #1 for the version
>> 2.27.0, as follows:
>> > [ ] +1, Approve the release
>> > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> >
>> >
>> > Reviewers are encouraged to test their own use cases with the
>> release candidate, and vote +1
>> >  if no issues are found.
>> >
>> > The complete staging area is available for your review, which
>> includes:
>> > * JIRA release notes [1],
>> > * the official Apache source release to be deployed to
>> dist.apache.org [2], which is signed with the key with fingerprint
>> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
>> > * all artifacts to be deployed to the Maven Central Repository [4],
>> > * source code tag "v2.27.0-RC1" [5],
>> > * website pull request listing the release [6], publishing the API
>> reference manual [7], and the blog post [8].
>> > * Python artifacts are deployed along with the source release to
>> the dist.apache.org [2].
>> > * Validation sheet with a tab for 2.27.0 release to help with
>> validation [9].
>> > * Docker images published to Docker Hub [10].
>> >
>> > The vote will be open for at least 72 hours, but given the
>> holidays, we will likely extend for a few more days. The release will be
>> adopted by majority approval, with at least 3 PMC affirmative votes.
>> >
>> > Thanks,
>> > -P.
>> >
>> > [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
>> > [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
>> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> > [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1145/
>> > [5] https://github.com/apache/beam/tree/v2.27.0-RC1
>> > [6] https://github.com/apache/beam/pull/13602
>> > [7] https://github.com/apache/beam-site/pull/610
>> > [8] https://github.com/apache/beam/pull/13603
>> > [9]
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
>> > [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>> >
>> >
>>
>

Re: [VOTE] Release 2.27.0, release candidate #1

[Pablo Estrada]

Alright! Thanks everyone for your validations. I'm cancelling this RC, and
I'll perform cherry picks to prepare the next one.

Please update this thread with any other cherry pick requests!
-P.

On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía  wrote:

> It might be a good idea to include also:
>
> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
> SDF Wrapper DoFn
> https://github.com/apache/beam/pull/13592
>
> So Java development experience is less affected (as with 2.26.0) (There
> is a flag to exclude but defaults matter).
>
> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev 
> wrote:
> >
> > We discovered a regression on CombineFn.from_callable() started in
> 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer we
> fix it in 2.27.0 as it leads to buggy behavior, so I vote -1.
> >
> > The fix to release branch is in flight:
> https://github.com/apache/beam/pull/13613.
> >
> >
> >
> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette 
> wrote:
> >>
> >> -1 (non-binding)
> >> Good news: I validated a dataframe pipeline on Dataflow which looked
> good (with expected performance improvements!)
> >> Bad news: I also tried to run the sql_taxi example pipeline (streaming
> SQL in python) on Dataflow and ran into PubSub IO related issues. The
> example fails in the same way with 2.26.0, but it works in 2.25.0. It's
> possible this is a Dataflow bug and not a Beam one, but I'd like to
> investigate further to make sure.
> >>
> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver 
> wrote:
> >>>
> >>> +1 (non-binding) Validated wordcount with Python source + Flink and
> Spark job server jars. Also checked that the ...:sql:udf jar was added and
> includes our cherry-picks. Thanks Pablo :)
> >>>
> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:
> 
>  +1 (binding).
> 
>  I validated python quickstarts. Thank you Pablo.
> 
>  On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre <
> j...@nanthrax.net> wrote:
> >
> > +1 (binding)
> >
> > Regards
> > JB
> >
> > Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit
> :
> >
> > Hi everyone,
> > Please review and vote on the release candidate #1 for the version
> 2.27.0, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > Reviewers are encouraged to test their own use cases with the
> release candidate, and vote +1
> >  if no issues are found.
> >
> > The complete staging area is available for your review, which
> includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to
> dist.apache.org [2], which is signed with the key with fingerprint
> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.27.0-RC1" [5],
> > * website pull request listing the release [6], publishing the API
> reference manual [7], and the blog post [8].
> > * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
> > * Validation sheet with a tab for 2.27.0 release to help with
> validation [9].
> > * Docker images published to Docker Hub [10].
> >
> > The vote will be open for at least 72 hours, but given the holidays,
> we will likely extend for a few more days. The release will be adopted by
> majority approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > -P.
> >
> > [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1145/
> > [5] https://github.com/apache/beam/tree/v2.27.0-RC1
> > [6] https://github.com/apache/beam/pull/13602
> > [7] https://github.com/apache/beam-site/pull/610
> > [8] https://github.com/apache/beam/pull/13603
> > [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
> > [10] https://hub.docker.com/search?q=apache%2Fbeam=image
> >
> >
>

Re: [VOTE] Release 2.27.0, release candidate #1

[Ismaël Mejía]

It might be a good idea to include also:

[BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in
SDF Wrapper DoFn
https://github.com/apache/beam/pull/13592

So Java development experience is less affected (as with 2.26.0) (There
is a flag to exclude but defaults matter).

On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev  wrote:
>
> We discovered a regression on CombineFn.from_callable() started in 2.26.0. 
> Even though it's not a regression in 2.27.0, I strongly prefer we fix it in 
> 2.27.0 as it leads to buggy behavior, so I vote -1.
>
> The fix to release branch is in flight: 
> https://github.com/apache/beam/pull/13613.
>
>
>
> On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette  wrote:
>>
>> -1 (non-binding)
>> Good news: I validated a dataframe pipeline on Dataflow which looked good 
>> (with expected performance improvements!)
>> Bad news: I also tried to run the sql_taxi example pipeline (streaming SQL 
>> in python) on Dataflow and ran into PubSub IO related issues. The example 
>> fails in the same way with 2.26.0, but it works in 2.25.0. It's possible 
>> this is a Dataflow bug and not a Beam one, but I'd like to investigate 
>> further to make sure.
>>
>> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver  wrote:
>>>
>>> +1 (non-binding) Validated wordcount with Python source + Flink and Spark 
>>> job server jars. Also checked that the ...:sql:udf jar was added and 
>>> includes our cherry-picks. Thanks Pablo :)
>>>
>>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:

 +1 (binding).

 I validated python quickstarts. Thank you Pablo.

 On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre  
 wrote:
>
> +1 (binding)
>
> Regards
> JB
>
> Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
>
> Hi everyone,
> Please review and vote on the release candidate #1 for the version 
> 2.27.0, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> Reviewers are encouraged to test their own use cases with the release 
> candidate, and vote +1
>  if no issues are found.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org 
> [2], which is signed with the key with fingerprint 
> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.27.0-RC1" [5],
> * website pull request listing the release [6], publishing the API 
> reference manual [7], and the blog post [8].
> * Python artifacts are deployed along with the source release to the 
> dist.apache.org [2].
> * Validation sheet with a tab for 2.27.0 release to help with validation 
> [9].
> * Docker images published to Docker Hub [10].
>
> The vote will be open for at least 72 hours, but given the holidays, we 
> will likely extend for a few more days. The release will be adopted by 
> majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> -P.
>
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1145/
> [5] https://github.com/apache/beam/tree/v2.27.0-RC1
> [6] https://github.com/apache/beam/pull/13602
> [7] https://github.com/apache/beam-site/pull/610
> [8] https://github.com/apache/beam/pull/13603
> [9] 
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
> [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>
>

Re: [VOTE] Release 2.27.0, release candidate #1

[Valentyn Tymofieiev]

We discovered a regression on CombineFn.from_callable() started in 2.26.0.
Even though it's not a regression in 2.27.0, I strongly prefer we fix it in
2.27.0 as it leads to buggy behavior, so I vote -1.

The fix to release branch is in flight:
https://github.com/apache/beam/pull/13613.



On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette  wrote:

> -1 (non-binding)
> Good news: I validated a dataframe pipeline on Dataflow which looked good
> (with expected performance improvements!)
> Bad news: I also tried to run the sql_taxi example pipeline (streaming SQL
> in python) on Dataflow and ran into PubSub IO related issues. The example
> fails in the same way with 2.26.0, but it works in 2.25.0. It's possible
> this is a Dataflow bug and not a Beam one, but I'd like to investigate
> further to make sure.
>
> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver  wrote:
>
>> +1 (non-binding) Validated wordcount with Python source + Flink and Spark
>> job server jars. Also checked that the ...:sql:udf jar was added and
>> includes our cherry-picks. Thanks Pablo :)
>>
>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:
>>
>>> +1 (binding).
>>>
>>> I validated python quickstarts. Thank you Pablo.
>>>
>>> On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre 
>>> wrote:
>>>
 +1 (binding)

 Regards
 JB

 Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :

 Hi everyone,
 Please review and vote on the release candidate #1 for the version 2.27
 .0, as follows:
 [ ] +1, Approve the release
 [ ] -1, Do not approve the release (please provide specific comments)


 Reviewers are encouraged to test their own use cases with the release
 candidate, and vote +1
  if no issues are found.

 The complete staging area is available for your review, which includes:
 * JIRA release notes [1],
 * the official Apache source release to be deployed to dist.apache.org [2],
 which is signed with the key with fingerprint
 C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
 * all artifacts to be deployed to the Maven Central Repository [4],
 * source code tag "v2.27.0-RC1" [5],
 * website pull request listing the release [6], publishing the API
 reference manual [7], and the blog post [8].
 * Python artifacts are deployed along with the source release to the
 dist.apache.org [2].
 * Validation sheet with a tab for 2.27.0 release to help with
 validation [9].
 * Docker images published to Docker Hub [10].

 The vote will be open for at least 72 hours, but given the holidays,
 we will likely extend for a few more days. The release will be adopted by
 majority approval, with at least 3 PMC affirmative votes.

 Thanks,
 -P.

 [1]
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380

 [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
 [3] https://dist.apache.org/repos/dist/release/beam/KEYS
 [4]
 https://repository.apache.org/content/repositories/orgapachebeam-1145/
 [5] https://github.com/apache/beam/tree/v2.27.0-RC1
 [6] https://github.com/apache/beam/pull/13602
 [7] https://github.com/apache/beam-site/pull/610
 [8] https://github.com/apache/beam/pull/13603
 [9]
 https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106

 [10] https://hub.docker.com/search?q=apache%2Fbeam=image

Re: [VOTE] Release 2.27.0, release candidate #1

[Brian Hulette]

-1 (non-binding)
Good news: I validated a dataframe pipeline on Dataflow which looked good
(with expected performance improvements!)
Bad news: I also tried to run the sql_taxi example pipeline (streaming SQL
in python) on Dataflow and ran into PubSub IO related issues. The example
fails in the same way with 2.26.0, but it works in 2.25.0. It's possible
this is a Dataflow bug and not a Beam one, but I'd like to investigate
further to make sure.

On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver  wrote:

> +1 (non-binding) Validated wordcount with Python source + Flink and Spark
> job server jars. Also checked that the ...:sql:udf jar was added and
> includes our cherry-picks. Thanks Pablo :)
>
> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:
>
>> +1 (binding).
>>
>> I validated python quickstarts. Thank you Pablo.
>>
>> On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre 
>> wrote:
>>
>>> +1 (binding)
>>>
>>> Regards
>>> JB
>>>
>>> Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
>>>
>>> Hi everyone,
>>> Please review and vote on the release candidate #1 for the version 2.27.
>>> 0, as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> Reviewers are encouraged to test their own use cases with the release
>>> candidate, and vote +1
>>>  if no issues are found.
>>>
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>> which is signed with the key with fingerprint
>>> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.27.0-RC1" [5],
>>> * website pull request listing the release [6], publishing the API
>>> reference manual [7], and the blog post [8].
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>> * Validation sheet with a tab for 2.27.0 release to help with
>>> validation [9].
>>> * Docker images published to Docker Hub [10].
>>>
>>> The vote will be open for at least 72 hours, but given the holidays, we
>>> will likely extend for a few more days. The release will be adopted by
>>> majority approval, with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> -P.
>>>
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
>>>
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4]
>>> https://repository.apache.org/content/repositories/orgapachebeam-1145/
>>> [5] https://github.com/apache/beam/tree/v2.27.0-RC1
>>> [6] https://github.com/apache/beam/pull/13602
>>> [7] https://github.com/apache/beam-site/pull/610
>>> [8] https://github.com/apache/beam/pull/13603
>>> [9]
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
>>>
>>> [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>>>
>>>
>>>

Re: [VOTE] Release 2.27.0, release candidate #1

[Kyle Weaver]

+1 (non-binding) Validated wordcount with Python source + Flink and Spark
job server jars. Also checked that the ...:sql:udf jar was added and
includes our cherry-picks. Thanks Pablo :)

On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay  wrote:

> +1 (binding).
>
> I validated python quickstarts. Thank you Pablo.
>
> On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre 
> wrote:
>
>> +1 (binding)
>>
>> Regards
>> JB
>>
>> Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
>>
>> Hi everyone,
>> Please review and vote on the release candidate #1 for the version 2.27.0,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> Reviewers are encouraged to test their own use cases with the release
>> candidate, and vote +1
>>  if no issues are found.
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint
>> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.27.0-RC1" [5],
>> * website pull request listing the release [6], publishing the API
>> reference manual [7], and the blog post [8].
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>> * Validation sheet with a tab for 2.27.0 release to help with validation
>> [9].
>> * Docker images published to Docker Hub [10].
>>
>> The vote will be open for at least 72 hours, but given the holidays, we
>> will likely extend for a few more days. The release will be adopted by
>> majority approval, with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> -P.
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
>>
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1145/
>> [5] https://github.com/apache/beam/tree/v2.27.0-RC1
>> [6] https://github.com/apache/beam/pull/13602
>> [7] https://github.com/apache/beam-site/pull/610
>> [8] https://github.com/apache/beam/pull/13603
>> [9]
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
>>
>> [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>>
>>
>>

Re: [VOTE] Release 2.27.0, release candidate #1

[Ahmet Altay]

+1 (binding).

I validated python quickstarts. Thank you Pablo.

On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre 
wrote:

> +1 (binding)
>
> Regards
> JB
>
> Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
>
> Hi everyone,
> Please review and vote on the release candidate #1 for the version 2.27.0,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> Reviewers are encouraged to test their own use cases with the release
> candidate, and vote +1
>  if no issues are found.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint
> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.27.0-RC1" [5],
> * website pull request listing the release [6], publishing the API
> reference manual [7], and the blog post [8].
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
> * Validation sheet with a tab for 2.27.0 release to help with validation
> [9].
> * Docker images published to Docker Hub [10].
>
> The vote will be open for at least 72 hours, but given the holidays, we
> will likely extend for a few more days. The release will be adopted by
> majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> -P.
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
>
> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1145/
> [5] https://github.com/apache/beam/tree/v2.27.0-RC1
> [6] https://github.com/apache/beam/pull/13602
> [7] https://github.com/apache/beam-site/pull/610
> [8] https://github.com/apache/beam/pull/13603
> [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
>
> [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>
>
>

Re: [VOTE] Release 2.27.0, release candidate #1

[Jean-Baptiste Onofre]

+1 (binding)

Regards
JB

> Le 23 déc. 2020 à 06:46, Pablo Estrada  a écrit :
> 
> Hi everyone,
> Please review and vote on the release candidate #1 for the version 2.27.0, as 
> follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
> Reviewers are encouraged to test their own use cases with the release 
> candidate, and vote +1
>  if no issues are found.
> 
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org 
>  [2], which is signed with the key with fingerprint 
> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.27.0-RC1" [5],
> * website pull request listing the release [6], publishing the API reference 
> manual [7], and the blog post [8].
> * Python artifacts are deployed along with the source release to the 
> dist.apache.org  [2].
> * Validation sheet with a tab for 2.27.0 release to help with validation [9].
> * Docker images published to Docker Hub [10].
> 
> The vote will be open for at least 72 hours, but given the holidays, we will 
> likely extend for a few more days. The release will be adopted by majority 
> approval, with at least 3 PMC affirmative votes.
> 
> Thanks,
> -P.
> 
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380
>  
> 
>  
> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/ 
> 
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS 
> 
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1145/ 
> 
> [5] https://github.com/apache/beam/tree/v2.27 
> .0-RC1
> [6] https://github.com/apache/beam/pull/13602 
>  
> [7] https://github.com/apache/beam-site/pull/610 
>  
> [8] https://github.com/apache/beam/pull/13603 
>  
> [9] 
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106
>  
> 
>  
> [10] https://hub.docker.com/search?q=apache%2Fbeam=image 
> 

[VOTE] Release 2.27.0, release candidate #1

[Pablo Estrada]

Hi everyone,
Please review and vote on the release candidate #1 for the version 2.27.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)


Reviewers are encouraged to test their own use cases with the release
candidate, and vote +1
 if no issues are found.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint
C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.27.0-RC1" [5],
* website pull request listing the release [6], publishing the API
reference manual [7], and the blog post [8].
* Python artifacts are deployed along with the source release to the
dist.apache.org [2].
* Validation sheet with a tab for 2.27.0 release to help with validation
[9].
* Docker images published to Docker Hub [10].

The vote will be open for at least 72 hours, but given the holidays, we
will likely extend for a few more days. The release will be adopted by
majority approval, with at least 3 PMC affirmative votes.

Thanks,
-P.

[1]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12349380

[2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1145/
[5] https://github.com/apache/beam/tree/v2.27.0-RC1
[6] https://github.com/apache/beam/pull/13602
[7] https://github.com/apache/beam-site/pull/610
[8] https://github.com/apache/beam/pull/13603
[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106

[10] https://hub.docker.com/search?q=apache%2Fbeam=image