I guess my vote is -0 since I don't have enough context on this issue. A number of people with more awareness of how severe this is have voted +1 so I will not try to block the release.
Kenn On Mon, Sep 27, 2021 at 2:11 PM Kenneth Knowles <k...@apache.org> wrote: > I have to disagree with the other PMC members here, or at least dig in to > the question: every pipeline that uses a Repeatedly trigger at the top > level will be rejected. Is this so rare in Python that it is OK? > > Kenn > > On Mon, Sep 27, 2021 at 1:56 PM Robert Burke <r...@google.com> wrote: > >> On it. Thanks! >> >> On Mon, Sep 27, 2021 at 1:18 PM Udi Meiri <eh...@google.com> wrote: >> >>> Daniel/Robert, feel free to make changes to this PR: >>> https://github.com/apache/beam/pull/15543 >>> >>> On Mon, Sep 27, 2021 at 12:08 PM Daniel Oliveira <danolive...@google.com> >>> wrote: >>> >>>> +1 >>>> >>>> I hadn't realized the pipelines were still finishing successfully. I >>>> retried wordcount with that in mind and confirmed it finishes successfully, >>>> so this isn't a blocker. >>>> >>>> Although maybe we should add this to the "Known Issues" because I can >>>> easily see those messages being interpreted as a pipeline failure. >>>> >>>> On Sun, Sep 26, 2021 at 7:26 PM Robert Burke <rob...@frantil.com> >>>> wrote: >>>> >>>>> That's on the SDK side, and it just means the PCollection metrics are >>>>> being returned, buy not handled by the SDK. At present the pipeline >>>>> results >>>>> only handle PTransform metrics. >>>>> >>>>> As such it's not a regression, as adding those is still under >>>>> development. >>>>> >>>>> On Sun, Sep 26, 2021, 7:02 PM Daniel Oliveira <danolive...@google.com> >>>>> wrote: >>>>> >>>>>> I tried validating wordcount with the Go SDK on Flink. The pipeline >>>>>> failed with a wall of errors like the following. I tried this on Flink >>>>>> 1.11, 1.12, and 1.13 job servers built from source at the RC1 commit, >>>>>> same >>>>>> errors on all of them. >>>>>> >>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" >>>>>>> payload:"\x8d%" labels:{key:"PCOLLECTION" value:"n9"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" >>>>>>> payload:"\x8d%" labels:{key:"PCOLLECTION" value:"n10"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" >>>>>>> payload:"\x8d%" labels:{key:"PCOLLECTION" value:"n8"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:element_count:v1" type:"beam:metrics:sum_int64:v1" >>>>>>> payload:"\x8d%" labels:{key:"PCOLLECTION" value:"n7"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:sampled_byte_size:v1" >>>>>>> type:"beam:metrics:distribution_int64:v1" >>>>>>> payload:"\x8d%\xfa\xbf\x05\x04\xa8\x0c" labels:{key:"PCOLLECTION" >>>>>>> value:"n7"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:sampled_byte_size:v1" >>>>>>> type:"beam:metrics:distribution_int64:v1" >>>>>>> payload:"\xb9\x05\xcf6\x05\x11" >>>>>>> labels:{key:"PCOLLECTION" value:"n9"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:sampled_byte_size:v1" >>>>>>> type:"beam:metrics:distribution_int64:v1" >>>>>>> payload:"\xc5\x05\x8a1\x04\x12" >>>>>>> labels:{key:"PCOLLECTION" value:"n8"} >>>>>>> 2021/09/26 18:21:49 Failed to deduce Step from MonitoringInfo: >>>>>>> urn:"beam:metric:sampled_byte_size:v1" >>>>>>> type:"beam:metrics:distribution_int64:v1" >>>>>>> payload:"\x8d%\xb3\xa7\x07\x14\"" labels:{key:"PCOLLECTION" >>>>>>> value:"n10"} >>>>>>> {...} >>>>>> >>>>>> >>>>>> @Robert Burke <r...@google.com> I think you might know what's going >>>>>> on here. Is this solvable with a cherry-pick and a new RC? >>>>>> >>>>>> On Fri, Sep 24, 2021 at 4:25 PM Ahmet Altay <al...@google.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 24, 2021 at 4:21 PM Udi Meiri <eh...@google.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 24, 2021 at 4:18 PM Ahmet Altay <al...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thank you for the update related to allow_unsafe_triggers. My vote >>>>>>>>> is still a +1. >>>>>>>>> >>>>>>>>> If we decide to move forward with this RC, could you please >>>>>>>>> include this bug in the known issues list under the changes.md for >>>>>>>>> this >>>>>>>>> release? >>>>>>>>> >>>>>>>> Yes, that's included in >>>>>>>> https://github.com/apache/beam/pull/15543/files. >>>>>>>> >>>>>>> >>>>>>> Thank you! >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 24, 2021 at 3:56 PM Chamikara Jayalath < >>>>>>>>> chamik...@google.com> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) >>>>>>>>>> Validated a few scenarios from the spreadsheet. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Cham >>>>>>>>>> >>>>>>>>>> On Fri, Sep 24, 2021 at 3:07 PM Robert Bradshaw < >>>>>>>>>> rober...@google.com> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding) >>>>>>>>>>> >>>>>>>>>>> All the artifacts and signatures look good. >>>>>>>>>>> >>>>>>>>>>> I don't think the unsafe trigger check is severe enough to block >>>>>>>>>>> the release. >>>>>>>>>>> >>>>>>>>>>> On Fri, Sep 24, 2021 at 2:36 PM Udi Meiri <eh...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> > >>>>>>>>>>> > Hi everyone, we found a bug during testing. It has to do with >>>>>>>>>>> Python SDK's allow_unsafe_triggers check. >>>>>>>>>>> > There is a preliminary fix that will go to master. >>>>>>>>>>> > >>>>>>>>>>> > For the 2.33.0 release, I'm leaning towards not making a new >>>>>>>>>>> RC since there is a workaround: pass the flag >>>>>>>>>>> --allow_unsafe_triggers. >>>>>>>>>>> > >>>>>>>>>>> > Please reevaluate your votes accordingly and recast if you've >>>>>>>>>>> changed your vote. >>>>>>>>>>> > >>>>>>>>>>> > Thanks >>>>>>>>>>> > >>>>>>>>>>> > On Fri, Sep 24, 2021 at 12:48 PM Alexey Romanenko < >>>>>>>>>>> aromanenko....@gmail.com> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> On 24 Sep 2021, at 20:45, Udi Meiri <eh...@google.com> wrote: >>>>>>>>>>> >> Alexey is this something that we should put in the release >>>>>>>>>>> notes, or some other change? >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> Yes, I think it could be helpful to mention that Beam >>>>>>>>>>> Jackson’s deps was bumped and it may require an update of Jackson’s >>>>>>>>>>> runtime >>>>>>>>>>> deps for Spark 2 users as well. >>>>>>>>>>> >> >>>>>>>>>>> >> — >>>>>>>>>>> >> Alexey >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> — >>>>>>>>>>> >>> Alexey >>>>>>>>>>> >>> >>>>>>>>>>> >>> On 24 Sep 2021, at 16:17, Alexey Romanenko < >>>>>>>>>>> aromanenko....@gmail.com> wrote: >>>>>>>>>>> >>> >>>>>>>>>>> >>> I checked with beam-samples [1] and noticed an issue to run >>>>>>>>>>> some pipelines with Spark 2 runner (Spark 3 seems is ok). >>>>>>>>>>> >>> It looks that it’s caused by new Jackson's version updated >>>>>>>>>>> recently [2], even if it’s a minor update but it works fine with a >>>>>>>>>>> previous >>>>>>>>>>> one. >>>>>>>>>>> >>> I’ll try to find a workaround and get back with a results of >>>>>>>>>>> this. >>>>>>>>>>> >>> >>>>>>>>>>> >>> [1] https://github.com/Talend/beam-samples/ >>>>>>>>>>> >>> [2] >>>>>>>>>>> https://github.com/apache/beam/commit/9694f70df1447e96684b665279679edafec13a0c >>>>>>>>>>> >>> >>>>>>>>>>> >>> — >>>>>>>>>>> >>> Alexey >>>>>>>>>>> >>> >>>>>>>>>>> >>> On 24 Sep 2021, at 11:17, Jan Lukavský <je...@seznam.cz> >>>>>>>>>>> wrote: >>>>>>>>>>> >>> >>>>>>>>>>> >>> +1 (non-binding) >>>>>>>>>>> >>> >>>>>>>>>>> >>> Validated several use-cases using non-portable Flink with >>>>>>>>>>> Java SDK. >>>>>>>>>>> >>> >>>>>>>>>>> >>> On 9/24/21 4:55 AM, Valentyn Tymofieiev wrote: >>>>>>>>>>> >>> >>>>>>>>>>> >>> +1. Ran several Python batch and streaming pipelines on >>>>>>>>>>> Dataflow and checked that Dataflow containers have required >>>>>>>>>>> dependencies of >>>>>>>>>>> Beam. >>>>>>>>>>> >>> >>>>>>>>>>> >>> On Thu, Sep 23, 2021 at 7:03 PM Robert Burke < >>>>>>>>>>> lostl...@apache.org> wrote: >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> +1 (non-binding) >>>>>>>>>>> >>>> I validated the Go Quickstart (wordcount), and my ray >>>>>>>>>>> tracer against the Go Direct runner, Dataflow, and Spark (ensuring >>>>>>>>>>> the rc1 >>>>>>>>>>> tagged container was used) and they executed successfully. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> I needed to manually synthesize a pseudo-version to ensure >>>>>>>>>>> I was using the tagged branch version >>>>>>>>>>> (v2.0.0-20210914211513-b358127f9859) >>>>>>>>>>> instead of simply using v2.33.0-RC1. >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> It either can't find the package with the right tagged >>>>>>>>>>> version, or it can't find the version. It's not clear to me what >>>>>>>>>>> the issue >>>>>>>>>>> is, but it's not notionally a release blocker. I'll investigate >>>>>>>>>>> further >>>>>>>>>>> once we have a full release, as it's probably some unspecified >>>>>>>>>>> behavior due >>>>>>>>>>> to how we transitioned to Go Modules (which strongly recommended >>>>>>>>>>> doing a >>>>>>>>>>> major version bump for such transitions, which seems a bit >>>>>>>>>>> excessive for >>>>>>>>>>> Beam as a whole...). >>>>>>>>>>> >>>> >>>>>>>>>>> >>>> On 2021/09/23 03:59:18, Ahmet Altay <al...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>> > +1 on the RC. I validated python quick start examples on >>>>>>>>>>> direct runners. >>>>>>>>>>> >>>> > Thank you Udi. >>>>>>>>>>> >>>> > >>>>>>>>>>> >>>> > On Wed, Sep 22, 2021 at 2:20 PM Robert Burke < >>>>>>>>>>> rob...@frantil.com> wrote: >>>>>>>>>>> >>>> > >>>>>>>>>>> >>>> > > Just an FYI that intend to validate the Go SDK for this >>>>>>>>>>> release but can't >>>>>>>>>>> >>>> > > get to it until tomorrow (Thursday). I'm catching up >>>>>>>>>>> from a week of >>>>>>>>>>> >>>> > > vacation. Apologies for the inconvenience. >>>>>>>>>>> >>>> > > >>>>>>>>>>> >>>> > > On Tue, Sep 21, 2021, 10:59 AM Udi Meiri < >>>>>>>>>>> eh...@google.com> wrote: >>>>>>>>>>> >>>> > > >>>>>>>>>>> >>>> > >> I updated the affected and fixed version fields for >>>>>>>>>>> >>>> > >> https://issues.apache.org/jira/browse/BEAM-12356. >>>>>>>>>>> >>>> > >> >>>>>>>>>>> >>>> > >> >>>>>>>>>>> >>>> > >> On Tue, Sep 21, 2021 at 10:48 AM Reuven Lax < >>>>>>>>>>> re...@google.com> wrote: >>>>>>>>>>> >>>> > >> >>>>>>>>>>> >>>> > >>> Unfortunate - I didn't realize that >>>>>>>>>>> >>>> > >>> https://github.com/apache/beam/pull/15480 didn't >>>>>>>>>>> make the cut. >>>>>>>>>>> >>>> > >>> >>>>>>>>>>> >>>> > >>> This bug was a regression in Beam 2.32.0, and is >>>>>>>>>>> blocking multiple users >>>>>>>>>>> >>>> > >>> from updating. >>>>>>>>>>> >>>> > >>> >>>>>>>>>>> >>>> > >>> On Tue, Sep 21, 2021 at 10:33 AM Udi Meiri < >>>>>>>>>>> u...@apache.org> wrote: >>>>>>>>>>> >>>> > >>> >>>>>>>>>>> >>>> > >>>> Hi everyone, >>>>>>>>>>> >>>> > >>>> Please review and vote on the release candidate #1 >>>>>>>>>>> for the version >>>>>>>>>>> >>>> > >>>> 2.33.0, as follows: >>>>>>>>>>> >>>> > >>>> [ ] +1, Approve the release >>>>>>>>>>> >>>> > >>>> [ ] -1, Do not approve the release (please provide >>>>>>>>>>> specific comments) >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> Reviewers are encouraged to test their own use cases >>>>>>>>>>> with the release >>>>>>>>>>> >>>> > >>>> candidate, and vote +1 if >>>>>>>>>>> >>>> > >>>> no issues are found. >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> The complete staging area is available for your >>>>>>>>>>> review, which includes: >>>>>>>>>>> >>>> > >>>> * JIRA release notes [1], >>>>>>>>>>> >>>> > >>>> * the official Apache source release to be deployed >>>>>>>>>>> to dist.apache.org >>>>>>>>>>> >>>> > >>>> [2], which is signed with the key with fingerprint >>>>>>>>>>> 587B049C36DAAFE6 [3], >>>>>>>>>>> >>>> > >>>> * all artifacts to be deployed to the Maven Central >>>>>>>>>>> Repository [4], >>>>>>>>>>> >>>> > >>>> * source code tag "v2.33.0-RC1" [5], >>>>>>>>>>> >>>> > >>>> * website pull request listing the release [6], the >>>>>>>>>>> blog post [6], and >>>>>>>>>>> >>>> > >>>> publishing the API reference manual [7]. >>>>>>>>>>> >>>> > >>>> * Java artifacts were built with Maven 3.6.3 and >>>>>>>>>>> OpenJDK 1.8.0_181. >>>>>>>>>>> >>>> > >>>> * Python artifacts are deployed along with the >>>>>>>>>>> source release to the >>>>>>>>>>> >>>> > >>>> dist.apache.org [2] and pypy[8]. >>>>>>>>>>> >>>> > >>>> * Validation sheet with a tab for 2.33.0 release to >>>>>>>>>>> help with >>>>>>>>>>> >>>> > >>>> validation [9]. >>>>>>>>>>> >>>> > >>>> * Docker images published to Docker Hub [10]. >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> The vote will be open for at least 72 hours. It is >>>>>>>>>>> adopted by majority >>>>>>>>>>> >>>> > >>>> approval, with at least 3 PMC affirmative votes. >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> For guidelines on how to try the release in your >>>>>>>>>>> projects, check out >>>>>>>>>>> >>>> > >>>> our blog post at >>>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/. >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> Thanks, >>>>>>>>>>> >>>> > >>>> Release Manager >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> [1] >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404 >>>>>>>>>>> >>>> > >>>> [2] >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.33.0/ >>>>>>>>>>> >>>> > >>>> [3] >>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS >>>>>>>>>>> >>>> > >>>> [4] >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1234/ >>>>>>>>>>> >>>> > >>>> [5] https://github.com/apache/beam/tree/v2.33.0-RC1 >>>>>>>>>>> >>>> > >>>> [6] https://github.com/apache/beam/pull/15543 >>>>>>>>>>> >>>> > >>>> [7] https://github.com/apache/beam-site/pull/619 >>>>>>>>>>> >>>> > >>>> [8] https://pypi.org/project/apache-beam/2.33.0rc1/ >>>>>>>>>>> >>>> > >>>> [9] >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1705275493 >>>>>>>>>>> >>>> > >>>> [10] >>>>>>>>>>> https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>> >>>>>>>>>>> >>>> > >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >> >>>>>>>>>>> >>>>>>>>>>
