After thinking this through a bit more, I am inclined to release RC1 with this noted as a known issue, unless there are other more compelling reasons to issues a second RC.
Why? - It is more-or-less by design that end users of Beam Python have dependencies shift under them; breakage and recovery (via pinning to known good versions) must be part of that design. - In many contexts, users will already know this and will have pinned dependencies which therefore won't be impacted. So I am still working through the other failures on https://github.com/apache/beam/pull/28663 to confirm if they are all benign before closing the vote. If someone wants to actually -1 the RC they can do that, but I won't (yet). Kenn On Mon, Oct 9, 2023 at 4:22 PM Kenneth Knowles <k...@apache.org> wrote: > OK I can cherrypick it so they have an upgrade fix. But also we should > instruct users to pin their fastavro version to a good version. That is > probably safer and easier than upgrading Beam. > > Our containers that we build have the version pinned, right? So will this > also cause all the prior containers to have slow start up? > > Kenn > > On Mon, Oct 9, 2023 at 4:13 PM Yi Hu via dev <dev@beam.apache.org> wrote: > >> Yes, and moreover, this specific issue will break the user the same way >> for *all* Beam versions (2.50.0, 2.49.0, etc) after Oct 3. That said the >> issue is not limited to Beam 2.50.0 though. >> >> On Mon, Oct 9, 2023 at 4:08 PM Kenneth Knowles <k...@apache.org> wrote: >> >>> If we had closed the release today, this would still have broken all our >>> users, correct? >>> >>> Kenn >>> >>> On Mon, Oct 9, 2023 at 3:37 PM Anand Inguva via dev <dev@beam.apache.org> >>> wrote: >>> >>>> There was a regression[1] on fastavro latest release 1.8.4. Fix was >>>> merged at https://github.com/apache/beam/pull/28896. The RC1 includes >>>> that version in the range for fastavro[2]. I think we need to CP >>>> https://github.com/apache/beam/pull/28896 to solve the fastavro >>>> regression. >>>> >>>> [1] https://github.com/apache/beam/issues/28811 >>>> [2] >>>> https://github.com/apache/beam/blob/cd653e33b342bd09c76c2bbaca12597fec5b4a2c/sdks/python/setup.py#L245 >>>> >>>> >>>> On Mon, Oct 9, 2023 at 3:15 PM Kenneth Knowles <k...@apache.org> wrote: >>>> >>>>> Ran a couple of Java pipelines "as a newb user" to make sure our >>>>> instructions weren't out of date. There are some errors in the >>>>> instructions >>>>> but they don't have to do with this release. >>>>> >>>>> Re-ran mass_comment.py on https://github.com/apache/beam/pull/28663. >>>>> There are enough red signals there that some triage is needed. Any help >>>>> triaging would be appreciated. >>>>> >>>>> I'll close the vote once everything is run and examined. >>>>> >>>>> Kenn >>>>> >>>>> On Sat, Oct 7, 2023 at 9:58 AM Yi Hu via dev <dev@beam.apache.org> >>>>> wrote: >>>>> >>>>>> +1 (non-binding) Tested on Java IO load tests ( >>>>>> https://github.com/bvolpato/DataflowTemplates/tree/56d18a31c1c95e58543d7a1656bd83d7e859b482/it) >>>>>> BigQueryIO, TextIO, BigtableIO, SpannerIO on Dataflow legacy runner and >>>>>> runner v2 >>>>>> >>>>>> >>>>>> On Fri, Oct 6, 2023 at 3:23 PM Kenneth Knowles <k...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> Additionally we need https://github.com/apache/beam/pull/28665/files >>>>>>> in order to run GHA tests. >>>>>>> >>>>>>> On Fri, Oct 6, 2023 at 3:19 PM Kenneth Knowles <k...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> That PR was prior to many cherry-picks so it is not the signal we >>>>>>>> need. I have updated it to the tip of the release-2.51.0 branch. >>>>>>>> >>>>>>>> There were some post-commit tests involving JPMS that I believe >>>>>>>> need https://github.com/apache/beam/pull/28726 to pass. >>>>>>>> >>>>>>>> Kenn >>>>>>>> >>>>>>>> On Fri, Oct 6, 2023 at 2:53 PM Valentyn Tymofieiev via dev < >>>>>>>> dev@beam.apache.org> wrote: >>>>>>>> >>>>>>>>> > PR to run tests against release branch [12]. >>>>>>>>> >>>>>>>>> https://github.com/apache/beam/pull/28663 is closed and test >>>>>>>>> signal is no longer available. did all the tests pass? >>>>>>>>> >>>>>>>>> On Fri, Oct 6, 2023 at 5:32 AM Alexey Romanenko < >>>>>>>>> aromanenko....@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) >>>>>>>>>> >>>>>>>>>> — >>>>>>>>>> Alexey >>>>>>>>>> >>>>>>>>>> > On 5 Oct 2023, at 18:38, Jean-Baptiste Onofré <j...@nanthrax.net> >>>>>>>>>> wrote: >>>>>>>>>> > >>>>>>>>>> > +1 (binding) >>>>>>>>>> > >>>>>>>>>> > Thanks ! >>>>>>>>>> > Regards >>>>>>>>>> > JB >>>>>>>>>> > >>>>>>>>>> > On Tue, Oct 3, 2023 at 7:58 PM Kenneth Knowles <k...@apache.org> >>>>>>>>>> wrote: >>>>>>>>>> >> >>>>>>>>>> >> Hi everyone, >>>>>>>>>> >> >>>>>>>>>> >> Please review and vote on the release candidate #1 for the >>>>>>>>>> version 2.51.0, as follows: >>>>>>>>>> >> >>>>>>>>>> >> [ ] +1, Approve the release >>>>>>>>>> >> [ ] -1, Do not approve the release (please provide specific >>>>>>>>>> comments) >>>>>>>>>> >> >>>>>>>>>> >> Reviewers are encouraged to test their own use cases with the >>>>>>>>>> release candidate, and vote +1 if no issues are found. Only PMC >>>>>>>>>> member >>>>>>>>>> votes will count towards the final vote, but votes from all community >>>>>>>>>> members is encouraged and helpful for finding regressions; you can >>>>>>>>>> either >>>>>>>>>> test your own use cases or use cases from the validation sheet [10]. >>>>>>>>>> >> >>>>>>>>>> >> The complete staging area is available for your review, which >>>>>>>>>> includes: >>>>>>>>>> >> >>>>>>>>>> >> GitHub Release notes [1], >>>>>>>>>> >> the official Apache source release to be deployed to >>>>>>>>>> dist.apache.org [2], which is signed with the key with >>>>>>>>>> fingerprint FFFFFFFF [3], >>>>>>>>>> >> all artifacts to be deployed to the Maven Central Repository >>>>>>>>>> [4], >>>>>>>>>> >> source code tag "v1.2.3-RC3" [5], >>>>>>>>>> >> website pull request listing the release [6], the blog post >>>>>>>>>> [6], and publishing the API reference manual [7]. >>>>>>>>>> >> Java artifacts were built with Gradle GRADLE_VERSION and >>>>>>>>>> OpenJDK/Oracle JDK JDK_VERSION. >>>>>>>>>> >> Python artifacts are deployed along with the source release to >>>>>>>>>> the dist.apache.org [2] and PyPI[8]. >>>>>>>>>> >> Go artifacts and documentation are available at pkg.go.dev [9] >>>>>>>>>> >> Validation sheet with a tab for 1.2.3 release to help with >>>>>>>>>> validation [10]. >>>>>>>>>> >> Docker images published to Docker Hub [11]. >>>>>>>>>> >> PR to run tests against release branch [12]. >>>>>>>>>> >> >>>>>>>>>> >> The vote will be open for at least 72 hours. It is adopted by >>>>>>>>>> majority approval, with at least 3 PMC affirmative votes. >>>>>>>>>> >> >>>>>>>>>> >> For guidelines on how to try the release in your projects, >>>>>>>>>> check out our blog post at >>>>>>>>>> https://beam.apache.org/blog/validate-beam-release/. >>>>>>>>>> >> >>>>>>>>>> >> Thanks, >>>>>>>>>> >> Kenn >>>>>>>>>> >> >>>>>>>>>> >> [1] https://github.com/apache/beam/milestone/15 >>>>>>>>>> >> [2] https://dist.apache.org/repos/dist/dev/beam/2.51.0 >>>>>>>>>> >> [3] https://dist.apache.org/repos/dist/release/beam/KEYS >>>>>>>>>> >> [4] >>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1356/ >>>>>>>>>> >> [5] https://github.com/apache/beam/tree/v2.51.0-RC1 >>>>>>>>>> >> [6] https://github.com/apache/beam/pull/28800 >>>>>>>>>> >> [7] https://github.com/apache/beam-site/pull/649 >>>>>>>>>> >> [8] https://pypi.org/project/apache-beam/2.51.0rc1/ >>>>>>>>>> >> [9] >>>>>>>>>> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.51.0-RC1/go/pkg/beam >>>>>>>>>> >> [10] >>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=437054928 >>>>>>>>>> >> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>>>>>>>> >> [12] https://github.com/apache/beam/pull/28663 >>>>>>>>>> >>>>>>>>>>
