We determined that the same issue exists in the 2.45 release, so we are going to continue finalizing the release candidate. Thank you for your patience.
Thanks, Danny On Wed, Mar 8, 2023 at 3:15 PM Reuven Lax <[email protected]> wrote: > We are trying to reproduce and debug the issue we saw to validate whether > it was a real regression or not. Will update when we know more. > > On Wed, Mar 8, 2023 at 11:31 AM Danny McCormick <[email protected]> > wrote: > >> >> @Reuven Lax <[email protected]> found a new potential regression in >> BigQuery I/O, so I have paused the release rollout. I had already pushed >> the Python artifacts and Go tags, but not the Java ones. We have since >> temporarily yanked <https://pypi.org/help/#yanked> the Python release >> and deleted the Go tags, they were live for around an hour. The possible >> regression is in Java, so neither of those releases should be affected, but >> x-lang may not work properly because it depends on versioning. I will >> update this thread with next steps when we know more. >> >> Thanks, >> Danny >> On Wed, Mar 8, 2023 at 5:59 AM Jan Lukavský <[email protected]> wrote: >> >>> +1 (binding) >>> >>> Tested Java SDK with Flink and Spark 3 runner. >>> >>> Thanks, >>> Jan >>> >>> On 3/8/23 01:53, Valentyn Tymofieiev via dev wrote: >>> >>> +1. Verified the composition of Python containers and ran Python >>> pipelines on Dataflow runner v1 and runner v2. >>> >>> On Tue, Mar 7, 2023 at 4:11 PM Ritesh Ghorse via dev < >>> [email protected]> wrote: >>> >>>> +1 (non-binding) >>>> Validated Go SDK quickstart on direct and dataflow runner >>>> >>>> On Tue, Mar 7, 2023 at 10:54 AM Alexey Romanenko < >>>> [email protected]> wrote: >>>> >>>>> +1 (binding) >>>>> >>>>> Tested with https://github.com/Talend/beam-samples/ >>>>> (Java SDK v8/v11/v17, Spark 3.x runner). >>>>> >>>>> --- >>>>> Alexey >>>>> >>>>> On 7 Mar 2023, at 07:38, Ahmet Altay via dev <[email protected]> >>>>> wrote: >>>>> >>>>> +1 (binding) - I validated python quickstarts on direct & dataflow >>>>> runners. >>>>> >>>>> Thank you for doing the release! >>>>> >>>>> On Sat, Mar 4, 2023 at 8:01 AM Chamikara Jayalath via dev < >>>>> [email protected]> wrote: >>>>> >>>>>> +1 (binding) >>>>>> >>>>>> Validated multi-language Java and Python pipelines. >>>>>> >>>>>> On Fri, Mar 3, 2023 at 1:59 PM Danny McCormick via dev < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> > I have encountered a failure in a Python pipeline running with >>>>>>> Runner v1: >>>>>>> >>>>>>> > RuntimeError: Beam SDK base version 2.46.0 does not match Dataflow >>>>>>> Python worker version 2.45.0. Please check Dataflow worker startup logs >>>>>>> and >>>>>>> make sure that correct version of Beam SDK is installed. >>>>>>> >>>>>>> > We should understand why Python ValidatesRunner tests (which have >>>>>>> passed) didn't catch this error. >>>>>>> >>>>>>> > This can be remediated in Dataflow containers without changes to >>>>>>> the release candidate. >>>>>>> >>>>>>> Good catch! I've kicked off a release to fix this, it should be done >>>>>>> later this evening - I won't be available when it completes, but I would >>>>>>> expect it to be around 5:00 PST. >>>>>>> >>>>>>> On Fri, Mar 3, 2023 at 3:49 PM Danny McCormick < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hey Reuven, could you provide some more context on the bug/why it >>>>>>>> is important? Does it meet the standard in >>>>>>>> https://beam.apache.org/contribute/release-guide/#7-triage-release-blocking-issues-in-github? >>>>>>>> >>>>>>>> >>>>>>>> The release branch was cut last Wednesday, so that is why it is not >>>>>>>> included. >>>>>>>> >>>>>>> >>>>>> Seems like this was a revert of a previous commit that was also not >>>>>> included in the 2.46.0 release branch ( >>>>>> https://github.com/apache/beam/pull/25627) ? >>>>>> >>>>>> If so we might not need a new RC but good to confirm. >>>>>> >>>>>> Thanks, >>>>>> Cham >>>>>> >>>>>> >>>>>>>> On Fri, Mar 3, 2023 at 3:24 PM Reuven Lax <[email protected]> wrote: >>>>>>>> >>>>>>>>> If possible, I would like to see if we could include >>>>>>>>> https://github.com/apache/beam/pull/25642 as we believe this bug >>>>>>>>> has been impacting multiple users. This was merged 4 days ago, but >>>>>>>>> this RC >>>>>>>>> cut does not seem to include it. >>>>>>>>> >>>>>>>>> On Fri, Mar 3, 2023 at 12:18 PM Valentyn Tymofieiev via dev < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> I have encountered a failure in a Python pipeline running with >>>>>>>>>> Runner v1: >>>>>>>>>> >>>>>>>>>> RuntimeError: Beam SDK base version 2.46.0 does not match >>>>>>>>>> Dataflow Python worker version 2.45.0. Please check Dataflow worker >>>>>>>>>> startup >>>>>>>>>> logs and make sure that correct version of Beam SDK is installed. >>>>>>>>>> >>>>>>>>>> We should understand why Python ValidatesRunner tests (which have >>>>>>>>>> passed) didn't catch this error. >>>>>>>>>> >>>>>>>>>> This can be remediated in Dataflow containers without changes to >>>>>>>>>> the release candidate. >>>>>>>>>> >>>>>>>>>> On Fri, Mar 3, 2023 at 11:22 AM Robert Bradshaw via dev < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> +1 (binding). >>>>>>>>>>> >>>>>>>>>>> I verified that the artifacts and signatures all look good, all >>>>>>>>>>> the >>>>>>>>>>> containers are pushed, and tested some pipelines with a fresh >>>>>>>>>>> install >>>>>>>>>>> from one of the Python wheels. >>>>>>>>>>> >>>>>>>>>>> On Fri, Mar 3, 2023 at 11:13 AM Danny McCormick >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> > >>>>>>>>>>> > > The released artifacts seem to be missing the last commit at >>>>>>>>>>> > > >>>>>>>>>>> https://github.com/apache/beam/commit/c528eab18b32342daed53b750fe330d30c7e5224 >>>>>>>>>>> > > . Is this essential to the release, or just useful for >>>>>>>>>>> validating it? >>>>>>>>>>> > >>>>>>>>>>> > It's strictly a test infrastructure change, it has no >>>>>>>>>>> functional impact. For context, the changes included were from >>>>>>>>>>> https://github.com/apache/beam/pull/25661 and >>>>>>>>>>> https://github.com/apache/beam/pull/25654, both were keeping >>>>>>>>>>> integration tests from running correctly. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> > On Fri, Mar 3, 2023 at 2:09 PM Robert Bradshaw < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> The released artifacts seem to be missing the last commit at >>>>>>>>>>> >> >>>>>>>>>>> https://github.com/apache/beam/commit/c528eab18b32342daed53b750fe330d30c7e5224 >>>>>>>>>>> >> . Is this essential to the release, or just useful for >>>>>>>>>>> validating it? >>>>>>>>>>> >> >>>>>>>>>>> >> On Fri, Mar 3, 2023 at 11:02 AM Danny McCormick >>>>>>>>>>> >> <[email protected]> wrote: >>>>>>>>>>> >> > >>>>>>>>>>> >> > Thanks for calling that out, and thanks for helping me fix >>>>>>>>>>> it! We should be all set now >>>>>>>>>>> >> > >>>>>>>>>>> >> > On Fri, Mar 3, 2023 at 1:38 PM Robert Bradshaw < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >> >> >>>>>>>>>>> >> >> It appears your public key is not published in >>>>>>>>>>> >> >> https://dist.apache.org/repos/dist/release/beam/KEYS . >>>>>>>>>>> >> >> >>>>>>>>>>> >> >> On Fri, Mar 3, 2023 at 8:33 AM Anand Inguva via dev < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > +1 (non-binding) >>>>>>>>>>> >> >> > Tested python wordcount quick start >>>>>>>>>>> https://beam.apache.org/get-started/quickstart-py/ on Direct >>>>>>>>>>> Runner and Dataflow Runner. >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > Thanks! >>>>>>>>>>> >> >> > >>>>>>>>>>> >> >> > On Fri, Mar 3, 2023 at 11:21 AM Bruno Volpato via dev < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> +1 (non-binding) >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> Tested with >>>>>>>>>>> https://github.com/GoogleCloudPlatform/DataflowTemplates (Java >>>>>>>>>>> SDK 11, Dataflow runner). >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> Thanks Danny! >>>>>>>>>>> >> >> >> >>>>>>>>>>> >> >> >> On Thu, Mar 2, 2023 at 5:16 PM Danny McCormick via dev < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >> >> >>> >>>>>>>>>>> >> >> >>> Hi everyone, >>>>>>>>>>> >> >> >>> Please review and vote on release candidate #1 for the >>>>>>>>>>> version 2.46.0, as follows: [ ] +1, Approve the release [ ] -1, Do >>>>>>>>>>> not >>>>>>>>>>> approve the release (please provide specific comments) Reviewers are >>>>>>>>>>> encouraged to test their own use cases with the release candidate, >>>>>>>>>>> and vote >>>>>>>>>>> +1 if no issues are found. The complete staging area is available >>>>>>>>>>> for your >>>>>>>>>>> review, which includes: * GitHub Release notes [1], * the official >>>>>>>>>>> Apache >>>>>>>>>>> source release to be deployed to dist.apache.org [2], which is >>>>>>>>>>> signed with the key with fingerprint >>>>>>>>>>> FC383FCDE7D7E86699954EF2509872C8031C4DFB [3], * all artifacts to be >>>>>>>>>>> deployed to the Maven Central Repository [4], * source code tag >>>>>>>>>>> "v2.46.0-RC1" [5], * website pull request listing the release [6], >>>>>>>>>>> the blog >>>>>>>>>>> post [6], and publishing the API reference manual [7]. * Java >>>>>>>>>>> artifacts >>>>>>>>>>> were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK >>>>>>>>>>> JDK_VERSION. * >>>>>>>>>>> Python artifacts are deployed along with the source release to the >>>>>>>>>>> dist.apache.org [2] and PyPI[8]. * Go artifacts and >>>>>>>>>>> documentation are available at pkg.go.dev [9] * Validation >>>>>>>>>>> sheet with a tab for 2.46.0 release to help with validation [10]. * >>>>>>>>>>> Docker >>>>>>>>>>> images published to Docker Hub [11]. >>>>>>>>>>> >> >> >>> * PR to run tests against release branch [12]. The >>>>>>>>>>> vote will be open for at least 72 hours. It is adopted by majority >>>>>>>>>>> approval, with at least 3 PMC affirmative votes. For guidelines on >>>>>>>>>>> how to >>>>>>>>>>> try the release in your projects, check out our blog post at >>>>>>>>>>> /blog/validate-beam-release/. Thanks, Danny [1] >>>>>>>>>>> https://github.com/apache/beam/milestone/9 [2] >>>>>>>>>>> https://dist.apache.org/repos/dist/dev/beam/2.46.0/ [3] >>>>>>>>>>> https://dist.apache.org/repos/dist/release/beam/KEYS [4] >>>>>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1306/ >>>>>>>>>>> [5] https://github.com/apache/beam/tree/v2.46.0-RC1 [6] >>>>>>>>>>> https://github.com/apache/beam/pull/25693 [7] >>>>>>>>>>> https://github.com/apache/beam-site/pull/641 [8] >>>>>>>>>>> https://pypi.org/project/apache-beam/2.46.0rc1/ [9] >>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam/sdks/[email protected]/go/pkg/beam >>>>>>>>>>> [10] >>>>>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=247587190 >>>>>>>>>>> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>>>>>>>>> >> >> >>> [12] https://github.com/apache/beam/pull/25600 >>>>>>>>>>> >>>>>>>>>> >>>>>
