> Just releasing Python can break multi-lang by default (unless expansion service is overridden manually) since we match versions across languages when picking the default expansion service.
Yes, that's why I proposed "the source code of release candidate (e.g. apache_beam/version.py) still reads 2.55.0. " Anyways it seems doing a full release is preferred as it reduces the risk of breakages. On Thu, Mar 28, 2024 at 11:38 AM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > > > On Thu, Mar 28, 2024 at 8:36 AM Chamikara Jayalath <chamik...@google.com> > wrote: > >> Just releasing Python can break multi-lang by default (unless expansion >> service is overridden manually) since we match versions across languages >> when picking the default expansion service. >> >> >> https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L42 >> > > Correct link: > https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L352 > > >> >> >> Thanks, >> Cham >> >> On Thu, Mar 28, 2024 at 8:26 AM Danny McCormick via dev < >> dev@beam.apache.org> wrote: >> >>> > The patch itself [1] is trivial, however, the release process is not >>> trivial. There is little documentation nor practice for a patch release >>> process. I could imagine two options >>> >>> I think there's not a ton of documentation because we haven't done it, >>> but all the release workflows were authored in such a way that they should >>> "just work", outside of cutting the release branch itself. So the workflow >>> should be almost identical to the existing one, but with several steps >>> skipped (cherry picks, beam website, most validation). Notably, this >>> shouldn't be any easier/harder if we're doing it for one language or all 3. >>> >>> I can take that on if needed. >>> >>> > Besides, there should be a Beam YAML validation workflow and added in >>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>> >>> > If we do a patch release for Python SDK, let's also patch another >>> known issue for which fix is available: >>> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >>> >>> +1 to both of these >>> >>> On Thu, Mar 28, 2024 at 11:25 AM Yi Hu via dev <dev@beam.apache.org> >>> wrote: >>> >>>> Thanks Valentyn for raising this. In this case, Python containers will >>>> also be included. Different from PyPI wheels, docker tag can override so it >>>> can stay with 2.55.0 >>>> >>>> On Thu, Mar 28, 2024 at 11:15 AM Valentyn Tymofieiev < >>>> valen...@google.com> wrote: >>>> >>>>> If we do a patch release for Python SDK, let's also patch another >>>>> known issue for which fix is available: >>>>> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >>>>> >>>>> On Thu, Mar 28, 2024 at 8:01 AM Yi Hu via dev <dev@beam.apache.org> >>>>> wrote: >>>>> >>>>>> 2.55.0 release manager here >>>>>> >>>>>> The patch itself [1] is trivial, however, the release process is not >>>>>> trivial. There is little documentation nor practice for a patch release >>>>>> process. I could imagine two options >>>>>> >>>>>> 1. Do a full "2.55.1" release >>>>>> >>>>>> 2. Do a patch release only for Python SDK, that is >>>>>> a. cherry-pick [1] into release-2.55.0 branch >>>>>> b. tag a 2.55.1rc1 release candidate - note that the source code of >>>>>> release candidate (e.g. apache_beam/version.py) still reads 2.55.0. This >>>>>> ensures Python SDK picks up the Java expansion service / job server of >>>>>> existing version (2.55.0). We did it once for Go SDK ( >>>>>> https://github.com/apache/beam/tree/sdks/v2.48.2) >>>>>> c. Build the release candidate for Python wheels (also Python >>>>>> containers? Not sure if it is needed) >>>>>> d. send out the RC for validation >>>>>> e. finalize the release >>>>>> >>>>>> If we decided to do a patch release I would prefer option 2. I can >>>>>> take on that if decided to do. However, if we decide do a full release >>>>>> (or >>>>>> both Java and Python) I would suggest defer to next release cycle, as the >>>>>> release process itself could take ~10 days minimum if there is single RC. >>>>>> >>>>>> Besides, there should be a Beam YAML validation workflow and added in >>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>>>>> >>>>>> >>>>>> [1] https://github.com/apache/beam/pull/30780 >>>>>> >>>>>> On Thu, Mar 28, 2024 at 10:22 AM Danny McCormick via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> +1 on a patch release - we've done a fair amount of work to make >>>>>>> releasing easier, and one of my hopes is that it will enable quick >>>>>>> patches >>>>>>> like this. I'd vote we try to fix the underlying Java piece as well, >>>>>>> though, doing a patch release for one language shouldn't be >>>>>>> significantly >>>>>>> cheaper than doing it for multiple languages. >>>>>>> >>>>>>> Thanks, >>>>>>> Danny >>>>>>> >>>>>>> On Wed, Mar 27, 2024 at 7:19 PM Robert Burke <rob...@frantil.com> >>>>>>> wrote: >>>>>>> >>>>>>>> +1 to a targeted patch release. >>>>>>>> >>>>>>>> We did the same for the Go SDK a little while back. It would be >>>>>>>> good to see what's different for a different SDK. >>>>>>>> >>>>>>>> On Wed, Mar 27, 2024, 4:01 PM Robert Bradshaw via dev < >>>>>>>> dev@beam.apache.org> wrote: >>>>>>>> >>>>>>>>> Given the severity of the breakage, and the simplicity of the >>>>>>>>> workaround, I'm in favor of a patch release. I think we could do >>>>>>>>> Python-only, which would make the process even more lightweight. >>>>>>>>> >>>>>>>>> On Wed, Mar 27, 2024 at 3:48 PM Jeff Kinard <j...@thekinards.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Beam 2.55 was released with a bug that causes WriteToJson on Beam >>>>>>>>>> YAML to fail when using the Java variant. This also affects any user >>>>>>>>>> attempting to use the Xlang JsonWriteTransformProvider - >>>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/JsonWriteTransformProvider.java >>>>>>>>>> >>>>>>>>>> This is due to a change to >>>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/build.gradle >>>>>>>>>> that removed >>>>>>>>>> a dependency on everit which also removed it from being packaged >>>>>>>>>> into the expansion service JAR: >>>>>>>>>> beam-sdks-java-extensions-sql-expansion-service-2.55.0.jar >>>>>>>>>> >>>>>>>>>> There is a temporary fix to disable the provider in Beam YAML: >>>>>>>>>> https://github.com/apache/beam/pull/30777 >>>>>>>>>> >>>>>>>>>> I think with the total loss of function, and a trivial fix, it is >>>>>>>>>> worth creating a patch release of Beam 2.55 to include this fix. >>>>>>>>>> >>>>>>>>>> - Jeff >>>>>>>>>> >>>>>>>>>>
