On Thu, Mar 28, 2024 at 8:36 AM Chamikara Jayalath <chamik...@google.com> wrote:
> Just releasing Python can break multi-lang by default (unless expansion > service is overridden manually) since we match versions across languages > when picking the default expansion service. > > > https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L42 > Correct link: https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L352 > > > Thanks, > Cham > > On Thu, Mar 28, 2024 at 8:26 AM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> > The patch itself [1] is trivial, however, the release process is not >> trivial. There is little documentation nor practice for a patch release >> process. I could imagine two options >> >> I think there's not a ton of documentation because we haven't done it, >> but all the release workflows were authored in such a way that they should >> "just work", outside of cutting the release branch itself. So the workflow >> should be almost identical to the existing one, but with several steps >> skipped (cherry picks, beam website, most validation). Notably, this >> shouldn't be any easier/harder if we're doing it for one language or all 3. >> >> I can take that on if needed. >> >> > Besides, there should be a Beam YAML validation workflow and added in >> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >> >> > If we do a patch release for Python SDK, let's also patch another known >> issue for which fix is available: >> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >> >> +1 to both of these >> >> On Thu, Mar 28, 2024 at 11:25 AM Yi Hu via dev <dev@beam.apache.org> >> wrote: >> >>> Thanks Valentyn for raising this. In this case, Python containers will >>> also be included. Different from PyPI wheels, docker tag can override so it >>> can stay with 2.55.0 >>> >>> On Thu, Mar 28, 2024 at 11:15 AM Valentyn Tymofieiev < >>> valen...@google.com> wrote: >>> >>>> If we do a patch release for Python SDK, let's also patch another known >>>> issue for which fix is available: >>>> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >>>> >>>> On Thu, Mar 28, 2024 at 8:01 AM Yi Hu via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>>> 2.55.0 release manager here >>>>> >>>>> The patch itself [1] is trivial, however, the release process is not >>>>> trivial. There is little documentation nor practice for a patch release >>>>> process. I could imagine two options >>>>> >>>>> 1. Do a full "2.55.1" release >>>>> >>>>> 2. Do a patch release only for Python SDK, that is >>>>> a. cherry-pick [1] into release-2.55.0 branch >>>>> b. tag a 2.55.1rc1 release candidate - note that the source code of >>>>> release candidate (e.g. apache_beam/version.py) still reads 2.55.0. This >>>>> ensures Python SDK picks up the Java expansion service / job server of >>>>> existing version (2.55.0). We did it once for Go SDK ( >>>>> https://github.com/apache/beam/tree/sdks/v2.48.2) >>>>> c. Build the release candidate for Python wheels (also Python >>>>> containers? Not sure if it is needed) >>>>> d. send out the RC for validation >>>>> e. finalize the release >>>>> >>>>> If we decided to do a patch release I would prefer option 2. I can >>>>> take on that if decided to do. However, if we decide do a full release (or >>>>> both Java and Python) I would suggest defer to next release cycle, as the >>>>> release process itself could take ~10 days minimum if there is single RC. >>>>> >>>>> Besides, there should be a Beam YAML validation workflow and added in >>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>>>> >>>>> >>>>> [1] https://github.com/apache/beam/pull/30780 >>>>> >>>>> On Thu, Mar 28, 2024 at 10:22 AM Danny McCormick via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> +1 on a patch release - we've done a fair amount of work to make >>>>>> releasing easier, and one of my hopes is that it will enable quick >>>>>> patches >>>>>> like this. I'd vote we try to fix the underlying Java piece as well, >>>>>> though, doing a patch release for one language shouldn't be significantly >>>>>> cheaper than doing it for multiple languages. >>>>>> >>>>>> Thanks, >>>>>> Danny >>>>>> >>>>>> On Wed, Mar 27, 2024 at 7:19 PM Robert Burke <rob...@frantil.com> >>>>>> wrote: >>>>>> >>>>>>> +1 to a targeted patch release. >>>>>>> >>>>>>> We did the same for the Go SDK a little while back. It would be good >>>>>>> to see what's different for a different SDK. >>>>>>> >>>>>>> On Wed, Mar 27, 2024, 4:01 PM Robert Bradshaw via dev < >>>>>>> dev@beam.apache.org> wrote: >>>>>>> >>>>>>>> Given the severity of the breakage, and the simplicity of the >>>>>>>> workaround, I'm in favor of a patch release. I think we could do >>>>>>>> Python-only, which would make the process even more lightweight. >>>>>>>> >>>>>>>> On Wed, Mar 27, 2024 at 3:48 PM Jeff Kinard <j...@thekinards.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Beam 2.55 was released with a bug that causes WriteToJson on Beam >>>>>>>>> YAML to fail when using the Java variant. This also affects any user >>>>>>>>> attempting to use the Xlang JsonWriteTransformProvider - >>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/JsonWriteTransformProvider.java >>>>>>>>> >>>>>>>>> This is due to a change to >>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/build.gradle >>>>>>>>> that removed >>>>>>>>> a dependency on everit which also removed it from being packaged >>>>>>>>> into the expansion service JAR: >>>>>>>>> beam-sdks-java-extensions-sql-expansion-service-2.55.0.jar >>>>>>>>> >>>>>>>>> There is a temporary fix to disable the provider in Beam YAML: >>>>>>>>> https://github.com/apache/beam/pull/30777 >>>>>>>>> >>>>>>>>> I think with the total loss of function, and a trivial fix, it is >>>>>>>>> worth creating a patch release of Beam 2.55 to include this fix. >>>>>>>>> >>>>>>>>> - Jeff >>>>>>>>> >>>>>>>>>
