On Thu, Mar 28, 2024 at 8:57 AM Jan Lukavský <je...@seznam.cz> wrote:
> +1 to either doing full release or deferring to 2.56.0. > +1. Given that validation/testing required for unupdated SDKs should be minimum, I don't think a full release will be that much overhead compared to just releasing Python SDK. Also this is a good opportunity to figure out any friction when performing a patch release I believe. - Cham > Jan > On 3/28/24 16:52, Yi Hu via dev wrote: > > > Just releasing Python can break multi-lang by default (unless expansion > service is overridden manually) since we match versions across languages > when picking the default expansion service. > > Yes, that's why I proposed "the source code of release candidate (e.g. > apache_beam/version.py) still reads 2.55.0. " Anyways it seems doing a full > release is preferred as it reduces the risk of breakages. > > On Thu, Mar 28, 2024 at 11:38 AM Chamikara Jayalath via dev < > dev@beam.apache.org> wrote: > >> >> >> On Thu, Mar 28, 2024 at 8:36 AM Chamikara Jayalath <chamik...@google.com> >> wrote: >> >>> Just releasing Python can break multi-lang by default (unless expansion >>> service is overridden manually) since we match versions across languages >>> when picking the default expansion service. >>> >>> >>> https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L42 >>> >> >> Correct link: >> https://github.com/apache/beam/blob/2f8854a3e34f31c1cc034f95ad36f317abc906ff/sdks/python/apache_beam/utils/subprocess_server.py#L352 >> >> >>> >>> >>> Thanks, >>> Cham >>> >>> On Thu, Mar 28, 2024 at 8:26 AM Danny McCormick via dev < >>> dev@beam.apache.org> wrote: >>> >>>> > The patch itself [1] is trivial, however, the release process is not >>>> trivial. There is little documentation nor practice for a patch release >>>> process. I could imagine two options >>>> >>>> I think there's not a ton of documentation because we haven't done it, >>>> but all the release workflows were authored in such a way that they should >>>> "just work", outside of cutting the release branch itself. So the workflow >>>> should be almost identical to the existing one, but with several steps >>>> skipped (cherry picks, beam website, most validation). Notably, this >>>> shouldn't be any easier/harder if we're doing it for one language or all 3. >>>> >>>> I can take that on if needed. >>>> >>>> > Besides, there should be a Beam YAML validation workflow and added in >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>>> >>>> > If we do a patch release for Python SDK, let's also patch another >>>> known issue for which fix is available: >>>> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >>>> >>>> +1 to both of these >>>> >>>> On Thu, Mar 28, 2024 at 11:25 AM Yi Hu via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>>> Thanks Valentyn for raising this. In this case, Python containers will >>>>> also be included. Different from PyPI wheels, docker tag can override so >>>>> it >>>>> can stay with 2.55.0 >>>>> >>>>> On Thu, Mar 28, 2024 at 11:15 AM Valentyn Tymofieiev < >>>>> valen...@google.com> wrote: >>>>> >>>>>> If we do a patch release for Python SDK, let's also patch another >>>>>> known issue for which fix is available: >>>>>> https://github.com/apache/beam/blob/master/CHANGES.md#known-issues-1 >>>>>> >>>>>> On Thu, Mar 28, 2024 at 8:01 AM Yi Hu via dev <dev@beam.apache.org> >>>>>> wrote: >>>>>> >>>>>>> 2.55.0 release manager here >>>>>>> >>>>>>> The patch itself [1] is trivial, however, the release process is not >>>>>>> trivial. There is little documentation nor practice for a patch release >>>>>>> process. I could imagine two options >>>>>>> >>>>>>> 1. Do a full "2.55.1" release >>>>>>> >>>>>>> 2. Do a patch release only for Python SDK, that is >>>>>>> a. cherry-pick [1] into release-2.55.0 branch >>>>>>> b. tag a 2.55.1rc1 release candidate - note that the source code >>>>>>> of release candidate (e.g. apache_beam/version.py) still reads 2.55.0. >>>>>>> This >>>>>>> ensures Python SDK picks up the Java expansion service / job server of >>>>>>> existing version (2.55.0). We did it once for Go SDK ( >>>>>>> https://github.com/apache/beam/tree/sdks/v2.48.2) >>>>>>> c. Build the release candidate for Python wheels (also Python >>>>>>> containers? Not sure if it is needed) >>>>>>> d. send out the RC for validation >>>>>>> e. finalize the release >>>>>>> >>>>>>> If we decided to do a patch release I would prefer option 2. I can >>>>>>> take on that if decided to do. However, if we decide do a full release >>>>>>> (or >>>>>>> both Java and Python) I would suggest defer to next release cycle, as >>>>>>> the >>>>>>> release process itself could take ~10 days minimum if there is single >>>>>>> RC. >>>>>>> >>>>>>> Besides, there should be a Beam YAML validation workflow and added >>>>>>> in >>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1368030253 >>>>>>> >>>>>>> >>>>>>> [1] https://github.com/apache/beam/pull/30780 >>>>>>> >>>>>>> On Thu, Mar 28, 2024 at 10:22 AM Danny McCormick via dev < >>>>>>> dev@beam.apache.org> wrote: >>>>>>> >>>>>>>> +1 on a patch release - we've done a fair amount of work to make >>>>>>>> releasing easier, and one of my hopes is that it will enable quick >>>>>>>> patches >>>>>>>> like this. I'd vote we try to fix the underlying Java piece as well, >>>>>>>> though, doing a patch release for one language shouldn't be >>>>>>>> significantly >>>>>>>> cheaper than doing it for multiple languages. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Danny >>>>>>>> >>>>>>>> On Wed, Mar 27, 2024 at 7:19 PM Robert Burke <rob...@frantil.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> +1 to a targeted patch release. >>>>>>>>> >>>>>>>>> We did the same for the Go SDK a little while back. It would be >>>>>>>>> good to see what's different for a different SDK. >>>>>>>>> >>>>>>>>> On Wed, Mar 27, 2024, 4:01 PM Robert Bradshaw via dev < >>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>> >>>>>>>>>> Given the severity of the breakage, and the simplicity of the >>>>>>>>>> workaround, I'm in favor of a patch release. I think we could do >>>>>>>>>> Python-only, which would make the process even more lightweight. >>>>>>>>>> >>>>>>>>>> On Wed, Mar 27, 2024 at 3:48 PM Jeff Kinard <j...@thekinards.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Beam 2.55 was released with a bug that causes WriteToJson on >>>>>>>>>>> Beam YAML to fail when using the Java variant. This also affects >>>>>>>>>>> any user >>>>>>>>>>> attempting to use the Xlang JsonWriteTransformProvider - >>>>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/src/main/java/org/apache/beam/sdk/io/json/providers/JsonWriteTransformProvider.java >>>>>>>>>>> >>>>>>>>>>> This is due to a change to >>>>>>>>>>> https://github.com/apache/beam/blob/master/sdks/java/io/json/build.gradle >>>>>>>>>>> that removed >>>>>>>>>>> a dependency on everit which also removed it from being packaged >>>>>>>>>>> into the expansion service JAR: >>>>>>>>>>> beam-sdks-java-extensions-sql-expansion-service-2.55.0.jar >>>>>>>>>>> >>>>>>>>>>> There is a temporary fix to disable the provider in Beam YAML: >>>>>>>>>>> https://github.com/apache/beam/pull/30777 >>>>>>>>>>> >>>>>>>>>>> I think with the total loss of function, and a trivial fix, it >>>>>>>>>>> is worth creating a patch release of Beam 2.55 to include this fix. >>>>>>>>>>> >>>>>>>>>>> - Jeff >>>>>>>>>>> >>>>>>>>>>>
