Awesome!! On Fri, Jan 5, 2024 at 2:11 PM XQ Hu via user <[email protected]> wrote:
> Great! And thank you! > > On Fri, Jan 5, 2024 at 2:49 PM Jack McCluskey via user < > [email protected]> wrote: > >> We are happy to present the new 2.53.0 release of Beam. >> This release includes both improvements and new functionality. >> For more information on changes in 2.53.0, check out the detailed release >> notes (https://github.com/apache/beam/milestone/17). >> >> Highlights >> * Python streaming users that use 2.47.0 and newer versions of Beam >> should update to version 2.53.0, which fixes a known issue ( >> https://github.com/apache/beam/issues/27330) >> >> I/Os >> * TextIO now supports skipping multiple header lines (Java)( >> https://github.com/apache/beam/issues/17990) >> * Python GCSIO is now implemented with GCP GCS Client instead of apitools >> (https://github.com/apache/beam/issues/25676) >> * Adding support for LowCardinality DataType in ClickHouse (Java) ( >> https://github.com/apache/beam/pull/29533) >> * Added support for handling bad records to KafkaIO (Java) ( >> https://github.com/apache/beam/pull/29546) >> * Add support for generating text embeddings in MLTransform for Vertex AI >> and Hugging Face Hub models. (https://github.com/apache/beam/pull/29564) >> * NATS IO connector added (Go) ( >> https://github.com/apache/beam/issues/29000) >> >> New Features / Improvements >> * The Python SDK now type checks `collections.abc.Collections` types >> properly. Some type hints that were erroneously allowed by the SDK may now >> fail (https://github.com/apache/beam/pull/29272) >> * Running multi-language pipelines locally no longer requires Docker. >> Instead, the same (generally auto-started) subprocess used to perform >> the >> expansion can also be used as the cross-language worker. >> * Framework for adding Error Handlers to composite transforms added in >> Java (https://github.com/apache/beam/pull/29164) >> * Python 3.11 images now include google-cloud-profiler ( >> https://github.com/apache/beam/pull/29651) >> >> Breaking Changes >> * Upgraded to go 1.21.5 to build, fixing CVE-2023-45285 ( >> https://security-tracker.debian.org/tracker/CVE-2023-45285) and >> CVE-2023-39326 ( >> https://security-tracker.debian.org/tracker/CVE-2023-39326) >> >> Deprecations >> * Euphoria DSL is deprecated and will be removed in a future release (not >> before 2.56.0) (https://github.com/apache/beam/issues/29451) >> >> Bugfixes >> * (Python) Fixed sporadic crashes in streaming pipelines that affected >> some users of 2.47.0 and newer SDKs ( >> https://github.com/apache/beam/issues/27330) >> * (Python) Fixed a bug that caused MLTransform to drop identical elements >> in the output PCollection (https://github.com/apache/beam/issues/29600) >> >> Thanks, >> >> Jack McCluskey >> -- >> >> >> Jack McCluskey >> SWE - DataPLS PLAT/ Dataflow ML >> RDU >> [email protected] >> >> >>
