This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch users/damccorm/websiteUpdate in repository https://gitbox.apache.org/repos/asf/beam.git
commit 24ba3059bdd97c60ee6e456d7413eb223c0a08d3 Author: Danny McCormick <dannymccorm...@google.com> AuthorDate: Tue Nov 7 09:45:01 2023 -0500 Website updates for the 2.52.0 release --- CHANGES.md | 2 +- website/www/site/config.toml | 2 +- website/www/site/content/en/blog/beam-2.52.0.md | 135 +++++++++++++++++++++ .../www/site/content/en/get-started/downloads.md | 14 ++- 4 files changed, 148 insertions(+), 5 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index fa6e51cde2e..4f5fab8bd99 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -53,7 +53,7 @@ * ([#X](https://github.com/apache/beam/issues/X)). --> -# [2.52.0] - Unreleased +# [2.52.0] - TODO - update to release date ## Highlights diff --git a/website/www/site/config.toml b/website/www/site/config.toml index c7b0cd3412e..e1b9fd612ed 100644 --- a/website/www/site/config.toml +++ b/website/www/site/config.toml @@ -104,7 +104,7 @@ github_project_repo = "https://github.com/apache/beam" [params] description = "Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow pipelines simplify the mechanics of large-scale batch and streaming data processing and can run on a number of runtimes like Apache Flink, Apache Spark, and Google Cloud Dataflow (a cloud service). Be [...] -release_latest = "2.51.0" +release_latest = "2.52.0" # The repository and branch where the files live in Github or Colab. This is used # to serve and stage from your local branch, but publish to the master branch. # e.g. https://github.com/{{< param branch_repo >}}/path/to/notebook.ipynb diff --git a/website/www/site/content/en/blog/beam-2.52.0.md b/website/www/site/content/en/blog/beam-2.52.0.md new file mode 100644 index 00000000000..9b427fce233 --- /dev/null +++ b/website/www/site/content/en/blog/beam-2.52.0.md @@ -0,0 +1,135 @@ +--- +title: "Apache Beam 2.52.0" +date: YYYY-MM-DD H:MM:00 Z - TODO update this when released +categories: + - blog + - release +authors: + - damccorm +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at +http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +We are happy to present the new 2.52.0 release of Beam. +This release includes both improvements and new functionality. +See the [download page](/get-started/downloads/#2520-2023-11-XX) for this release. - TODO update this anchor when released + +<!--more--> + +For more information on changes in 2.52.0, check out the [detailed release notes](https://github.com/apache/beam/milestone/16). + +## Highlights + +* Previously deprecated Avro-dependent code (Beam Release 2.46.0) has been finally removed from Java SDK "core" package. +Please, use `beam-sdks-java-extensions-avro` instead. This will allow to easily update Avro version in user code without +potential breaking changes in Beam "core" since the Beam Avro extension already supports the latest Avro versions and +should handle this. ([#25252](https://github.com/apache/beam/issues/25252)). +* Publishing Java 21 SDK container images now supported as part of Apache Beam release process. ([#28120](https://github.com/apache/beam/issues/28120)) + * Direct Runner and Dataflow Runner support running pipelines on Java21 (experimental until tests fully setup). For other runners (Flink, Spark, Samza, etc) support status depend on runner projects. + +## New Features / Improvements + +* Add `UseDataStreamForBatch` pipeline option to the Flink runner. When it is set to true, Flink runner will run batch + jobs using the DataStream API. By default the option is set to false, so the batch jobs are still executed + using the DataSet API. +* `upload_graph` as one of the Experiments options for DataflowRunner is no longer required when the graph is larger than 10MB for Java SDK ([PR#28621](https://github.com/apache/beam/pull/28621). +* state amd side input cache has been enabled to a default of 100 MB. Use `--max_cache_memory_usage_mb=X` to provide cache size for the user state API and side inputs. (Python) ([#28770](https://github.com/apache/beam/issues/28770)). +* Beam YAML stable release. Beam pipelines can now be written using YAML and leverage the Beam YAML framework which includes a preliminary set of IO's and turnkey transforms. More information can be found in the YAML root folder and in the [README](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/yaml/README.md). + +## Breaking Changes + +* `org.apache.beam.sdk.io.CountingSource.CounterMark` uses custom `CounterMarkCoder` as a default coder since all Avro-dependent +classes finally moved to `extensions/avro`. In case if it's still required to use `AvroCoder` for `CounterMark`, then, +as a workaround, a copy of "old" `CountingSource` class should be placed into a project code and used directly +([#25252](https://github.com/apache/beam/issues/25252)). +* Renamed `host` to `firestoreHost` in `FirestoreOptions` to avoid potential conflict of command line arguments (Java) ([#29201](https://github.com/apache/beam/pull/29201)). + +## Bugfixes + +* Fixed "Desired bundle size 0 bytes must be greater than 0" in Java SDK's BigtableIO.BigtableSource when you have more cores than bytes to read (Java) [#28793](https://github.com/apache/beam/issues/28793). +* `watch_file_pattern` arg of the [RunInference](https://github.com/apache/beam/blob/104c10b3ee536a9a3ea52b4dbf62d86b669da5d9/sdks/python/apache_beam/ml/inference/base.py#L997) arg had no effect prior to 2.52.0. To use the behavior of arg `watch_file_pattern` prior to 2.52.0, follow the documentation at https://beam.apache.org/documentation/ml/side-input-updates/ and use `WatchFilePattern` PTransform as a SideInput. ([#28948](https://github.com/apache/beam/pulls/28948)) +* `MLTransform` doesn't output artifacts such as min, max and quantiles. Instead, `MLTransform` will add a feature to output these artifacts as human readable format - [#29017](https://github.com/apache/beam/issues/29017). For now, to use the artifacts such as min and max that were produced by the eariler `MLTransform`, use `read_artifact_location` of `MLTransform`, which reads artifacts that were produced earlier in a different `MLTransform` ([#29016](https://github.com/apache/beam/pull [...] + +## Security Fixes +* Fixed [CVE-2023-39325](https://www.cve.org/CVERecord?id=CVE-2023-39325) (Java/Python/Go) ([#29118](https://github.com/apache/beam/issues/29118)). + +## List of Contributors + +According to git shortlog, the following people contributed to the 2.52.0 release. Thank you to all contributors! + +Ahmed Abualsaud +Ahmet Altay +Aleksandr Dudko +Alexey Romanenko +Anand Inguva +Andrei Gurau +Andrey Devyatkin +BjornPrime +Bruno Volpato +Bulat +Chamikara Jayalath +Damon +Danny McCormick +Devansh Modi +Dominik Dębowczyk +Ferran Fernández Garrido +Hai Joey Tran +Israel Herraiz +Jack McCluskey +Jan Lukavský +JayajP +Jeff Kinard +Jeffrey Kinard +Jiangjie Qin +Jing +Joar Wandborg +Johanna Öjeling +Julien Tournay +Kanishk Karanawat +Kenneth Knowles +Kerry Donny-Clark +Luís Bianchin +Minbo Bae +Pranav Bhandari +Rebecca Szper +Reuven Lax +Ritesh Ghorse +Robert Bradshaw +Robert Burke +RyuSA +Shunping Huang +Steven van Rossum +Svetak Sundhar +Tony Tang +Vitaly Terentyev +Vivek Sumanth +Vlado Djerek +Yi Hu +aku019 +brucearctor +caneff +damccorm +ddebowczyk92 +dependabot[bot] +dpcollins-google +edman124 +gabry.wu +illoise +johnjcasey +jonathan-lemos +kennknowles +liferoad +magicgoody +martin trieu +nancyxu123 +pablo rodriguez defino +tvalentyn \ No newline at end of file diff --git a/website/www/site/content/en/get-started/downloads.md b/website/www/site/content/en/get-started/downloads.md index cc71f3101eb..d7bd150ae97 100644 --- a/website/www/site/content/en/get-started/downloads.md +++ b/website/www/site/content/en/get-started/downloads.md @@ -96,10 +96,18 @@ versions denoted `0.x.y`. ## Releases +### 2.52.0 (2023-10-11) +Official [source code download](https://downloads.apache.org/beam/2.52.0/apache-beam-2.52.0-source-release.zip). +[SHA-512](https://downloads.apache.org/beam/2.52.0/apache-beam-2.52.0-source-release.zip.sha512). +[signature](https://downloads.apache.org/beam/2.52.0/apache-beam-2.52.0-source-release.zip.asc). + +[Release notes](https://github.com/apache/beam/releases/tag/v2.52.0) +[Blog post](/blog/beam-2.52.0). + ### 2.51.0 (2023-10-11) -Official [source code download](https://downloads.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip). -[SHA-512](https://downloads.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip.sha512). -[signature](https://downloads.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip.asc). +Official [source code download](https://archive.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip). +[SHA-512](https://archive.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip.sha512). +[signature](https://archive.apache.org/beam/2.51.0/apache-beam-2.51.0-source-release.zip.asc). [Release notes](https://github.com/apache/beam/releases/tag/v2.51.0) [Blog post](/blog/beam-2.51.0).