This is an automated email from the ASF dual-hosted git repository.
Amar3tto pushed a change to branch website-2-75
in repository https://gitbox.apache.org/repos/asf/beam.git
discard c2699ad9b65 Update release date
discard 80d0f450e97 Update Beam website to release 2.75.0
add f08c259a02e Bump google.golang.org/api from 0.286.0 to 0.287.0 in
/sdks (#39185)
add e87f50e3a9a Fix Python VarIntCoder OverflowError on uint64 values
(#39047)
add d0d754c635d Acknowledge messages asap in pubsub test (#39183)
add 8c0f0e02a82 Update Python Dependencies (#39139)
add 2f7d95d1cf9 [Gemini] Add batching support to Java Remote Inference
(#39067)
add 5f3fc19127e [Gemini] Add retry filter support to the Java Remote
RetryHandler (#39004)
add dac2848f4ae Upgrade TFX stack to 1.21.x and remove numpy/pandas bounds
add c2a62c12f94 Append timestamp to artifact_location and output_vocab
add 11cb59ed70c Upgrade tfx stack. Remove some upper bounds.
add 781ec5cdecf Consolidate the test requirements file to examples.
add 1567148726c Run one-hot test only
add b597f364dbd Remove ml_extra in one-hot load test
add 54783520416 Revert "Run one-hot test only"
add 1876512bb88 Minor edit on requirements.txt
add bc9937ade50 Fix typo
add 4a976d393c9 Merge pull request #39163 from apache/resolution-too-deep
add 00823b29fa4 Migrate MqttIO to HiveMQ Mqtt client (#39180)
add 8610459f7b8 [Gemini] Port client-side throttling to the Java SDK
(#39021)
add ca66eda1bc1 delta lake python/yaml (#39052)
add eed2511dcce Add "-Pjava21Home" to stage docker step (#39181)
add 7b3e2df2ed6 Bump transformers (#39195)
add c38839082de Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks
(#39199)
add 22613182291 Bump docker/build-push-action from 7.2.0 to 7.3.0 (#39198)
add db9f9d56022 Bump docker/setup-qemu-action from 4.1.0 to 4.2.0 (#39197)
add 70329063989 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#39200)
add a2a14f511e9 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#39201)
add 95e783d0eef Install wget
add fff406dd6c1 Merge pull request #39189 from apache/validate-275-rc1
add 956fe0c90e4 [Flink] Bound SDF self-checkpoint residual state on the
portable runner (#39191)
add b9a506743a1 Bump cryptography from 47.0.0 to 48.0.1 in Python
container images (#39204)
add 371a33a54b4 Bump docker/login-action from 4.2.0 to 4.3.0 (#39208)
add 8d1728e8455 Bump docker/setup-buildx-action from 4.1.0 to 4.2.0
(#39209)
add a3d32977f46 Bump github.com/testcontainers/testcontainers-go from
0.42.0 to 0.43.0 in /sdks (#39053)
add 0ec08474e61 Bump docker/login-action from 4.3.0 to 4.4.0 (#39217)
add 2940f8fc6a0 Added Criteo TFT fill_in_missing helper (#39011)
add 0b407cd78ce Fix lost state updates in Set state (#39175)
add 09d63ccbb36 Bump pyo3 from 0.27.2 to 0.29.0 in
/sdks/python/apache_beam/examples/wordcount_rust/word_processing (#38953)
add 4d975e5ec60 Avoid RequestResponseIO throttler access when throttler is
absent (#39112)
add 904f8898ab2 Fix PostCommit Java DataflowV2 workflow (#39232)
add 4bf5a6afb50 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#39227)
add 4cde7ed933a Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#39229)
add 7502ae2d529 Bump golang.org/x/text from 0.38.0 to 0.39.0 in /sdks
(#39230)
add f35fa826f44 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#39231)
add bdc26675c91 Drop envoy-data-plane/betterproto dependency from Python
SDK (#39213)
add c6e13b60fa4 Revert "Fix Python VarIntCoder OverflowError on uint64
values (#39047)" (#39218)
add 20494736bb9 Add MLTransform embedding benchmarks (#38917)
add d1c1a681d83 [aws2] Make SqsIOWriteBatchesTest timeout tests
timing-independent (#38971)
add 4a82e692e78 remove todos etc with pubsublite deprecation (#39237)
add 954de0d7846 Update Beam website to release 2.75.0
add 5b7064e44a9 Update release date
add 5fd72c328d4 Update CHANGES.md
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c2699ad9b65)
\
N -- N -- N refs/heads/website-2-75 (5fd72c328d4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
...m_PostCommit_Python_Xlang_Messaging_Direct.json | 2 +-
.../beam_Inference_Python_Benchmarks_Dataflow.yml | 43 +++-
.github/workflows/beam_PostCommit_Go.yml | 2 +-
.../workflows/beam_PostCommit_Go_Dataflow_ARM.yml | 2 +-
.../workflows/beam_PostCommit_Java_DataflowV2.yml | 4 +-
.../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 2 +-
.github/workflows/beam_PostCommit_Python_Arm.yml | 2 +-
.../beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 2 +-
.../beam_PostCommit_Yaml_Xlang_Direct.yml | 6 +-
.../workflows/beam_PreCommit_CommunityMetrics.yml | 2 +-
.github/workflows/beam_PreCommit_PythonDocker.yml | 2 +-
.../beam_PreCommit_Xlang_Generated_Transforms.yml | 6 +-
.../workflows/beam_PreCommit_Yaml_Xlang_Direct.yml | 6 +-
.../workflows/beam_Publish_Beam_SDK_Snapshots.yml | 2 +-
.../workflows/beam_Publish_Python_VLLM_Image.yml | 2 +-
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 2 +-
.github/workflows/build_release_candidate.yml | 8 +-
.github/workflows/build_runner_image.yml | 6 +-
.github/workflows/build_wheels.yml | 2 +-
.github/workflows/finalize_release.yml | 2 +-
...flow_MLTransform_Image_Embedding_CPU_Batch.txt} | 38 +--
...flow_MLTransform_Image_Embedding_GPU_Batch.txt} | 40 ++--
...Dataflow_MLTransform_One_Hot_Encoding_Batch.txt | 2 +-
..._Dataflow_MLTransform_Text_Embedding_Batch.txt} | 40 ++--
.../republish_released_docker_containers.yml | 2 +-
.../workflows/run_rc_validation_go_wordcount.yml | 2 +
.test-infra/tools/refresh_looker_metrics.py | 5 +-
CHANGES.md | 39 +++
infra/iam/roles/roles_config.yaml | 2 +-
.../FlinkStreamingPortablePipelineTranslator.java | 1 -
.../functions/FlinkExecutableStageFunction.java | 19 +-
.../FlinkStreamingPortablePipelineTranslator.java | 1 -
.../functions/FlinkExecutableStageFunction.java | 19 +-
.../streaming/ExecutableStageDoFnOperator.java | 18 +-
.../control/BundleCheckpointHandlers.java | 24 +-
.../control/BundleCheckpointHandlersTest.java | 125 ++++++++++
sdks/go.mod | 54 ++---
sdks/go.sum | 108 ++++-----
.../go/test/integration/io/xlang/jdbc/jdbc_test.go | 6 +-
.../sdk/io/aws2/sqs/SqsIOWriteBatchesTest.java | 97 ++++++--
.../components/throttling/AdaptiveThrottler.java | 105 ++++++++
.../components/throttling/ReactiveThrottler.java | 78 ++++++
.../throttling/AdaptiveThrottlerTest.java | 107 +++++++++
.../DeltaReadSchemaTransformProviderTest.java | 127 ++++++++++
sdks/java/io/expansion-service/build.gradle | 1 +
sdks/java/io/google-cloud-platform/build.gradle | 1 +
.../sdk/io/gcp/datastore/AdaptiveThrottler.java | 111 ---------
.../beam/sdk/io/gcp/datastore/DatastoreV1.java | 1 +
.../io/gcp/datastore/AdaptiveThrottlerTest.java | 114 ---------
sdks/java/io/mqtt/build.gradle | 3 +-
.../java/org/apache/beam/sdk/io/mqtt/MqttIO.java | 182 ++++++++------
.../org/apache/beam/sdk/io/mqtt/MqttIOTest.java | 204 +++++++++-------
.../sdk/ml/inference/remote/RemoteInference.java | 33 ++-
.../beam/sdk/ml/inference/remote/RetryHandler.java | 27 ++-
.../ml/inference/remote/RemoteInferenceTest.java | 40 +++-
.../sdk/ml/inference/remote/RetryHandlerTest.java | 145 ++++++++++++
sdks/python/.yapfignore | 1 +
.../online_clustering/clustering_pipeline/setup.py | 2 +-
.../mltransform_generate_vocab_requirements.txt | 10 +-
.../ml_transform/mltransform_image_embedding.py | 263 +++++++++++++++++++++
.../mltransform_image_embedding_test.py | 105 ++++++++
.../mltransform_one_hot_encoding_requirements.txt} | 16 +-
.../ml_transform/mltransform_text_embedding.py | 186 +++++++++++++++
.../mltransform_text_embedding_test.py | 94 ++++++++
.../wordcount_rust/word_processing/Cargo.lock | 65 +----
.../wordcount_rust/word_processing/Cargo.toml | 2 +-
.../apache_beam/io/components/rate_limit.proto | 64 +++++
.../apache_beam/io/components/rate_limit_pb2.py | 63 +++++
.../apache_beam/io/components/rate_limiter.py | 39 ++-
.../apache_beam/io/components/rate_limiter_test.py | 62 ++++-
.../apache_beam/io/gcp/pubsub_integration_test.py | 20 +-
sdks/python/apache_beam/io/requestresponse.py | 5 +-
sdks/python/apache_beam/io/requestresponse_test.py | 10 +
.../mltransform_embedding_tests_requirements.txt} | 5 +-
.../apache_beam/runners/worker/bundle_processor.py | 37 ++-
.../benchmarks/cloudml/criteo_tft/criteo.py | 30 ++-
.../benchmarks/cloudml/criteo_tft/criteo_test.py | 93 ++++++++
.../mltransform_image_embedding_benchmark.py | 128 ++++++++++
.../mltransform_one_hot_encoding_benchmark.py | 79 ++-----
.../mltransform_text_embedding_benchmark.py | 117 +++++++++
.../testing/load_tests/dataflow_cost_benchmark.py | 143 ++++++++++-
sdks/python/apache_beam/transforms/external.py | 1 +
sdks/python/apache_beam/transforms/managed.py | 4 +-
.../apache_beam/transforms/userstate_test.py | 69 ++++++
sdks/python/apache_beam/yaml/integration_tests.py | 22 ++
sdks/python/apache_beam/yaml/standard_io.yaml | 2 +
.../python/apache_beam/yaml/tests/delta.yaml | 24 +-
sdks/python/apache_beam/yaml/yaml_io.py | 23 ++
.../container/ml/py310/base_image_requirements.txt | 38 ++-
.../container/ml/py310/gpu_image_requirements.txt | 48 ++--
.../container/ml/py311/base_image_requirements.txt | 38 ++-
.../container/ml/py311/gpu_image_requirements.txt | 48 ++--
.../container/ml/py312/base_image_requirements.txt | 38 ++-
.../container/ml/py312/gpu_image_requirements.txt | 48 ++--
.../container/ml/py313/base_image_requirements.txt | 38 ++-
.../container/py310/base_image_requirements.txt | 36 ++-
.../container/py311/base_image_requirements.txt | 36 ++-
.../container/py312/base_image_requirements.txt | 36 ++-
.../container/py313/base_image_requirements.txt | 36 ++-
.../container/py314/base_image_requirements.txt | 34 ++-
sdks/python/pyproject.toml | 1 +
sdks/python/scripts/run_lint.sh | 1 +
sdks/python/setup.py | 6 +-
sdks/standard_expansion_services.yaml | 1 +
.../dsls/sql/extensions/create-external-table.md | 52 ----
website/www/site/content/en/performance/_index.md | 3 +
.../mltransform-image-embedding-cpu/_index.md | 48 ++++
.../mltransform-image-embedding-gpu/_index.md | 48 ++++
.../_index.md | 24 +-
website/www/site/data/performance.yaml | 49 +++-
110 files changed, 3330 insertions(+), 1187 deletions(-)
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_Table_Row_Inference_Stream.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Image_Embedding_CPU_Batch.txt}
(59%)
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_Table_Row_Inference_Stream.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Image_Embedding_GPU_Batch.txt}
(54%)
copy
.github/workflows/load-tests-pipeline-options/{beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Generate_Vocab_Batch.txt
=>
beam_Inference_Python_Benchmarks_Dataflow_MLTransform_Text_Embedding_Batch.txt}
(62%)
create mode 100644
runners/java-fn-execution/src/test/java/org/apache/beam/runners/fnexecution/control/BundleCheckpointHandlersTest.java
create mode 100644
sdks/java/io/components/src/main/java/org/apache/beam/sdk/io/components/throttling/AdaptiveThrottler.java
create mode 100644
sdks/java/io/components/src/main/java/org/apache/beam/sdk/io/components/throttling/ReactiveThrottler.java
create mode 100644
sdks/java/io/components/src/test/java/org/apache/beam/sdk/io/components/throttling/AdaptiveThrottlerTest.java
create mode 100644
sdks/java/io/delta/src/test/java/org/apache/beam/sdk/io/delta/DeltaReadSchemaTransformProviderTest.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/AdaptiveThrottler.java
delete mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/datastore/AdaptiveThrottlerTest.java
create mode 100644
sdks/java/ml/inference/remote/src/test/java/org/apache/beam/sdk/ml/inference/remote/RetryHandlerTest.java
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_image_embedding.py
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_image_embedding_test.py
rename
sdks/python/apache_beam/{ml/transforms/mltransform_tests_requirements.txt =>
examples/ml_transform/mltransform_one_hot_encoding_requirements.txt} (82%)
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_text_embedding.py
create mode 100644
sdks/python/apache_beam/examples/ml_transform/mltransform_text_embedding_test.py
create mode 100644 sdks/python/apache_beam/io/components/rate_limit.proto
create mode 100644 sdks/python/apache_beam/io/components/rate_limit_pb2.py
copy sdks/python/{.coveragerc =>
apache_beam/ml/transforms/mltransform_embedding_tests_requirements.txt} (91%)
create mode 100644
sdks/python/apache_beam/testing/benchmarks/cloudml/criteo_tft/criteo_test.py
create mode 100644
sdks/python/apache_beam/testing/benchmarks/inference/mltransform_image_embedding_benchmark.py
create mode 100644
sdks/python/apache_beam/testing/benchmarks/inference/mltransform_text_embedding_benchmark.py
copy
.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/kustomization.yaml
=> sdks/python/apache_beam/yaml/tests/delta.yaml (66%)
create mode 100644
website/www/site/content/en/performance/mltransform-image-embedding-cpu/_index.md
create mode 100644
website/www/site/content/en/performance/mltransform-image-embedding-gpu/_index.md
copy website/www/site/content/en/performance/{mltransformonehot =>
mltransform-text-embedding}/_index.md (55%)
