This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/google.golang.org/api-0.259.0
in repository https://gitbox.apache.org/repos/asf/beam.git
omit dec53d04247 Bump google.golang.org/api from 0.257.0 to 0.259.0 in /sdks
add 98999f5455d Update CONTRIBUTING.md (#37187)
add 881bbaa0aac
DynamicDestinationsHelper.ConstantTimePartitioningClusteringDestinations is
parsing per element json configuration for partitioning and clustering which is
expensive. Cache the outcome of evaluation so it's done once. (#37014)
add 511036ad997 Revise CHANGES.md for upcoming release (#37243)
add bfff80b6fa4 [Dataflow Streaming] Add WindmillTagEncodingV2. (#37151)
add be924b74637 Update BeamModulePlugin.groovy (#37233)
add 6bedec369f7 Bump cloud.google.com/go/bigtable from 1.40.1 to 1.41.0 in
/sdks (#37120)
add 7694450ecc2 Fix CONNECTION_STRING format in tox.ini (#37250)
add d7fa511ee8a [yaml] : add more error handling tests and website example
(#37245)
add ef0a03c90b2 Flink 2.0 support (#37175)
add 139724d907c Map TIMSETAMP(12) BQ type -> timestamp-nanos Avro type in
default schemafactory (#37257)
add 815041e0a2d Support for RateLimiter in Beam Remote Model Handler
(#37218)
add 21c0842a322 Add note about providers to yaml jinja section.
add 38c88a8df5f Merge pull request #37263 Add note about providers to yaml
jinja section.
add 8801afdbd32 Update to upstream caller (#37270)
add 1575b298cb8 [Docs] Fix grammar: 'should triggered' -> 'should be
triggered' (#37278)
add c481b90f47a Update Python Dependencies (#37280)
add 990276f7271 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks
(#37282)
add 335e96b5073 [yaml] : Add final expand pipeline ut (#37260)
add 072dd821095 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#37283)
add d0223389f47 update RateLimiter execution function name (#37287)
add be8dfed0530 Bump github.com/aws/aws-sdk-go-v2/config from 1.32.6 to
1.32.7 in /sdks (#37292)
add 697d2992aeb feat: Add support for custom prediction routes in Vertex
AI inference (#37155)
add e5cac6c73cb Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#37291)
add 3783f5891db Add step to free disk space in workflow (#37246)
add fdd25a4eeb6 Update trivial inference for Python 3.14 (#37248)
add 13303461281 Bump Python dev container version (#37293)
add 752da38e076 Catch breaking import error (#37295)
add 4dcc27d548e Update users.yml - fix bad role label (#37192)
add aad58648737 Support picosecond tiemstamps when writing GenericRecord
and Beam Rows. (#37294)
add 331b08eb940 Add new features to changes.md (#37308)
add daa0f15359c Bump golang.org/x/text from 0.32.0 to 0.33.0 in /sdks
(#37301)
add d8b08408986 Bump google.golang.org/api from 0.257.0 to 0.259.0 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (dec53d04247)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/google.golang.org/api-0.259.0
(d8b08408986)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/build.gradle | 4 +-
.github/trigger_files/beam_PostCommit_Python.json | 5 +-
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 2 +
CHANGES.md | 42 +-
CONTRIBUTING.md | 7 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 5 +-
gradle.properties | 2 +-
infra/iam/users.yml | 3 +-
.../apache/beam/runners/core/StateNamespaces.java | 8 +
.../org/apache/beam/runners/core/StateTags.java | 17 +
runners/flink/2.0/build.gradle | 43 ++
.../job-server-container/build.gradle | 0
.../flink/{1.19 => 2.0}/job-server/build.gradle | 2 +-
.../flink/FlinkBatchTranslationContext.java | 58 +--
.../runners/flink/FlinkExecutionEnvironments.java | 189 +++----
.../runners/flink/FlinkMiniClusterEntryPoint.java | 4 +-
.../flink/FlinkPipelineExecutionEnvironment.java | 89 +---
.../beam/runners/flink/FlinkPipelineOptions.java | 49 +-
.../beam/runners/flink/FlinkPipelineRunner.java | 11 +-
.../runners/flink/FlinkPipelineTranslator.java | 10 +-
.../FlinkStreamingPortablePipelineTranslator.java | 4 +-
.../flink/FlinkStreamingTransformTranslators.java | 17 +-
.../runners/flink/FlinkTransformOverrides.java | 10 +-
.../translation/functions/FlinkDoFnFunction.java | 4 +-
.../FlinkExecutableStageContextFactory.java | 4 +-
.../functions/FlinkExecutableStageFunction.java | 4 +-
.../FlinkExecutableStagePruningFunction.java | 4 +-
.../FlinkMergingNonShuffleReduceFunction.java | 4 +-
.../functions/FlinkMultiOutputPruningFunction.java | 4 +-
.../functions/FlinkPartialReduceFunction.java | 4 +-
.../translation/functions/FlinkReduceFunction.java | 4 +-
.../functions/FlinkStatefulDoFnFunction.java | 4 +-
.../functions/ImpulseSourceFunction.java | 2 +-
.../translation/types/CoderTypeInformation.java | 3 +-
.../translation/types/EncodedValueSerializer.java | 13 +-
.../types/EncodedValueTypeInformation.java | 3 +-
.../types/UnversionedTypeSerializerSnapshot.java | 3 +-
.../wrappers/streaming/DoFnOperator.java | 4 +-
.../streaming/io/StreamingImpulseSource.java | 10 +-
.../wrappers/streaming/io/TestStreamSource.java | 2 +-
.../streaming/io/UnboundedSourceWrapper.java | 23 +-
.../state/FlinkBroadcastStateInternals.java | 4 +-
.../streaming/state/FlinkStateInternals.java | 2 +-
.../runners/flink/EncodedValueComparatorTest.java | 3 +-
.../flink/FlinkExecutionEnvironmentsTest.java | 60 ++-
.../FlinkPipelineExecutionEnvironmentTest.java | 44 +-
.../runners/flink/FlinkPipelineOptionsTest.java | 9 +-
.../flink/FlinkRequiresStableInputTest.java | 10 +-
.../apache/beam/runners/flink/FlinkRunnerTest.java | 9 +-
.../beam/runners/flink/FlinkSavepointTest.java | 9 +-
.../beam/runners/flink/FlinkSubmissionTest.java | 42 +-
.../apache/beam/runners/flink/ReadSourceTest.java | 17 +-
.../adapter/BeamFlinkDataStreamAdapterTest.java | 6 +-
.../flink/streaming/BoundedSourceRestoreTest.java | 8 +-
.../flink/streaming/FlinkStateInternalsTest.java | 4 +-
.../flink/streaming/MemoryStateBackendWrapper.java | 10 +-
.../runners/flink/streaming/StreamSources.java | 2 +-
.../functions/FlinkDoFnFunctionTest.java | 4 +-
.../FlinkExecutableStageFunctionTest.java | 12 +-
.../functions/FlinkStatefulDoFnFunctionTest.java | 4 +-
.../functions/ImpulseSourceFunctionTest.java | 2 +-
.../streaming/io/UnboundedSourceWrapperTest.java | 72 ++-
.../stableinput/BufferingDoFnRunnerTest.java | 6 +-
.../2.0/src/test/resources/flink-test-config.yaml | 16 +-
runners/flink/flink_runner.gradle | 16 +-
.../flink/FlinkBatchPipelineTranslator.java | 2 +-
.../beam/runners/flink/FlinkPipelineOptions.java | 8 +-
.../flink/FlinkStreamingTransformTranslators.java | 2 +-
.../flink/metrics/FlinkMetricContainerBase.java | 4 +-
.../streaming/io/UnboundedSourceWrapper.java | 2 +-
.../worker/StreamingModeExecutionContext.java | 12 +-
.../dataflow/worker/WindmillTimerInternals.java | 6 +-
.../worker/windmill/state/WindmillTagEncoding.java | 5 +-
.../windmill/state/WindmillTagEncodingV1.java | 3 +-
.../windmill/state/WindmillTagEncodingV2.java | 406 +++++++++++++++
.../processing/ComputationWorkExecutorFactory.java | 14 +-
.../worker/StreamingModeExecutionContextTest.java | 3 +-
.../dataflow/worker/WorkerCustomSourcesTest.java | 6 +-
.../windmill/state/WindmillTagEncodingV2Test.java | 576 +++++++++++++++++++++
sdks/go.mod | 46 +-
sdks/go.sum | 92 ++--
.../license_scripts/pull_licenses_java.py | 14 +-
.../AvroGenericRecordToStorageApiProto.java | 3 +
.../io/gcp/bigquery/BeamRowToStorageApiProto.java | 3 +
.../sdk/io/gcp/bigquery/BigQueryAvroUtils.java | 12 +-
.../gcp/bigquery/DynamicDestinationsHelpers.java | 47 +-
.../AvroGenericRecordToStorageApiProtoTest.java | 34 ++
.../gcp/bigquery/BeamRowToStorageApiProtoTest.java | 17 +
.../io/gcp/bigquery/BigQueryTimestampPicosIT.java | 23 +-
.../examples/inference/rate_limiter_vertex_ai.py | 85 +++
.../apache_beam/examples/rate_limiter_simple.py | 2 +-
.../apache_beam/io/components/rate_limiter.py | 44 +-
.../apache_beam/io/components/rate_limiter_test.py | 24 +-
sdks/python/apache_beam/ml/inference/base.py | 30 +-
sdks/python/apache_beam/ml/inference/base_test.py | 61 +++
.../vertex_ai_custom_prediction/Dockerfile | 10 +-
.../vertex_ai_custom_prediction/README.md | 103 ++++
.../vertex_ai_custom_prediction/echo_server.py} | 28 +-
.../ml/inference/vertex_ai_inference.py | 72 ++-
.../ml/inference/vertex_ai_inference_it_test.py | 47 ++
.../ml/inference/vertex_ai_inference_test.py | 70 +++
.../apache_beam/runners/dataflow/internal/names.py | 2 +-
.../typehints/native_type_compatibility.py | 2 +-
.../typehints/native_type_compatibility_test.py | 2 +-
sdks/python/apache_beam/typehints/opcodes.py | 24 +-
.../apache_beam/typehints/trivial_inference.py | 27 +-
sdks/python/apache_beam/yaml/yaml_ml.py | 9 +
.../python/apache_beam/yaml/yaml_transform_test.py | 155 ++++++
.../apache_beam/yaml/yaml_transform_unit_test.py | 29 ++
.../container/ml/py310/base_image_requirements.txt | 41 +-
.../container/ml/py310/gpu_image_requirements.txt | 51 +-
.../container/ml/py311/base_image_requirements.txt | 45 +-
.../container/ml/py311/gpu_image_requirements.txt | 53 +-
.../container/ml/py312/base_image_requirements.txt | 45 +-
.../container/ml/py312/gpu_image_requirements.txt | 53 +-
.../container/ml/py313/base_image_requirements.txt | 49 +-
.../container/py310/base_image_requirements.txt | 33 +-
.../container/py311/base_image_requirements.txt | 37 +-
.../container/py312/base_image_requirements.txt | 35 +-
.../container/py313/base_image_requirements.txt | 41 +-
sdks/python/tox.ini | 2 +-
settings.gradle.kts | 12 +-
.../content/en/documentation/sdks/yaml-errors.md | 58 +++
.../www/site/content/en/documentation/sdks/yaml.md | 2 +
.../shortcodes/flink_java_pipeline_options.html | 19 +-
.../shortcodes/flink_python_pipeline_options.html | 19 +-
126 files changed, 2830 insertions(+), 982 deletions(-)
create mode 100644 runners/flink/2.0/build.gradle
copy runners/flink/{1.17 => 2.0}/job-server-container/build.gradle (100%)
copy runners/flink/{1.19 => 2.0}/job-server/build.gradle (95%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkBatchTranslationContext.java
(69%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkExecutionEnvironments.java
(78%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkMiniClusterEntryPoint.java
(95%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkPipelineExecutionEnvironment.java
(69%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkPipelineOptions.java (90%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkPipelineRunner.java (94%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkPipelineTranslator.java
(77%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkStreamingPortablePipelineTranslator.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkStreamingTransformTranslators.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/FlinkTransformOverrides.java
(90%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkDoFnFunction.java
(98%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkExecutableStageContextFactory.java
(94%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkExecutableStageFunction.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkExecutableStagePruningFunction.java
(95%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkMergingNonShuffleReduceFunction.java
(97%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkMultiOutputPruningFunction.java
(96%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkPartialReduceFunction.java
(97%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkReduceFunction.java
(97%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkStatefulDoFnFunction.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/functions/ImpulseSourceFunction.java
(98%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/types/CoderTypeInformation.java
(96%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/types/EncodedValueSerializer.java
(89%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/types/EncodedValueTypeInformation.java
(94%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/types/UnversionedTypeSerializerSnapshot.java
(98%)
copy runners/flink/{1.20 =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/DoFnOperator.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/StreamingImpulseSource.java
(86%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/TestStreamSource.java
(97%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedSourceWrapper.java
(95%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/state/FlinkBroadcastStateInternals.java
(99%)
copy runners/flink/{ =>
2.0}/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/state/FlinkStateInternals.java
(99%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/EncodedValueComparatorTest.java
(96%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkExecutionEnvironmentsTest.java
(91%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkPipelineExecutionEnvironmentTest.java
(90%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkPipelineOptionsTest.java
(95%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkRequiresStableInputTest.java
(97%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkRunnerTest.java (92%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkSavepointTest.java (98%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/FlinkSubmissionTest.java (89%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/ReadSourceTest.java (81%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/adapter/BeamFlinkDataStreamAdapterTest.java
(96%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/streaming/BoundedSourceRestoreTest.java
(97%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/streaming/FlinkStateInternalsTest.java
(98%)
copy runners/flink/{1.19 =>
2.0}/src/test/java/org/apache/beam/runners/flink/streaming/MemoryStateBackendWrapper.java
(91%)
copy runners/flink/{1.19 =>
2.0}/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
(97%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/functions/FlinkDoFnFunctionTest.java
(96%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/functions/FlinkExecutableStageFunctionTest.java
(97%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/functions/FlinkStatefulDoFnFunctionTest.java
(96%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/functions/ImpulseSourceFunctionTest.java
(99%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/UnboundedSourceWrapperTest.java
(93%)
copy runners/flink/{ =>
2.0}/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunnerTest.java
(97%)
copy learning/katas/go/common_transforms/aggregation/lesson-info.yaml =>
runners/flink/2.0/src/test/resources/flink-test-config.yaml (85%)
create mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillTagEncodingV2.java
create mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/state/WindmillTagEncodingV2Test.java
create mode 100644
sdks/python/apache_beam/examples/inference/rate_limiter_vertex_ai.py
copy .test-infra/kubernetes/cassandra/LargeITCluster/setup.sh =>
sdks/python/apache_beam/ml/inference/test_resources/vertex_ai_custom_prediction/Dockerfile
(84%)
create mode 100644
sdks/python/apache_beam/ml/inference/test_resources/vertex_ai_custom_prediction/README.md
copy sdks/python/apache_beam/{examples/wordcount_dataframe.py =>
ml/inference/test_resources/vertex_ai_custom_prediction/echo_server.py} (63%)
