This is an automated email from the ASF dual-hosted git repository.
dependabot[bot] pushed a change to branch
dependabot/pip/sdks/python/container/ml/py312/torch-2.12.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard c037ac3de35 Bump torch from 2.7.1 to 2.12.0 in
/sdks/python/container/ml/py312
add 081c61ee5b4 Fix Xlang IO Direct installGcpTest uv cache lock (#38862)
add 8051efce4c2 Turn to manual build for Go for CodeQL (#38907)
add 1cf3545e5b7 Fix Dataflow legacy worker abort loop thread death issue
(#38894)
add 8e859798a6f Bump cloud.google.com/go/bigtable from 1.48.0 to 1.49.0 in
/sdks (#38913)
add 48fe93cb0eb Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#38916)
add 60621107046 Bump golang.org/x/net from 0.55.0 to 0.56.0 in /sdks
(#38915)
add 51e35e52440 [IcebergIO] Improve TableCache (#38882)
add c297c0dc4ab Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#38912)
add 61a7368bdb7 Use p310_ml_test (#38890)
add 135a5593441 Replace ClassLoadingStrategy with custom loading strategy
(#38906)
add a40685ccc62 Don't preempt snapshot runs (#38904)
add 8ea81a9f289 Bump @grpc/grpc-js from 1.14.3 to 1.14.4 in
/sdks/typescript (#38922)
add 6f30f147462 Adds a new agent SKILL for developing new I/O connectors
(#38910)
add 782953e3422 Add Delta Lake source to the Java Managed API (#38902)
add afbbc31369b Implement Asynchronous wrapper for DoFn in Java SDK
(#38609)
add 8d15c63d551 Updates references to the I/O connector skill (#38923)
add 6a30a3d68de Updates CHANGES.md to mention the Delta Lake source
(#38924)
add 03f4f797953 [SQL] Support positional parameters (#38880)
add 5307c7138af Support MAP type in RowJson and fix datetime parsing with
spaces (#38865)
add 56a41422765 SQL DDL documentation (#37539)
add a88686c1594 Add DoFnRunner::finishKey() method (#38454)
add 705db250d51 [Dataflow Streaming] Activate SourceState Finalizers
before submitting workitem to harness threads (#38921)
add e8249afb1f7 Fix wordcount_rust requirements.txt and documentation
(#38877)
add 64bb91099a9 Exclude testSideInputNotReadyTimer from Spark batch suites
(#38939)
add 7d642a024a4 Add Gemini RunInference example notebook (#38943)
add 712d981ea79 update agent skills table (#38818)
add 6ffee9feebb [Infra] Add beam_viewer and beam_writer roles for GSoC
2026 participant (#38933)
add ebde16e3713 Add instrumentation for memory profiling in Python SDK
(#38853)
add 2f5806158d7 Bump torch from 2.7.1 to 2.12.0 in
/sdks/python/container/ml/py312
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c037ac3de35)
\
N -- N -- N
refs/heads/dependabot/pip/sdks/python/container/ml/py312/torch-2.12.0
(2f5806158d7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agent/skills/README.md | 4 +
.../skills/developing-new-io-connectors/SKILL.md | 347 ++++++++
.agent/skills/io-connectors/SKILL.md | 3 +-
.../IO_Iceberg_Integration_Tests.json | 2 +-
.github/trigger_files/beam_PostCommit_Python.json | 2 +-
.../beam_PostCommit_Python_Xlang_IO_Direct.json | 2 +-
.../beam_PostCommit_Yaml_Xlang_Direct.json | 2 +-
.../beam_PostCommit_Yaml_Xlang_Direct.yml | 2 +-
.../workflows/beam_Publish_Beam_SDK_Snapshots.yml | 5 -
.github/workflows/codeql.yml | 30 +-
CHANGES.md | 8 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 4 +-
.../notebooks/beam-ml/run_inference_gemini.ipynb | 609 ++++++++++++++
infra/iam/users.yml | 8 +-
.../model/pipeline/v1/external_transforms.proto | 2 +
.../org/apache/beam/runners/core/DoFnRunner.java | 12 +
.../runners/core/LateDataDroppingDoFnRunner.java | 6 +
.../apache/beam/runners/core/SimpleDoFnRunner.java | 3 +
.../beam/runners/core/StatefulDoFnRunner.java | 6 +
.../SimplePushbackSideInputDoFnRunnerTest.java | 4 +
.../flink/metrics/DoFnRunnerWithMetricsUpdate.java | 4 +
.../streaming/ExecutableStageDoFnOperator.java | 3 +
.../streaming/stableinput/BufferingDoFnRunner.java | 3 +
.../wrappers/streaming/DoFnOperatorTest.java | 3 +
.../worker/AssignWindowsParDoFnFactory.java | 3 +
.../worker/BatchModeUngroupingParDoFn.java | 3 +
.../CreateIsmShardKeyAndSortKeyDoFnFactory.java | 3 +
.../dataflow/worker/DataflowProcessFnRunner.java | 6 +
.../runners/dataflow/worker/ForwardingParDoFn.java | 6 +
.../dataflow/worker/GroupAlsoByWindowFnRunner.java | 4 +
.../dataflow/worker/GroupAlsoByWindowsParDoFn.java | 6 +
.../worker/PairWithConstantKeyDoFnFactory.java | 3 +
.../dataflow/worker/PartialGroupByKeyParDoFns.java | 6 +
.../ReifyTimestampAndWindowsParDoFnFactory.java | 3 +
.../runners/dataflow/worker/SimpleParDoFn.java | 3 +
.../StreamingKeyedWorkItemSideInputDoFnRunner.java | 6 +
.../StreamingKeyedWorkItemSideInputParDoFn.java | 3 +
.../worker/StreamingModeExecutionContext.java | 2 +-
.../StreamingPCollectionViewWriterParDoFn.java | 3 +
.../worker/StreamingSideInputDoFnRunner.java | 6 +
.../worker/ToIsmRecordForMultimapDoFnFactory.java | 3 +
.../runners/dataflow/worker/ValuesDoFnFactory.java | 3 +
.../WorkerCustomSourceOperationExecutor.java | 2 +-
.../util/common/worker/FlattenOperation.java | 3 +-
.../worker/util/common/worker/MapTaskExecutor.java | 7 +-
.../worker/util/common/worker/Operation.java | 4 +-
.../worker/util/common/worker/ParDoFn.java | 4 +
.../worker/util/common/worker/ParDoOperation.java | 4 +-
.../worker/util/common/worker/ReadOperation.java | 2 +-
.../worker/SimplePartialGroupByKeyParDoFn.java | 5 +
.../worker/util/common/worker/WorkExecutor.java | 2 +-
.../worker/util/common/worker/WriteOperation.java | 3 +-
.../work/processing/StreamingWorkScheduler.java | 5 +-
.../worker/IntrinsicMapTaskExecutorTest.java | 11 +-
.../worker/StreamingSideInputDoFnRunnerTest.java | 4 +
.../util/common/worker/ExecutorTestUtils.java | 2 +-
.../util/common/worker/MapTaskExecutorTest.java | 11 +-
.../util/common/worker/ParDoOperationTest.java | 8 +-
runners/spark/job-server/spark_job_server.gradle | 2 +
runners/spark/spark_runner.gradle | 2 +
.../translation/batch/DoFnRunnerFactory.java | 4 +
.../translation/batch/DoFnRunnerWithMetrics.java | 4 +
.../spark/translation/DoFnRunnerWithMetrics.java | 4 +
.../translation/SparkInputDataProcessorTest.java | 4 +
sdks/go.mod | 30 +-
sdks/go.sum | 70 +-
.../apache/beam/sdk/transforms/AsyncWrapper.java | 779 ++++++++++++++++++
.../java/org/apache/beam/sdk/util/RowJson.java | 65 +-
.../beam/sdk/util/RowJsonValueExtractors.java | 9 +-
.../beam/sdk/transforms/AsyncWrapperTest.java | 878 +++++++++++++++++++++
.../java/org/apache/beam/sdk/util/RowJsonTest.java | 26 +-
.../extensions/protobuf/ProtoByteBuddyUtils.java | 49 +-
.../meta/provider/iceberg/IcebergMetastore.java | 13 +-
.../sql/meta/provider/iceberg/IcebergTable.java | 19 +-
.../meta/provider/iceberg/PubsubToIcebergIT.java | 4 +-
.../extensions/sql/impl/CalciteQueryPlanner.java | 93 ++-
.../{BeamSqlAliasTest => BeamSqlAliasTest.java} | 12 +-
.../extensions/sql/BeamSqlDslParametersTest.java | 78 ++
sdks/java/io/delta/build.gradle | 1 +
.../delta/DeltaReadSchemaTransformProvider.java} | 94 +--
.../org/apache/beam/sdk/io/delta/DeltaIOTest.java | 41 +
.../iceberg/AssignDestinationsAndPartitions.java | 5 +-
.../beam/sdk/io/iceberg/CreateReadTasksDoFn.java | 7 +-
.../beam/sdk/io/iceberg/IcebergCatalogConfig.java | 10 +-
.../org/apache/beam/sdk/io/iceberg/IcebergIO.java | 2 +-
.../beam/sdk/io/iceberg/IcebergScanConfig.java | 3 +-
.../beam/sdk/io/iceberg/IncrementalScanSource.java | 6 +-
.../apache/beam/sdk/io/iceberg/ReadFromTasks.java | 7 +-
.../beam/sdk/io/iceberg/RecordWriterManager.java | 140 ++--
.../org/apache/beam/sdk/io/iceberg/ScanSource.java | 3 +-
.../org/apache/beam/sdk/io/iceberg/TableCache.java | 198 +++--
.../beam/sdk/io/iceberg/WatchForSnapshots.java | 7 +-
.../sdk/io/iceberg/WriteDirectRowsToFiles.java | 12 +-
.../sdk/io/iceberg/WriteGroupedRowsToFiles.java | 12 +-
.../io/iceberg/WritePartitionedRowsToFiles.java | 166 ++--
.../sdk/io/iceberg/WriteUngroupedRowsToFiles.java | 12 +-
.../sdk/io/iceberg/RecordWriterManagerTest.java | 121 +--
.../apache/beam/sdk/io/iceberg/TableCacheTest.java | 126 +++
.../java/org/apache/beam/sdk/managed/Managed.java | 4 +
.../apache_beam/examples/wordcount_rust/README.md | 4 +-
.../examples/wordcount_rust/requirements.txt | 2 +-
.../python/apache_beam/options/pipeline_options.py | 76 ++
.../apache_beam/options/pipeline_options_test.py | 37 +
.../options/pipeline_options_validator.py | 2 +
sdks/python/container/Dockerfile | 3 +
.../container/base_image_requirements_manual.txt | 2 +
sdks/python/container/boot.go | 90 ++-
.../container/ml/py310/base_image_requirements.txt | 44 +-
.../container/ml/py310/gpu_image_requirements.txt | 60 +-
.../container/ml/py311/base_image_requirements.txt | 44 +-
.../container/ml/py311/gpu_image_requirements.txt | 60 +-
.../container/ml/py312/base_image_requirements.txt | 44 +-
.../container/ml/py312/gpu_image_requirements.txt | 60 +-
.../container/ml/py313/base_image_requirements.txt | 44 +-
sdks/python/container/profiler.go | 336 ++++++++
.../container/py310/base_image_requirements.txt | 41 +-
.../container/py311/base_image_requirements.txt | 41 +-
.../container/py312/base_image_requirements.txt | 41 +-
.../container/py313/base_image_requirements.txt | 41 +-
.../container/py314/base_image_requirements.txt | 41 +-
sdks/typescript/package-lock.json | 15 +-
sdks/typescript/package.json | 2 +-
.../site/content/en/documentation/dsls/sql/ddl.md | 344 ++++++++
.../content/en/documentation/dsls/sql/overview.md | 2 +
.../en/documentation/io/built-in/iceberg.md | 2 +-
.../layouts/partials/section-menu/en/sdks.html | 1 +
website/www/site/layouts/shortcodes/tab.html | 22 +
127 files changed, 5038 insertions(+), 808 deletions(-)
create mode 100644 .agent/skills/developing-new-io-connectors/SKILL.md
create mode 100644 examples/notebooks/beam-ml/run_inference_gemini.ipynb
create mode 100644
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/AsyncWrapper.java
create mode 100644
sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/AsyncWrapperTest.java
rename
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/{BeamSqlAliasTest
=> BeamSqlAliasTest.java} (92%)
create mode 100644
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslParametersTest.java
copy
sdks/java/io/{iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergReadSchemaTransformProvider.java
=>
delta/src/main/java/org/apache/beam/sdk/io/delta/DeltaReadSchemaTransformProvider.java}
(51%)
create mode 100644
sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/TableCacheTest.java
create mode 100644 sdks/python/container/profiler.go
create mode 100644 website/www/site/content/en/documentation/dsls/sql/ddl.md
