This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a change to branch timermgr
in repository https://gitbox.apache.org/repos/asf/beam.git
omit 39bf64ea117 [timers] adjust debugging
omit 0ec68ab8f84 Timers received!
omit 616474c6cc8 [go-timers] things I'm pretty sure work
add 4ce8eeda196 Read API Source v2 (#25392)
add 27c55107e56 Moving to 2.47.0-SNAPSHOT on master branch.
add 011296c1465 Apply shallow clone for jenkins (#25595)
add a1f09f7038e Bump golang.org/x/net from 0.5.0 to 0.7.0 in /sdks (#25543)
add a34f4591de4 Bump github.com/containerd/containerd from 1.6.12 to
1.6.18 in /sdks (#25512)
add 19e63e1da78 Add mock modules (#25603)
add 96c68ab058d Add missing entries to mass_comment.py (#25609)
add 948f7cdf489 Fix go postcommit naming (#25613)
add 160be6b14a4 Bump to Dataproc 2.1 (#25616)
add dcfadddf52a Prioritise Avro providers from "extensions/core" (#25611)
add 441d44acb67 Bump golang.org/x/oauth2 in /sdks (#25621)
add bddfd86afd2 restructure ml overview website page (#25607)
add de71d132f6a [Beam#25046]add support for flink 1.16 for Beam Flink
Runner (#25584)
add 5a46e0083f5 Update CHANGE.md to 2.47.0 (#25628)
add 2bd1cebfd78 Bump google.golang.org/api from 0.109.0 to 0.110.0 in
/sdks (#25619)
add 21baee4fa55 Bump go.mongodb.org/mongo-driver from 1.11.1 to 1.11.2 in
/sdks (#25605)
add 0bb5be45bf1 Fix typo in docstring (#25630)
add 124e1d6a5f1 Merge pull request #25627: #25626 Use the correct
constructor when creating StreamWriter
add 9d4df48864d Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#25637)
add 8ec5673ac1c One formatter for Timestamp fields in Storage writes
(#25472)
add cf7ba6d1c59 Update mock requirement in /sdks/python (#25503)
add fabd1f6d7a6 Add ReadFrom/WriteTo Csv/Json as top-level transforms to
the Python SDK. (#25614)
add a3d82e64f31 Merge pull request #25642: #25626 Revert to old
constructor.
add d5810397e60 Fix example code on from-spark.md
add 170c4184ce1 Merge pull request #25633 from andreykot/patch-1
add 1087abeb734 Fix paginated windmill sorted list state.
add 62e9a1239bb Merge pull request #25639: Fix paginated windmill sorted
list state
add 7db76df6936 Disable triggering most IO_Direct precommit on buildSrc
and core change (#25612)
add fed35133ee1 Fix ts version on master (#25651)
add b3273e63960 gen protos before running py38-docs in
build_release_candidate (#25650)
add d7d6a7fae69 whitelist->allowlist in tox.ini (#25647)
add dba64c6ee9e Add typescript to changed version files (#25663)
add 1558dbf30d7 Fix tensorflowhub caching issue (#25661)
add cb8381f36ee Fix Python3.7 PostCommit Azure integration test (#25654)
add 3bff0733df0 Bump golang.org/x/net in /learning/katas/go (#25645)
add f44c553478e Fix flaky load_test.go/TestSample test (#25673)
add e8d690253ef Bump google.golang.org/api from 0.110.0 to 0.111.0 in
/sdks (#25671)
add e1491a637db Create tft criteo grafana dashboards (#25666)
add 347a401dec3 adjust log level (#25692)
add ef581e7ad44 Sickbay autosharding test on Dataflow V2 (#25677)
add 913fefa8f7d Reverts Python Kafka source to use the Java SDF-based
implementation (#25684)
add 4a244b4d507 Add timeout to unit tests causing SickBay PostCommit
timeout (#25664)
add a08c4e1fc49 Remove dask tests (#25697)
add f30f2677eb6 [Bigtable] Migrate BigtableIO to use the veneer client
under the hood (#24015)
add 253fbc794b2 changed metric name to be descriptive of the pipeline type
(#25682)
add f49b450fc15 Evaluate removal of RDD caching for MEMORY_ONLY in the
Spark Dataset runner (#25327)
add 60ac6ef4215 Externalize datastore namespace for ToB (#25708)
add 39cab94361b Add a 30 second sleep in mass_comment.py to avoid
overwhelming Jenkins (#25714)
add 4da60251729 Stop emitting upon truncate in Java PeriodicSequence
(#25716)
add 5a2b93ddaa7 [Spark Dataset runner] Avoid copying outputs for most
cases in ParDo translation (related to #24711) (#25624)
add 9b6b72f3379 [Spark Dataset runner] Prevent stackoverflow in ParDo load
tests when using RDDs to break linage for large number of consequtive ParDos
(#25732)
add fcf111d299a Update documentation by adding parameters for Dataflow
Runner (#25561)
add 08198142087 [Playground] Apache Kafka based Kafka emulator for
Playground examples (#25357)
add 6452dc79822 [Flink Runner] Add new Source classes that are based on
FLIP-27 Source API. (#25525)
add fd13e14dd67 Bump golang.org/x/sys from 0.5.0 to 0.6.0 in /sdks (#25727)
add b3a9aa795ce Bump github.com/tetratelabs/wazero in /sdks (#25689)
add 2011cfcc63a Fail when main session cannot be loaded. (#25617)
add 3b76f4ab6ed Run Playground precommit checks in CI, check gofmt,
reformat Go code (#25733)
add 889d4db025a Give an option to override the number of shards in BQ
streaming insert addresses (#25569)
add 0a4fdc70d44 Bump distlib from 0.3.1 to 0.3.6 in /sdks/python (#25730)
add 4bbd7b89944 Bump golang.org/x/net from 0.7.0 to 0.8.0 in /sdks (#25729)
add be4cbba4b1d add logging
add 237c088cf9a add logging
add e4ddb861756 Merge pull request #25744: Add more logging to Storage API
writes
add 0d804d102b2 Use credential for mongodb performance tests (#25631)
add 188f030065f Run the quickstart command has incorrect flag name
add 92459a2c29a Merge pull request #25703 Run the quickstart command has
incorrect flag name.
add de8f31775b8 [Spark Dataset runner] Change default storage level to
Sparks respective default MEMORY_AND_DISK (closes #25737) (#25738)
add 5d1bcfbae18 [Spark Dataset runner] Remove obsolete code (#25734)
add ec2138ba89e Fix SDF typo. (#25765)
add e12ad36ba83 Document fields for Schema Transforms (#25644)
add c0b6db91ccc Fix DatastoreIO bug causing low throughput on certain
writes (#25701)
add 63b1e5a7394 Add support for passing labels as --labels '{ "name":
"wrench", "mass": "1_3kg", "count": "3" }' (#25739)
add 36486447e4d passing consumer configs from user-passed parameters
(#25766)
add db54a796bde Update chromedriver-binary requirement from <111,>=100 to
>=100,<112 in /sdks/python (#25757)
add 21053bb37f9 Fix contact us block on beam website apache#22695 (#25759)
add 5d8770e41d3 Add machine_type alias to dataflow pipeline options
(#25768)
add 6ccefa778a6 Update pandas requirement from <1.4.0 to <1.6.0 in
/sdks/python (#25756)
add 1c0ea9742da Bump google.golang.org/protobuf from 1.28.1 to 1.29.0 in
/sdks (#25771)
add 48c143de3be Playground integration test run and editing (#25033)
add 60ec8e57c12 adding cache expire for example catalog (#25760)
add 01cb7e9618d Disable Spark UI for load tests (#25792)
add 13a7cbf73f9 Bump cloud.google.com/go/spanner from 1.43.0 to 1.44.0 in
/sdks (#25295)
add 48b15274edc Bump github.com/golang/protobuf from 1.5.2 to 1.5.3 in
/sdks (#25772)
add b1ea4d35540 Update Beam website to release 2.46.0 (#25694)
add 3aa88fb2c31 Updates 2.45.0 artifacts to use archive.apache.org
add 9bdfbdd33f0 Merge pull request #25801: Updates 2.45.0 artifacts to use
archive.apache.org
add dedfd11e2a5 Accept pymongo 4.x for Python SDK (#25786)
add 3362db951b0 Update jupyter-client requirement from <8.0.3,>=6.1.11 to
>=6.1.11,<8.0.4 in /sdks/python (#25726)
add cc2a783bc6a XGBoost modelhandler implementation (#24965)
add fb040ff9c10 Merge pull request #25790: update to new BOM
add 227a846dc2d Merge pull request #25804: Reuse client when constructing
StreamWriter
add 17f7a5d1aff make incoming message public (#25803)
add e235761ba52 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#25816)
add 038e490c35a Bump grpcio from 1.49.1 to 1.51.3 in /sdks/python (#25814)
add 917921fa177 [Go SDK] Fix most compile and bugs on Windows. (#25634)
add 509052c0488 Mark that 2.46 has been released now that the release is
finalized (#25818)
add 0d515df6520 Infrastructure-as-Code to provision a private GKE
autopilot kubernetes cluster and strimzi kafka (#25686)
add 8b3f2964344 Remove save main session option in pubsubio_perf_test
(#25822)
add 5de5ce8f30e LOG error for illegal ramBytes (#25763)
add 3de1eb18345 [25705] Refactor BigQuery SchemaTransforms naming (#25706)
add 366495683eb updated error message
add 1c8d85c048c linting
add afce68d95c0 Merge pull request #25826: Corrected
GscUtil.RewriteOp.OnFailure() error message
add 04c2de61e5d Fix OutputSampler's coder. (#25805)
add daaaaafc0e0 [BEAM-14307] Re-Fix Slow Side input pattern bug in sample
add 4189320e49e Merge pull request #25829: [BEAM-14307] Re-Fix Slow Side
input pattern bug in sample
add 81517a7fffd Add support for common collections.abc types (#25798)
add 6b18305a625 Improve stage packages race condition by checking if
already staged before each try (#25827)
add 6270ec9d014 Bump wazero to v1.0.0-rc.1 (#25691)
add 20510623041 Re-enable Storage Write API tests for Runner v2
add 6f0dfae90be Merge pull request #21814: Re-enable Storage Write API
tests for Runner v2
add 8eefbd4e2b5 Drop myself from Java reviewers
add 77d03e952c7 Merge pull request #25800: Drop myself from Java reviewers
add 5ffb2d539f2 Update release docs with problems encountered during
release (#25596)
add 4cc0bccdf56 Fix excessive checkStateNotNull in JdbcUtil (#25847)
add 4c45f410247 Bump google.golang.org/api from 0.111.0 to 0.112.0 in
/sdks (#25837)
add 99c734db530 Bump github.com/testcontainers/testcontainers-go in /sdks
(#25789)
add e2afe3a60cc Use buf to generate protobuf stubs for ToB backend (#25821)
add 9e044312424 Reorganizing the ML content (#25700)
add 7adcb417f8e [Spark Dataset runner] Implement producer-consumer fusion
for ParDos (closes #25761) (#25762)
add 3f1f46853da Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
(#25836)
add 720762b76b5 Update google-cloud-recommendations-ai requirement in
/sdks/python (#25835)
add 5cb1711c39c Update Protobuf and GCP dependencies in Beam Python SDK
(#24599)
add 14ca840f346 Change the type of Samza Runner MaxBundleSize pipeline
option to match with Flink Runner (#25854)
add 1879587c01c Bump grpcio-tools from 1.37.0 to 1.51.3 in /sdks/python
(#25585)
add 831918f5713 Update sqlalchemy requirement in /sdks/python (#25855)
add eac926878c2 Use autoSharding from config when BigQueryIO.Write is
created. (#25862)
add 738825c90fb Revert "Adding support for DLQ for ZetaSQL (#25426)"
add b2222b5e5d0 Merge pull request #25863: Revert "Adding support for DLQ
for ZetaSQL (#25426)"
add 213b3ed6bf1 Use self hosted hadoop docker image in hdfsIntegrationTest
(#25867)
add cbc1a7661bd Revert "Update sqlalchemy requirement in /sdks/python
(#25855)" (#25872)
add b2afbe0b26f Minor fix: move the getattr out of the loop as it is not
needed to be inside (#25859)
add 821a77beb00 Multi language/runinference example (#25083)
add d9f81b8592d Upgrade tox from 3.x.x to 4.x.x (#25785)
add 167a4c5fb59 Remove changes from MANIFEST.in (#25877)
add b4ee5bd60e0 Update pytest-xdist requirement in /sdks/python (#25833)
add 6f82f34eeb1 Onboard Performance tests using the Change point analysis
tool. (#25754)
add a9826ee2db1 Update default splitting behavior to get fewer, larger,
splits
add de578e606e5 Merge pull request #25871: Update default splitting
behavior to get fewer, larger, splits
add 7555d989799 Make conversion to Iterable tighter (#25866)
add fad7f96dc85 Add support for CPython 3.11 opcodes (#25795)
add ac57b8709d3 Optimize change stream connector with more efficient
batching and blind writes, and add transaction/query tags (#25718)
add e8fad9ca2ba Adding DLQ support for ZetaSQL (#25873)
add 4e96e700c2d Disable firestore integration tests temporarily (#25891)
add 315b53d6611 [Go SDK]: Align filesystem List behaviors (#25811)
add 4ceb475ebe4 Google Cloud Bigtable Change Stream Connector (#25797)
add e8eccaf6290 Update beam.Pardo doc comment about SDFs (#25781)
add 476c43356db add write options with create disposition for bigqueryio
(#25352)
add b5ce1106a24 Add one example to learn beam by doing (#25719)
add 293871bd244 [go-timers] things I'm pretty sure work
add f2a1b8ae45a Timers received!
add c4062732f3c [timers] adjust debugging
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (39bf64ea117)
\
N -- N -- N refs/heads/timermgr (c4062732f3c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/REVIEWERS.yml | 1 -
.github/workflows/playground_backend_precommit.yml | 68 +
.github/workflows/python_tests.yml | 6 +-
.github/workflows/run_perf_alert_tool.yml | 24 +-
.test-infra/dataproc/flink_cluster.sh | 3 +-
.test-infra/jenkins/CommonJobProperties.groovy | 4 +
.test-infra/jenkins/PrecommitJobBuilder.groovy | 18 +-
.test-infra/jenkins/README.md | 32 +-
.../job_PerformanceTests_MongoDBIO_IT.groovy | 4 +-
.test-infra/jenkins/job_PostCommit_Go.groovy | 2 +-
...ob_PostCommit_Python_MongoDBIO_Load_Test.groovy | 4 +-
.../job_PreCommit_Java_Amqp_IO_Direct.groovy | 41 -
.../job_PreCommit_Java_Cassandra_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Cdap_IO_Direct.groovy | 44 -
.../job_PreCommit_Java_Debezium_IO_Direct.groovy | 45 -
...b_PreCommit_Java_ElasticSearch_IO_Direct.groovy | 48 -
.../job_PreCommit_Java_HCatalog_IO_Direct.groovy | 43 -
.../job_PreCommit_Java_Hbase_IO_Direct.groovy | 43 -
.test-infra/jenkins/job_PreCommit_Java_IOs.groovy | 146 +
.../job_PreCommit_Java_InfluxDb_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Jms_IO_Direct.groovy | 41 -
.../job_PreCommit_Java_Kudu_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Mqtt_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Neo4j_IO_Direct.groovy | 44 -
.../job_PreCommit_Java_RabbitMq_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Redis_IO_Direct.groovy | 42 -
...job_PreCommit_Java_Singlestore_IO_Direct.groovy | 43 -
.../job_PreCommit_Java_Snowflake_IO_Direct.groovy | 45 -
.../job_PreCommit_Java_Solr_IO_Direct.groovy | 42 -
.../job_PreCommit_Java_Splunk_IO_Direct.groovy | 41 -
.../job_PreCommit_Java_Thrift_IO_Direct.groovy | 41 -
.../job_PreCommit_Java_Tika_IO_Direct.groovy | 41 -
.test-infra/kafka/README.md | 36 +
.../kafka/strimzi/01-strimzi-operator/README.md | 28 +
.../strimzi/01-strimzi-operator/kustomization.yaml | 8 +-
.../strimzi/01-strimzi-operator/namespace.yaml | 9 +-
...10-ServiceAccount-strimzi-cluster-operator.yaml | 10 +-
...-ClusterRole-strimzi-cluster-operator-role.yaml | 170 +
.../020-RoleBinding-strimzi-cluster-operator.yaml | 18 +-
...-ClusterRole-strimzi-cluster-operator-role.yaml | 55 +
...lusterRoleBinding-strimzi-cluster-operator.yaml | 18 +-
...-ClusterRole-strimzi-cluster-operator-role.yaml | 48 +
.../022-RoleBinding-strimzi-cluster-operator.yaml | 18 +-
...-ClusterRole-strimzi-cluster-operator-role.yaml | 82 +
.../023-RoleBinding-strimzi-cluster-operator.yaml | 18 +-
.../030-ClusterRole-strimzi-kafka-broker.yaml | 19 +-
...i-cluster-operator-kafka-broker-delegation.yaml | 20 +-
.../031-ClusterRole-strimzi-entity-operator.yaml | 61 +
...luster-operator-entity-operator-delegation.yaml | 20 +-
.../033-ClusterRole-strimzi-kafka-client.yaml | 20 +-
...i-cluster-operator-kafka-client-delegation.yaml | 21 +-
.../01-strimzi-operator/v0.33.2/040-Crd-kafka.yaml | 6208 ++++++++++++++++++++
.../v0.33.2/041-Crd-kafkaconnect.yaml | 1952 ++++++
.../v0.33.2/042-Crd-strimzipodset.yaml | 135 +
.../v0.33.2/043-Crd-kafkatopic.yaml | 270 +
.../v0.33.2/044-Crd-kafkauser.yaml | 702 +++
.../v0.33.2/045-Crd-kafkamirrormaker.yaml | 1228 ++++
.../v0.33.2/046-Crd-kafkabridge.yaml | 1178 ++++
.../v0.33.2/047-Crd-kafkaconnector.yaml | 146 +
.../v0.33.2/048-Crd-kafkamirrormaker2.yaml | 1992 +++++++
.../v0.33.2/049-Crd-kafkarebalance.yaml | 159 +
.../050-ConfigMap-strimzi-cluster-operator.yaml | 52 +
.../060-Deployment-strimzi-cluster-operator.yaml | 155 +
.../01-strimzi-operator/v0.33.2/kustomization.yaml | 46 +
.../kafka/strimzi/02-kafka-persistent/README.md | 28 +
.../base/v0.33.2/kafka-persistent.yaml | 53 +
.../base/v0.33.2/kustomization.yaml | 7 +-
.../gke-internal-load-balanced/kustomization.yaml | 11 +-
.../gke-internal-load-balanced/listeners.yaml | 52 +
.test-infra/kafka/strimzi/README.md | 160 +
.../kubernetes/mongodb/load-balancer/mongo.yml | 5 +
.../Python_TFT_Criteo_benchmarks.json | 156 +
.test-infra/terraform/OWNERS | 5 +
.test-infra/terraform/README.md | 31 +
.../terraform/google-cloud-platform/README.md | 44 +
.../google-kubernetes-engine/README.md | 56 +
.../google-kubernetes-engine/main.tf | 73 +
.../modules/01-setup/README.md | 26 +
.../modules/01-setup/iam.tf | 36 +-
.../modules/01-setup/output.tf | 19 +-
.../modules/01-setup/provider.tf | 21 +-
.../modules/01-setup/services.tf | 25 +-
.../modules/01-setup/variables.tf | 21 +-
.../modules/02-network/README.md | 26 +
.../modules/02-network/nat.tf | 43 +
.../modules/02-network/network.tf | 65 +
.../modules/02-network/output.tf | 24 +-
.../modules/02-network/provider.tf | 19 +-
.../modules/02-network/services.tf | 20 +-
.../modules/02-network/variables.tf | 29 +-
.../modules/03-cluster/README.md | 22 +
.../modules/03-cluster/cluster.tf} | 43 +-
.../modules/03-cluster/provider.tf | 19 +-
.../modules/03-cluster/services.tf | 20 +-
.../modules/03-cluster/variables.tf | 53 +
.../modules/04-bastion/README.md | 22 +
.../modules/04-bastion/compute.tf | 64 +
.../modules/04-bastion/data.tf | 44 +
.../modules/04-bastion/provider.tf | 19 +-
.../modules/04-bastion/services.tf | 20 +-
.../modules/04-bastion/tinyproxy.conf | 368 ++
.../modules/04-bastion/variables.tf | 67 +
.../google-kubernetes-engine/variables.tf | 42 +
.../google-kubernetes-engine/versions.tf | 29 +-
CHANGES.md | 45 +-
build.gradle.kts | 39 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 16 +-
.../apache/beam/examples/snippets/Snippets.java | 4 +-
.../get-started/learn_beam_basics_by_doing.ipynb | 1095 ++++
gradle.properties | 6 +-
learning/katas/go/go.mod | 3 +-
learning/katas/go/go.sum | 20 +-
learning/tour-of-beam/backend/README.md | 8 +-
learning/tour-of-beam/backend/function.go | 6 +-
.../backend/integration_tests/client_pg.go | 2 +-
.../backend/internal/fs_content/load.go | 18 +-
.../backend/internal/fs_content/load_test.go | 8 +-
.../backend/internal/service/content.go | 2 +-
.../backend/internal/service/mock_helper.go | 2 +-
.../backend/internal/service/pg_adapter.go | 2 +-
.../backend/internal/storage/schema.go | 16 +-
.../tour-of-beam/backend/playground_api/api.pb.go | 3517 -----------
.../backend/playground_api}/api/v1/api.pb.go | 0
.../backend/playground_api}/api/v1/api_grpc.pb.go | 0
.../backend/playground_api/{ => api/v1}/mock.go | 185 +-
.../backend/playground_api/api_grpc.pb.go | 791 ---
learning/tour-of-beam/frontend/pubspec.lock | 7 +
playground/backend/build.gradle.kts | 16 +
playground/backend/cmd/server/controller.go | 30 +-
playground/backend/cmd/server/http.go | 2 +-
playground/backend/cmd/server/wrapper.go | 2 +-
playground/backend/containers/go/Dockerfile | 7 +
playground/backend/containers/go/build.gradle | 8 +
playground/backend/containers/java/Dockerfile | 7 +-
playground/backend/containers/java/build.gradle | 9 +
playground/backend/containers/python/Dockerfile | 7 +
playground/backend/containers/python/build.gradle | 8 +
playground/backend/containers/scio/Dockerfile | 6 +
playground/backend/containers/scio/build.gradle | 8 +
playground/backend/env_setup.sh | 2 +-
playground/backend/internal/cache/cache.go | 12 +-
.../backend/internal/cache/redis/redis_cache.go | 6 +-
.../internal/cache/redis/redis_cache_test.go | 4 +-
.../internal/code_processing/code_processing.go | 29 +-
.../code_processing/code_processing_test.go | 8 +-
.../internal/constants/datastore_constants.go | 2 +-
.../internal/constants/emulator_constants.go | 2 +-
.../backend/internal/db/datastore/datastore_db.go | 10 +-
.../internal/db/mapper/datastore_mapper_test.go | 2 +-
.../db/schema/migration/migrations_test.go | 2 +-
playground/backend/internal/emulators/emulator.go | 51 +-
playground/backend/internal/emulators/kafka.go | 109 +-
.../backend/internal/environment/application.go | 37 +-
.../internal/environment/environment_service.go | 91 +-
.../environment/environment_service_test.go | 14 +-
.../backend/internal/environment/property.go | 2 +-
playground/backend/internal/executors/executor.go | 2 +-
.../backend/internal/executors/executor_builder.go | 48 +-
playground/backend/internal/fs_tool/fs.go | 24 +
playground/backend/internal/logger/logger.go | 5 +-
.../backend/internal/preparers/go_preparers.go | 8 +-
.../backend/internal/preparers/java_preparers.go | 18 +-
playground/backend/internal/preparers/preparer.go | 6 +-
.../backend/internal/preparers/preparer_test.go | 2 +-
.../backend/internal/preparers/python_preparers.go | 10 +-
.../internal/preparers/python_preparers_test.go | 2 +-
.../backend/internal/preparers/scio_preparers.go | 10 +-
.../setup_tools/life_cycle/life_cycle_setuper.go | 44 +-
.../life_cycle/life_cycle_setuper_test.go | 3 +-
.../internal/streaming/run_output_writer.go | 7 +-
.../backend/internal/tests/test_data/test_data.go | 6 +-
playground/backend/internal/utils/common.go | 4 +-
.../backend/internal/utils/preparers_utils.go | 4 +-
playground/backend/internal/utils/system_utils.go | 4 +-
.../backend/internal/validators/java_validators.go | 4 +-
.../backend/internal/validators/scio_validators.go | 2 +-
playground/backend/logging.properties | 1 +
playground/buf.gen.yaml | 10 +
playground/frontend/README.md | 10 +-
playground/frontend/assets/translations/en.yaml | 1 +
playground/frontend/build.gradle | 42 +
.../integration_test/common/common_finders.dart | 23 +
...tness_mode_test.dart => embedded_run_test.dart} | 33 +-
.../toggle_brightness_mode_test.dart | 20 +-
.../standalone_cancel_running_example_test.dart | 56 +
.../standalone_change_example_sdk_run_test.dart | 23 +-
...alone_change_pipeline_options_and_run_test.dart | 123 +
.../standalone_default_examples_test.dart | 35 +-
.../integration_test/standalone_editing_test.dart | 139 +
.../standalone_example_selector_test.dart | 158 +
.../standalone_run_shortcuts_test.dart | 81 +
.../pipeline_options_dropdown_body.dart | 18 +-
.../pipeline_options_dropdown_input.dart | 3 +
.../pipeline_options_form.dart | 42 +-
.../pipeline_options_row.dart | 75 +
.../example_list/example_item_actions.dart | 13 +-
.../modules/examples/components/filter/filter.dart | 34 +-
.../examples/components/web_scroll_converter.dart | 52 +
.../shortcuts/constants/global_shortcuts.dart | 4 +-
.../notifiers/example_selector_state.dart | 59 +-
.../lib/pages/standalone_playground/screen.dart | 9 +
.../assets/translations/en.yaml | 8 +
.../lib/playground_components.dart | 5 +-
.../lib/src/cache/example_cache.dart | 3 -
.../lib/src/controllers/code_runner.dart | 127 +-
.../example_loaders/examples_loader.dart | 32 +-
.../lib/src/controllers/playground_controller.dart | 43 +-
...ntroller.dart => result_filter_controller.dart} | 10 +-
.../snippet_file_editing_controller.dart | 2 +-
...type_controller.dart => unread_controller.dart} | 40 +-
.../{models/outputs.dart => enums/output_tab.dart} | 20 +-
.../lib/src/enums/output_tab.g.dart | 164 +
.../outputs.dart => enums/result_filter.dart} | 14 +-
.../outputs.dart => enums/unread_entry.dart} | 20 +-
.../lib/src/models/intents.dart | 5 +
.../lib/src/models/outputs.dart | 6 -
.../repositories/code_client/grpc_code_client.dart | 13 +-
.../lib/src/repositories/code_repository.dart | 103 +-
.../src/repositories/models/run_code_result.dart | 40 +-
.../outputs.dart => util/connectivity_result.dart} | 18 +-
.../logical_keyboard_key.dart} | 20 +-
.../outputs.dart => util/native_platform.dart} | 20 +-
.../lib/src/widgets/complexity.dart | 13 +-
.../lib/src/widgets/output/graph/graph.dart | 105 -
.../output/graph/painter.dart} | 20 +-
.../{result_filter_bubble.dart => graph_tab.dart} | 37 +-
.../lib/src/widgets/output/graph_tab_content.dart | 108 +
.../lib/src/widgets/output/output.dart | 121 +-
.../lib/src/widgets/output/output_area.dart | 81 -
.../lib/src/widgets/output/output_result.dart | 69 -
.../lib/src/widgets/output/output_tab.dart | 91 +-
.../src/widgets/output/result_filter_bubble.dart | 24 +-
...{output_tabs.dart => result_filter_button.dart} | 53 +-
.../src/widgets/output/result_filter_popover.dart | 43 +-
.../{result_filter_bubble.dart => result_tab.dart} | 41 +-
.../lib/src/widgets/output/result_tab_content.dart | 85 +
.../lib/src/widgets/scroll_2d.dart | 84 +
.../lib/src/widgets/tabs/tab_bar.dart | 19 +-
.../{tabs/tab_bar.dart => unread/builder.dart} | 30 +-
.../lib/src/widgets/unread/clearer.dart | 58 +
.../unread/marker.dart} | 19 +-
.../frontend/playground_components/pubspec.yaml | 9 +-
.../examples_loader_test.mocks.dart | 46 +-
.../controllers/playground_controller_test.dart | 4 +-
.../src/controllers/unread_controller_test.dart | 77 +
.../src/repositories/code_repository_test.dart | 129 +-
.../lib/src/common_finders.dart | 11 +-
.../lib/src/examples/go/minimal_word_count.dart | 2 +-
.../lib/src/examples/go/word_count.dart | 2 +-
.../lib/src/examples/java/aggregation_max.dart | 2 +-
.../lib/src/examples/java/minimal_word_count.dart | 2 +-
.../lib/src/examples/python/aggregation_mean.dart | 2 +-
.../examples/python/word_count_with_metrics.dart | 2 +-
.../lib/src/examples/scio/minimal_word_count.dart | 2 +-
.../playground_components_dev/lib/src/expect.dart | 15 +-
.../playground_components_dev/lib/src/finder.dart | 69 +
.../lib/src/widget_tester.dart | 110 +-
.../playground_components_dev/pubspec.yaml | 5 +-
playground/frontend/pubspec.lock | 28 +-
playground/frontend/pubspec.yaml | 6 +-
.../states/example_selector_state_test.dart | 18 +-
playground/infrastructure/api/v1/api_pb2.py | 205 +-
playground/infrastructure/api/v1/api_pb2.pyi | 508 +-
playground/kafka-emulator/build.gradle | 53 +
.../org/apache/beam/playground/KafkaEmulator.java | 72 +
.../org/apache/beam/playground/package-info.java} | 23 +-
release/src/main/scripts/mass_comment.py | 55 +-
release/src/main/scripts/set_version.sh | 1 +
release/src/main/scripts/start_snapshot_build.sh | 2 +-
.../io/source/compat/FlinkSourceCompat.java | 17 +-
.../io/source/compat/SplitEnumeratorCompat.java | 20 +-
.../streaming/io/source/compat/package-info.java | 19 +-
.../streaming/io/source/SourceTestCompat.java | 62 +
.../io/source/compat/SplitEnumeratorCompat.java | 20 +-
.../io/source/compat/FlinkSourceCompat.java | 14 +-
.../streaming/io/source/SourceTestCompat.java | 93 +
runners/flink/1.16/build.gradle | 34 +
.../job-server-container/build.gradle | 0
.../flink/1.16/job-server/build.gradle | 28 +-
runners/flink/flink_runner.gradle | 2 +
.../flink/translation/utils/SerdeUtils.java | 85 +
.../wrappers/streaming/io/source/FlinkSource.java | 152 +
.../streaming/io/source/FlinkSourceReaderBase.java | 399 ++
.../streaming/io/source/FlinkSourceSplit.java | 76 +
.../io/source/FlinkSourceSplitEnumerator.java | 181 +
.../io/source/bounded/FlinkBoundedSource.java | 67 +
.../source/bounded/FlinkBoundedSourceReader.java | 146 +
.../streaming/io/source/bounded/package-info.java | 20 +-
.../io/source/impulse/BeamImpulseSource.java | 107 +
.../streaming/io/source/impulse/package-info.java | 20 +-
.../wrappers/streaming/io/source/package-info.java | 20 +-
.../io/source/unbounded/FlinkUnboundedSource.java | 64 +
.../unbounded/FlinkUnboundedSourceReader.java | 272 +
.../io/source/unbounded/package-info.java | 20 +-
.../runners/flink/ReadSourceStreamingTest.java | 3 +-
.../apache/beam/runners/flink/ReadSourceTest.java | 3 +-
.../flink/streaming/GroupByNullKeyTest.java | 3 +-
.../flink/streaming/TopWikipediaSessionsTest.java | 3 +-
.../streaming/io/TestBoundedCountingSource.java | 150 +
.../wrappers/streaming/io/TestCountingSource.java | 63 +-
.../io/source/FlinkSourceReaderTestBase.java | 403 ++
.../io/source/FlinkSourceSplitEnumeratorTest.java | 151 +
.../wrappers/streaming/io/source/TestSource.java | 23 +-
.../bounded/FlinkBoundedSourceReaderTest.java | 146 +
.../unbounded/FlinkUnboundedSourceReaderTest.java | 318 +
runners/google-cloud-dataflow-java/build.gradle | 4 +-
.../beam/runners/dataflow/util/PackageUtil.java | 10 +-
.../runners/dataflow/util/PackageUtilTest.java | 10 +-
.../dataflow/worker/WindmillStateReader.java | 15 +-
.../dataflow/worker/WindmillStateReaderTest.java | 58 +-
.../beam/runners/samza/SamzaPipelineOptions.java | 8 +-
.../translation/PipelineTranslator.java | 97 +-
.../translation/TransformTranslator.java | 7 +
.../batch/DoFnPartitionIteratorFactory.java | 131 +-
.../translation/batch/DoFnRunnerFactory.java | 296 +
.../translation/batch/DoFnRunnerWithMetrics.java | 19 +-
.../translation/batch/ParDoTranslatorBatch.java | 173 +-
.../runners/spark/SparkCommonPipelineOptions.java | 16 +-
scripts/ci/release/comment_pr_trigger_phrases.sh | 1 +
sdks/go.mod | 105 +-
sdks/go.sum | 355 +-
sdks/go/examples/wasm/README.md | 6 +-
sdks/go/pkg/beam/core/core.go | 2 +-
.../runtime/xlangx/expansionx/download_test.go | 2 +-
sdks/go/pkg/beam/core/util/symtab/symtab_test.go | 3 +
sdks/go/pkg/beam/io/bigqueryio/bigquery.go | 43 +-
sdks/go/pkg/beam/io/filesystem/filesystem.go | 1 +
sdks/go/pkg/beam/io/filesystem/gcs/gcs.go | 52 +-
sdks/go/pkg/beam/io/filesystem/local/local_test.go | 24 +-
sdks/go/pkg/beam/io/filesystem/memfs/memory.go | 7 +-
.../go/pkg/beam/io/filesystem/memfs/memory_test.go | 17 +-
sdks/go/pkg/beam/io/filesystem/s3/s3.go | 7 +-
sdks/go/pkg/beam/io/filesystem/s3/s3_test.go | 2 +-
sdks/go/pkg/beam/io/filesystem/s3/util.go | 9 -
sdks/go/pkg/beam/io/filesystem/s3/util_test.go | 27 -
sdks/go/pkg/beam/io/filesystem/util.go | 12 +
sdks/go/pkg/beam/pardo.go | 34 +-
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 16 +-
sdks/go/pkg/beam/runners/dataflow/dataflow_test.go | 52 +
.../go/pkg/beam/util/fsx/prefix.go | 17 +-
.../go/pkg/beam/util/fsx/prefix_test.go | 37 +-
.../test/integration/internal/jars/run_nonunix.go | 3 +-
.../integration/io/bigqueryio/bigqueryio_test.go | 193 +
.../test/integration/io/bigqueryio/helper_test.go | 81 +
.../container/license_scripts/dep_urls_java.yaml | 2 +-
.../apache/beam/sdk/coders/RowCoderGenerator.java | 12 +-
.../src/main/java/org/apache/beam/sdk/io/Read.java | 14 +-
.../org/apache/beam/sdk/schemas/io/Providers.java | 11 +-
.../beam/sdk/transforms/PeriodicSequence.java | 8 +-
.../transforms/resourcehints/ResourceHints.java | 25 +-
.../java/org/apache/beam/sdk/PipelineTest.java | 2 +
.../sdk/coders/PCollectionCustomCoderTest.java | 2 +
.../java/org/apache/beam/sdk/io/AvroIOTest.java | 3 +
.../java/org/apache/beam/sdk/io/FileIOTest.java | 3 +
.../beam/sdk/transforms/GroupIntoBatchesTest.java | 2 +
.../beam/sdk/transforms/MapElementsTest.java | 3 +
.../org/apache/beam/sdk/transforms/ParDoTest.java | 3 +
.../beam/sdk/transforms/SplittableDoFnTest.java | 3 +
.../org/apache/beam/sdk/transforms/ViewTest.java | 3 +
.../beam/sdk/transforms/WithTimestampsTest.java | 3 +
.../beam/sdk/extensions/avro/io/AvroIOTest.java | 3 +
.../beam/sdk/extensions/gcp/util/GcsUtil.java | 4 +-
.../ExternalSchemaIOTransformRegistrar.java | 60 +-
sdks/java/extensions/sql/build.gradle | 2 +-
.../provider/bigtable/BigtableClientWrapper.java | 86 +-
.../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 14 +-
.../sql/zetasql/BeamZetaSqlCalcRelTest.java | 70 -
.../sql/zetasql/ZetaSqlDialectSpecTest.java | 14 +-
.../harness/data/PCollectionConsumerRegistry.java | 38 +-
.../beam/fn/harness/debug/OutputSampler.java | 39 +-
.../beam/fn/harness/debug/DataSamplerTest.java | 31 +-
.../beam/fn/harness/debug/OutputSamplerTest.java | 49 +-
sdks/java/io/google-cloud-platform/build.gradle | 12 +-
...eryExportReadSchemaTransformConfiguration.java} | 18 +-
...BigQueryExportReadSchemaTransformProvider.java} | 36 +-
...ileLoadsWriteSchemaTransformConfiguration.java} | 18 +-
...ueryFileLoadsWriteSchemaTransformProvider.java} | 32 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 359 +-
.../beam/sdk/io/gcp/bigquery/BigQueryOptions.java | 11 +
.../io/gcp/bigquery/BigQuerySchemaIOProvider.java | 9 +-
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 18 +-
.../io/gcp/bigquery/BigQueryStorageSourceBase.java | 65 +-
.../BigQueryStorageStreamBundleSource.java | 381 ++
.../bigquery/StorageApiWriteUnshardedRecords.java | 8 +-
.../bigquery/StorageApiWritesShardedRecords.java | 10 +-
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 39 +-
...ueryStorageWriteApiSchemaTransformProvider.java | 52 +-
.../beam/sdk/io/gcp/bigtable/BigtableConfig.java | 135 +-
.../io/gcp/bigtable/BigtableConfigTranslator.java | 377 ++
.../beam/sdk/io/gcp/bigtable/BigtableIO.java | 555 +-
.../sdk/io/gcp/bigtable/BigtableReadOptions.java | 41 +-
.../beam/sdk/io/gcp/bigtable/BigtableService.java | 25 +-
.../io/gcp/bigtable/BigtableServiceFactory.java | 195 +
.../sdk/io/gcp/bigtable/BigtableServiceImpl.java | 618 +-
.../sdk/io/gcp/bigtable/BigtableWriteOptions.java | 121 +
...pConverter.java => FixedCredentialFactory.java} | 33 +-
.../bigtable/VendoredListenableFutureAdapter.java | 10 +-
.../changestreams/ByteStringRangeHelper.java | 118 +-
.../changestreams/ChangeStreamMetrics.java | 145 +
.../changestreams/ChangeStreamMutation.java | 27 -
.../bigtable/changestreams/TimestampConverter.java | 14 +-
.../changestreams/action/ActionFactory.java | 18 +-
.../changestreams/action/ChangeStreamAction.java | 116 +-
.../action/DetectNewPartitionsAction.java | 27 +-
.../action/GenerateInitialPartitionsAction.java | 46 +-
.../action/ReadChangeStreamPartitionAction.java | 126 +-
.../dao/BigtableChangeStreamAccessor.java | 194 +
.../changestreams/dao/ChangeStreamDao.java | 71 +-
.../gcp/bigtable/changestreams/dao/DaoFactory.java | 36 +-
.../changestreams/dao/MetadataTableAdminDao.java | 95 +-
.../changestreams/dao/MetadataTableDao.java | 165 +-
.../dofn/DetectNewPartitionsDoFn.java | 21 +-
.../changestreams/dofn/InitializeDoFn.java | 33 +-
.../dofn/ReadChangeStreamPartitionDoFn.java | 18 +-
.../encoder/MetadataTableEncoder.java | 62 +
.../changestreams/encoder/package-info.java | 2 +
.../changestreams/model/PartitionRecord.java | 66 +-
.../ReadChangeStreamPartitionProgressTracker.java | 2 +-
.../changestreams/restriction/StreamProgress.java | 64 +-
.../beam/sdk/io/gcp/datastore/DatastoreV1.java | 21 +-
.../beam/sdk/io/gcp/pubsub/PubsubClient.java | 2 +-
.../PubsubLiteReadSchemaTransformProvider.java | 38 +-
.../PubsubLiteWriteSchemaTransformProvider.java | 15 +-
.../io/gcp/pubsublite/internal/PubsubLiteSink.java | 3 +-
.../pubsublite/internal/SubscribeTransform.java | 6 +-
.../MetadataSpannerConfigFactory.java | 2 +-
.../action/ChildPartitionsRecordAction.java | 3 +-
.../action/DetectNewPartitionsAction.java | 19 +-
.../changestreams/dao/PartitionMetadataDao.java | 68 +-
.../apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java | 16 +-
...ueryExportReadSchemaTransformProviderTest.java} | 49 +-
...FileLoadsWriteSchemaTransformProviderTest.java} | 50 +-
...eryIOStorageReadWithStreamBundleSourceTest.java | 2156 +++++++
.../bigquery/TableRowToStorageApiProtoTest.java | 60 +-
...StorageWriteApiSchemaTransformProviderTest.java | 5 +-
.../sdk/io/gcp/bigtable/BigtableConfigTest.java | 90 +-
.../gcp/bigtable/BigtableConfigTranslatorTest.java | 299 +
.../beam/sdk/io/gcp/bigtable/BigtableIOTest.java | 330 +-
.../io/gcp/bigtable/BigtableServiceImplTest.java | 558 +-
.../beam/sdk/io/gcp/bigtable/BigtableWriteIT.java | 138 +-
.../changestreams/ByteStringRangeHelperTest.java | 186 +
.../changestreams/TimestampConverterTest.java | 22 +-
.../action/ChangeStreamActionTest.java | 179 +
.../action/DetectNewPartitionsActionTest.java | 122 +
.../GenerateInitialPartitionsActionTest.java | 115 +
.../ReadChangeStreamPartitionActionTest.java | 202 +
.../dao/MetadataTableAdminDaoTest.java | 126 +
.../changestreams/dao/MetadataTableDaoTest.java | 140 +
.../changestreams/dofn/InitializeDoFnTest.java | 123 +
...adChangeStreamPartitionProgressTrackerTest.java | 84 +
.../beam/sdk/io/gcp/pubsublite/ReadWriteIT.java | 2 +-
.../pubsublite/internal/PubsubLiteSinkTest.java | 31 +-
.../action/ChildPartitionsRecordActionTest.java | 4 +-
.../dao/PartitionMetadataDaoTest.java | 29 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcUtil.java | 19 +-
.../org/apache/beam/sdk/io/jdbc/SchemaUtil.java | 6 +-
.../org/apache/beam/sdk/io/jdbc/JdbcIOTest.java | 110 +-
.../KafkaReadSchemaTransformConfiguration.java | 37 +-
.../io/kafka/KafkaReadSchemaTransformProvider.java | 51 +-
.../kafka/KafkaWriteSchemaTransformProvider.java | 18 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 2 +-
.../KafkaReadSchemaTransformProviderTest.java | 10 +-
.../apache/beam/sdk/io/mongodb/MongoDBIOIT.java | 34 +-
sdks/java/testing/load-tests/build.gradle | 7 +
.../coders/proto2_coder_test_messages_pb2.py | 319 +-
.../apache_beam/examples/inference/README.md | 67 +
.../inference/multi_language_inference/README.md | 69 +
.../beam/examples/MultiLangRunInference.java | 92 +
.../multi_language_custom_transform}/__init__.py | 3 +-
.../composite_transform.py | 122 +
.../multi_language_custom_transform/setup.py} | 10 +-
.../inference/xgboost_iris_classification.py | 153 +
.../io/azure/integration_test/docker-compose.yml | 2 +-
.../io/external/xlang_kafkaio_perf_test.py | 3 +-
sdks/python/apache_beam/io/fileio.py | 9 +-
sdks/python/apache_beam/io/gcp/bigquery.py | 13 +-
sdks/python/apache_beam/io/gcp/bigquery_test.py | 12 +-
.../io/gcp/datastore/v1new/datastoreio.py | 2 +-
.../io/gcp/datastore/v1new/datastoreio_test.py | 21 +-
.../apache_beam/io/gcp/pubsub_io_perf_test.py | 3 -
.../io/hdfs_integration_test/docker-compose.yml | 4 +-
sdks/python/apache_beam/io/kafka.py | 7 +-
sdks/python/apache_beam/io/textio.py | 166 +-
sdks/python/apache_beam/io/textio_test.py | 33 +
sdks/python/apache_beam/ml/gcp/__init__.py | 9 +
.../python/apache_beam/ml/gcp/naturallanguageml.py | 14 +-
.../apache_beam/ml/gcp/naturallanguageml_test.py | 2 +-
.../apache_beam/ml/gcp/videointelligenceml.py | 18 +-
.../apache_beam/ml/gcp/videointelligenceml_test.py | 6 +-
.../apache_beam/ml/inference/pytorch_inference.py | 4 +-
.../ml/inference/tensorflow_inference_it_test.py | 29 +
.../ml/inference/tensorflow_tests_requirements.txt | 2 +-
.../apache_beam/ml/inference/xgboost_inference.py | 334 ++
.../ml/inference/xgboost_inference_it_test.py | 350 ++
.../ml/inference/xgboost_inference_test.py | 443 ++
.../xgboost_tests_requirements.txt} | 3 +
.../python/apache_beam/options/pipeline_options.py | 5 +-
sdks/python/apache_beam/pipeline.py | 2 +-
.../runners/dataflow/internal/apiclient.py | 21 +-
.../runners/dataflow/internal/apiclient_test.py | 19 +
.../python/apache_beam/runners/pipeline_context.py | 10 +-
.../runners/portability/abstract_job_service.py | 2 +-
.../runners/portability/fn_api_runner/execution.py | 4 +-
.../runners/portability/fn_api_runner/fn_runner.py | 2 +-
.../portability/fn_api_runner/fn_runner_test.py | 12 +-
.../portability/fn_api_runner/translations.py | 2 +-
.../portability/fn_api_runner/worker_handlers.py | 2 +-
.../runners/portability/local_job_service.py | 2 +-
.../apache_beam/runners/worker/log_handler.py | 2 +-
.../apache_beam/runners/worker/sdk_worker.py | 7 +-
.../apache_beam/runners/worker/sdk_worker_main.py | 26 +-
.../python/apache_beam/testing/analyzers/README.md | 9 +-
.../apache_beam/testing/analyzers/constants.py | 2 +-
.../testing/analyzers/github_issues_utils.py | 3 +-
.../apache_beam/testing/analyzers/perf_analysis.py | 17 +-
.../testing/analyzers/perf_analysis_utils.py | 20 +-
.../testing/analyzers/tests_config.yaml | 59 +-
.../benchmarks/cloudml/cloudml_benchmark_test.py | 24 -
sdks/python/apache_beam/transforms/core.py | 4 +-
sdks/python/apache_beam/transforms/external.py | 2 +-
.../typehints/native_type_compatibility.py | 23 +
.../typehints/native_type_compatibility_test.py | 37 +
sdks/python/apache_beam/typehints/opcodes.py | 36 +-
.../apache_beam/typehints/trivial_inference.py | 129 +-
sdks/python/apache_beam/version.py | 2 +-
sdks/python/build-requirements.txt | 17 +-
sdks/python/container/Dockerfile | 4 +-
.../container/base_image_requirements_manual.txt | 5 +-
.../container/license_scripts/dep_urls_py.yaml | 3 +
.../container/py310/base_image_requirements.txt | 88 +-
.../container/py37/base_image_requirements.txt | 85 +-
.../container/py38/base_image_requirements.txt | 82 +-
.../container/py39/base_image_requirements.txt | 82 +-
sdks/python/gen_protos.py | 46 +-
sdks/python/pytest.ini | 1 +
sdks/python/scripts/generate_pydoc.sh | 6 +-
sdks/python/setup.py | 53 +-
sdks/python/test-suites/direct/common.gradle | 35 +-
sdks/python/test-suites/tox/common.gradle | 4 +-
sdks/python/test-suites/tox/py38/build.gradle | 21 +-
sdks/python/tox.ini | 56 +-
sdks/typescript/package.json | 2 +-
settings.gradle.kts | 6 +
website/www/site/config.toml | 2 +-
website/www/site/content/en/blog/beam-2.46.0.md | 228 +
.../site/content/en/contribute/release-guide.md | 16 +-
.../site/content/en/documentation/ml/about-ml.md | 154 +
.../content/en/documentation/ml/data-processing.md | 2 +-
.../en/documentation/ml/inference-overview.md | 64 +
.../en/documentation/ml/large-language-modeling.md | 29 +-
.../documentation/ml/multi-language-inference.md | 133 +
.../site/content/en/documentation/ml/overview.md | 142 +-
.../documentation/sdks/python-machine-learning.md | 61 +-
.../python/elementwise/runinference-pytorch.md | 63 +
.../python/elementwise/runinference-sklearn.md | 64 +
.../transforms/python/elementwise/runinference.md | 86 +-
.../www/site/content/en/get-started/downloads.md | 14 +-
.../www/site/content/en/get-started/from-spark.md | 9 +-
.../site/content/en/get-started/quickstart/go.md | 2 +-
website/www/site/i18n/footer/en.yaml | 4 +-
website/www/site/layouts/partials/footer.html | 2 +-
.../partials/section-menu/en/documentation.html | 67 +-
.../www/site/layouts/shortcodes/playground.html | 6 +-
website/www/site/static/images/ml-workflows.svg | 2 +-
564 files changed, 38602 insertions(+), 10640 deletions(-)
create mode 100644 .github/workflows/playground_backend_precommit.yml
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Amqp_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Cassandra_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Cdap_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Debezium_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_ElasticSearch_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_HCatalog_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Hbase_IO_Direct.groovy
create mode 100644 .test-infra/jenkins/job_PreCommit_Java_IOs.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_InfluxDb_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Jms_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Kudu_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Mqtt_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Neo4j_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_RabbitMq_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Redis_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Singlestore_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Snowflake_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Solr_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Splunk_IO_Direct.groovy
delete mode 100644
.test-infra/jenkins/job_PreCommit_Java_Thrift_IO_Direct.groovy
delete mode 100644 .test-infra/jenkins/job_PreCommit_Java_Tika_IO_Direct.groovy
create mode 100644 .test-infra/kafka/README.md
create mode 100644 .test-infra/kafka/strimzi/01-strimzi-operator/README.md
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/kustomization.yaml (86%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/namespace.yaml (86%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/010-ServiceAccount-strimzi-cluster-operator.yaml
(88%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/020-ClusterRole-strimzi-cluster-operator-role.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/020-RoleBinding-strimzi-cluster-operator.yaml
(70%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/021-ClusterRole-strimzi-cluster-operator-role.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/021-ClusterRoleBinding-strimzi-cluster-operator.yaml
(70%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/022-ClusterRole-strimzi-cluster-operator-role.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/022-RoleBinding-strimzi-cluster-operator.yaml
(68%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/023-ClusterRole-strimzi-cluster-operator-role.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/023-RoleBinding-strimzi-cluster-operator.yaml
(69%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/030-ClusterRole-strimzi-kafka-broker.yaml
(66%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/030-ClusterRoleBinding-strimzi-cluster-operator-kafka-broker-delegation.yaml
(57%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/031-ClusterRole-strimzi-entity-operator.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml
(57%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/033-ClusterRole-strimzi-kafka-client.yaml
(62%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/033-ClusterRoleBinding-strimzi-cluster-operator-kafka-client-delegation.yaml
(55%)
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/040-Crd-kafka.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/041-Crd-kafkaconnect.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/042-Crd-strimzipodset.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/043-Crd-kafkatopic.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/044-Crd-kafkauser.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/045-Crd-kafkamirrormaker.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/046-Crd-kafkabridge.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/047-Crd-kafkaconnector.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/048-Crd-kafkamirrormaker2.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/049-Crd-kafkarebalance.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/050-ConfigMap-strimzi-cluster-operator.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/060-Deployment-strimzi-cluster-operator.yaml
create mode 100644
.test-infra/kafka/strimzi/01-strimzi-operator/v0.33.2/kustomization.yaml
create mode 100644 .test-infra/kafka/strimzi/02-kafka-persistent/README.md
create mode 100644
.test-infra/kafka/strimzi/02-kafka-persistent/base/v0.33.2/kafka-persistent.yaml
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/02-kafka-persistent/base/v0.33.2/kustomization.yaml
(86%)
copy sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt =>
.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/kustomization.yaml
(80%)
create mode 100644
.test-infra/kafka/strimzi/02-kafka-persistent/overlays/gke-internal-load-balanced/listeners.yaml
create mode 100644 .test-infra/kafka/strimzi/README.md
create mode 100644
.test-infra/metrics/grafana/dashboards/perftests_metrics/Python_TFT_Criteo_benchmarks.json
create mode 100644 .test-infra/terraform/OWNERS
create mode 100644 .test-infra/terraform/README.md
create mode 100644 .test-infra/terraform/google-cloud-platform/README.md
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/README.md
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/main.tf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/README.md
copy
playground/frontend/integration_test/miscellaneous_ui/toggle_brightness_mode_test.dart
=>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/iam.tf
(50%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/output.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/provider.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/services.tf
(76%)
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/01-setup/variables.tf
(64%)
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/README.md
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/nat.tf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/network.tf
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/output.tf
(66%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/provider.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/services.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/02-network/variables.tf
(51%)
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/03-cluster/README.md
rename .test-infra/{jenkins/job_PreCommit_Java_Clickhouse_IO_Direct.groovy =>
terraform/google-cloud-platform/google-kubernetes-engine/modules/03-cluster/cluster.tf}
(51%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/03-cluster/provider.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/03-cluster/services.tf
(78%)
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/03-cluster/variables.tf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/README.md
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/compute.tf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/data.tf
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/provider.tf
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/services.tf
(78%)
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/tinyproxy.conf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/modules/04-bastion/variables.tf
create mode 100644
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/variables.tf
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
.test-infra/terraform/google-cloud-platform/google-kubernetes-engine/versions.tf
(77%)
create mode 100644
examples/notebooks/get-started/learn_beam_basics_by_doing.ipynb
delete mode 100644 learning/tour-of-beam/backend/playground_api/api.pb.go
copy {playground/backend/internal =>
learning/tour-of-beam/backend/playground_api}/api/v1/api.pb.go (100%)
copy {playground/backend/internal =>
learning/tour-of-beam/backend/playground_api}/api/v1/api_grpc.pb.go (100%)
rename learning/tour-of-beam/backend/playground_api/{ => api/v1}/mock.go (86%)
delete mode 100644 learning/tour-of-beam/backend/playground_api/api_grpc.pb.go
copy
playground/frontend/integration_test/{miscellaneous_ui/toggle_brightness_mode_test.dart
=> embedded_run_test.dart} (58%)
create mode 100644
playground/frontend/integration_test/standalone_cancel_running_example_test.dart
create mode 100644
playground/frontend/integration_test/standalone_change_pipeline_options_and_run_test.dart
create mode 100644
playground/frontend/integration_test/standalone_editing_test.dart
create mode 100644
playground/frontend/integration_test/standalone_example_selector_test.dart
create mode 100644
playground/frontend/integration_test/standalone_run_shortcuts_test.dart
create mode 100644
playground/frontend/lib/modules/editor/components/pipeline_options_dropdown/pipeline_options_row.dart
create mode 100644
playground/frontend/lib/modules/examples/components/web_scroll_converter.dart
copy
playground/frontend/playground_components/lib/src/controllers/{output_filter_type_controller.dart
=> result_filter_controller.dart} (79%)
copy
playground/frontend/playground_components/lib/src/controllers/{output_filter_type_controller.dart
=> unread_controller.dart} (51%)
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
enums/output_tab.dart} (78%)
create mode 100644
playground/frontend/playground_components/lib/src/enums/output_tab.g.dart
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
enums/result_filter.dart} (80%)
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
enums/unread_entry.dart} (78%)
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
util/connectivity_result.dart} (78%)
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
util/logical_keyboard_key.dart} (76%)
copy playground/frontend/playground_components/lib/src/{models/outputs.dart =>
util/native_platform.dart} (78%)
delete mode 100644
playground/frontend/playground_components/lib/src/widgets/output/graph/graph.dart
copy
playground/frontend/playground_components/lib/src/{controllers/output_filter_type_controller.dart
=> widgets/output/graph/painter.dart} (67%)
copy
playground/frontend/playground_components/lib/src/widgets/output/{result_filter_bubble.dart
=> graph_tab.dart} (65%)
create mode 100644
playground/frontend/playground_components/lib/src/widgets/output/graph_tab_content.dart
delete mode 100644
playground/frontend/playground_components/lib/src/widgets/output/output_area.dart
delete mode 100644
playground/frontend/playground_components/lib/src/widgets/output/output_result.dart
rename
playground/frontend/playground_components/lib/src/widgets/output/{output_tabs.dart
=> result_filter_button.dart} (55%)
copy
playground/frontend/playground_components/lib/src/widgets/output/{result_filter_bubble.dart
=> result_tab.dart} (63%)
create mode 100644
playground/frontend/playground_components/lib/src/widgets/output/result_tab_content.dart
create mode 100644
playground/frontend/playground_components/lib/src/widgets/scroll_2d.dart
copy
playground/frontend/playground_components/lib/src/widgets/{tabs/tab_bar.dart =>
unread/builder.dart} (58%)
create mode 100644
playground/frontend/playground_components/lib/src/widgets/unread/clearer.dart
copy
playground/frontend/playground_components/lib/src/{controllers/output_filter_type_controller.dart
=> widgets/unread/marker.dart} (69%)
create mode 100644
playground/frontend/playground_components/test/src/controllers/unread_controller_test.dart
create mode 100644 playground/kafka-emulator/build.gradle
create mode 100644
playground/kafka-emulator/src/main/java/org/apache/beam/playground/KafkaEmulator.java
copy playground/{frontend/playground_components/lib/src/models/outputs.dart =>
kafka-emulator/src/main/java/org/apache/beam/playground/package-info.java} (77%)
copy
playground/frontend/playground_components/lib/src/controllers/output_filter_type_controller.dart
=>
runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/compat/FlinkSourceCompat.java
(62%)
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/compat/SplitEnumeratorCompat.java
(66%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/compat/package-info.java
(78%)
create mode 100644
runners/flink/1.12/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/SourceTestCompat.java
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
runners/flink/1.13/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/compat/SplitEnumeratorCompat.java
(67%)
rename
playground/frontend/playground_components/lib/src/controllers/output_filter_type_controller.dart
=>
runners/flink/1.14/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/compat/FlinkSourceCompat.java
(68%)
create mode 100644
runners/flink/1.14/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/SourceTestCompat.java
create mode 100644 runners/flink/1.16/build.gradle
copy runners/flink/{1.12 => 1.16}/job-server-container/build.gradle (100%)
copy playground/frontend/playground_components/lib/src/models/intents.dart =>
runners/flink/1.16/job-server/build.gradle (52%)
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/utils/SerdeUtils.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSource.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSourceReaderBase.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSourceSplit.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSourceSplitEnumerator.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/bounded/FlinkBoundedSource.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/bounded/FlinkBoundedSourceReader.java
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/bounded/package-info.java
(78%)
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/impulse/BeamImpulseSource.java
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/impulse/package-info.java
(78%)
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/package-info.java
(78%)
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/unbounded/FlinkUnboundedSource.java
create mode 100644
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/unbounded/FlinkUnboundedSourceReader.java
copy playground/frontend/playground_components/lib/src/models/outputs.dart =>
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/unbounded/package-info.java
(78%)
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/TestBoundedCountingSource.java
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSourceReaderTestBase.java
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/FlinkSourceSplitEnumeratorTest.java
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamMetrics.java
=>
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/TestSource.java
(66%)
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/bounded/FlinkBoundedSourceReaderTest.java
create mode 100644
runners/flink/src/test/java/org/apache/beam/runners/flink/translation/wrappers/streaming/io/source/unbounded/FlinkUnboundedSourceReaderTest.java
create mode 100644
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnRunnerFactory.java
copy playground/backend/internal/constants/emulator_constants.go =>
sdks/go/pkg/beam/util/fsx/prefix.go (70%)
copy learning/katas/go/go.mod => sdks/go/pkg/beam/util/fsx/prefix_test.go (57%)
create mode 100644 sdks/go/test/integration/io/bigqueryio/bigqueryio_test.go
create mode 100644 sdks/go/test/integration/io/bigqueryio/helper_test.go
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformReadConfiguration.java
=> BigQueryExportReadSchemaTransformConfiguration.java} (80%)
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformReadProvider.java
=> BigQueryExportReadSchemaTransformProvider.java} (81%)
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformWriteConfiguration.java
=> BigQueryFileLoadsWriteSchemaTransformConfiguration.java} (77%)
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformWriteProvider.java
=> BigQueryFileLoadsWriteSchemaTransformProvider.java} (88%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageStreamBundleSource.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableConfigTranslator.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceFactory.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableWriteOptions.java
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/{changestreams/TimestampConverter.java
=> FixedCredentialFactory.java} (51%)
delete mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamMutation.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/BigtableChangeStreamAccessor.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/encoder/MetadataTableEncoder.java
rename
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformReadProviderTest.java
=> BigQueryExportReadSchemaTransformProviderTest.java} (82%)
rename
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQuerySchemaTransformWriteProviderTest.java
=> BigQueryFileLoadsWriteSchemaTransformProviderTest.java} (85%)
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOStorageReadWithStreamBundleSourceTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableConfigTranslatorTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ByteStringRangeHelperTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/ChangeStreamActionTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/DetectNewPartitionsActionTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/GenerateInitialPartitionsActionTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/ReadChangeStreamPartitionActionTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/MetadataTableAdminDaoTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/MetadataTableDaoTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/InitializeDoFnTest.java
create mode 100644
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/restriction/ReadChangeStreamPartitionProgressTrackerTest.java
create mode 100644
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md
create mode 100644
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/apache/beam/examples/MultiLangRunInference.java
copy sdks/python/apache_beam/{ml/gcp =>
examples/inference/multi_language_inference/multi_language_custom_transform/multi_language_custom_transform}/__init__.py
(99%)
create mode 100644
sdks/python/apache_beam/examples/inference/multi_language_inference/multi_language_custom_transform/multi_language_custom_transform/composite_transform.py
copy sdks/python/apache_beam/{ml/inference/tensorflow_tests_requirements.txt
=>
examples/inference/multi_language_inference/multi_language_custom_transform/setup.py}
(84%)
create mode 100644
sdks/python/apache_beam/examples/inference/xgboost_iris_classification.py
create mode 100644 sdks/python/apache_beam/ml/inference/xgboost_inference.py
create mode 100644
sdks/python/apache_beam/ml/inference/xgboost_inference_it_test.py
create mode 100644
sdks/python/apache_beam/ml/inference/xgboost_inference_test.py
copy sdks/python/apache_beam/ml/{gcp/__init__.py =>
inference/xgboost_tests_requirements.txt} (96%)
create mode 100644 website/www/site/content/en/blog/beam-2.46.0.md
create mode 100644 website/www/site/content/en/documentation/ml/about-ml.md
create mode 100644
website/www/site/content/en/documentation/ml/inference-overview.md
create mode 100644
website/www/site/content/en/documentation/ml/multi-language-inference.md
create mode 100644
website/www/site/content/en/documentation/transforms/python/elementwise/runinference-pytorch.md
create mode 100644
website/www/site/content/en/documentation/transforms/python/elementwise/runinference-sklearn.md
