This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a change to branch tvalentyn-patch-3
in repository https://gitbox.apache.org/repos/asf/beam.git
discard a5193922220 Further restrict allowed Pandas versions.
omit d118e4ffe39 Require Pandas 1.2.0 or above.
omit 1a5fe93d2bf Add a note to update build-requirements.txt
omit f2ce6cc04ba Restore the lower bound.
omit 18075f5c2e0 Update upper bound for numpy.
add 7ad44c84585 Handle schema updates in Storage API writes.
add f5020e7ac2b Merge pull request #24145: Handle updates to table schema
when using Storage API writes.
add 9cecec3f74c Bump cloud.google.com/go/storage from 1.28.1 to 1.29.0 in
/sdks (#25095)
add 5e1ebee8b47 Allow to set timeout for finishing a remote bundle in
Samza portable runner (#25031)
add e379c23c885 Fix truncate copy job when WRITE_TRUNCATE in BigQuery
batch load (#25101)
add 482401411b7 fix(sec): upgrade torch to 1.13.1 (#24933)
add 4dad3c696d4 [#24515] Delete the JRH (#24967)
add cd20288318d Support DoFn metrics in portable Samza Runner (#25068)
add 5cfc09cd65d Add I/O Standards Page (#24962)
add 897fffdeda5 Add more info in documentation (#25099)
add b3aa2e89489 Fix Python PreCommit dependency compatibility test not run
or redundant setup (#25115)
add 6029a57a4d7 Move CombineValues override to non portable overrides
(#25106)
add 54aad800535 Remove kileys from Java reviewers
add 459d6b2543a Merge pull request #25124 from kileys/remove
add 5e3b41788ab Exclude UsesMultimapState in Samza validatesPortableRunner
tests (#25121)
add 633c51242e9 Add GraalSystems (#23651)
add 93ca2fe53ce Bump timeout for default transient expansion service
(#25111)
add b1aba9d24e8 #20812 handle @RequiresStableInput in portable flink
(#22889)
add 6ba647333c4 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#25128)
add 6b8f8c09f5e Initial start of a yaml-based declarative way of building
pipelines. (#24667)
add 3d9318bff08 Remove deprecated first implementations of data channel
connector code on the runner side (#25104)
add 9aa2c529c51 Adding autoservice annotations to all
SchemaTransformProviders (#25107)
add 89cd0599708 Add working example for Per Entity Training (#25081)
add 860b05ff66e Add documentation on typescript IOs.
add d9145257b37 Merge pull request #25071 Restore documentation on
typescript IOs.
add 0bcb50edde5 Adds a reference x-lang RunInference Javadoc
add b7c07e26472 Merge pull request #25148: Adds a reference x-lang
RunInference Javadoc
add 299a7b01308 Drop '2' from BeamFnDataGrpcMultiplexer2 and
BeamFnDataInboundObserver2 (#25144)
add a96afe2c57c [BEAM-12164] Enable postgres IT for change streams (#25152)
add b49ef57ddd2 update cloudpickle version (#25142)
add 72d2776bf3a [ToB][Frontend] Gradle tasks for ToB frontend (#24919)
add 6d6f9958734 Remove databaseRole from metadata DB requests in
SpannerIO.ReadChangeStream (#25108) (#25141)
add d688d425938 Update typehint code to allow primitive composite types in
python 3.9+ (#25147)
add 91efbd3a933 Fix intruction -> instruction typo
add e8484a790e3 Fix inconsistent cache bug
add 1ecb4ef6d6f Merge pull request #25159: Fix inconsistent cache bug
add b0c594167b4 Increase default expansion service timeout
add 48712b26e67 Merge pull request #25169: Increase default expansion
service timeout
add d39e9c5779b Further increase transient expansion service timeout
(#25176)
add 0c2a1683e5f Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#25182)
add b497cc4a98a Adds the SampleDataRequest/Response (#25065)
add 76f3a5919fe Update chromedriver-binary requirement in /sdks/python
(#25178)
add 6676646c8b1 Bump google.golang.org/grpc from 1.52.0 to 1.52.3 in /sdks
(#25181)
add 67b5b368353 Initial commit of boilerplate setup of change stream
pipeline for bigtable
add f565a94b9b3 Merge pull request #25153 from tonytanger/initial_cdc
add 13a489efc9a Fix SchemaTransform identifiers
add 2c45ac4519b Merge pull request #25175: Fix SchemaTransform identifiers
add f07241b0983 Fix a couple typos caught by an internal linter (#25188)
add 61045b670bf Update Dataflow container versions (#25192)
add 40243ffc284 update GCP cloud libraries BOM to 26.5.0
add 3bf17574c6f Merge pull request #25194: update GCP cloud libraries BOM
to 26.5.0
add bfb6a8ea0a8 Fixing a table existence verifier for BQ
add 2bcec868da0 Merge pull request #25195 from pabloem/fix-verify-table-bq
add 1c71463118c Add a bunch of convenience methods for nullable top level
schema fields to make schemas a bit easier to read in Java
add 0b7394b4486 Updated to use existing addNullableField vs doing it
directly
add 423cf9aded6 Merge branch 'apache:master' into
schema-convenience-functions
add 2fbc731f5f6 Merge remote-tracking branch
'origin/schema-convenience-functions' into schema-convenience-functions
add c8cdc8ede70 Merge pull request #25196: Schema convenience functions
add c9e8aba3141 Log BatchElement statistics.
add bb92863be87 Merge pull request #25208 Log BatchElement statistics.
add 66aa607845b [24469] Implement CsvIO.Write and supporting classes
(#24630)
add af42717f461 Add some more links to the case studies page.
add cd54ad5ad67 Merge pull request #25125 Add some more links to the case
studies page.
add 2b96716ef5f Implement mongodbio.Read with an SDF (#25160)
add 6eef233d1c9 Fix website.
add ac117b97766 Fix Jdbc Write after window assigned (#25173)
add 5041714f136 Update Github issues link for Go SDK (#25161)
add 14e8de6e99a Fix typo. (#21864)
add 634b0453469 Standardizing naming and URN for Pubsub Read Schema
Transform (#25170)
add 05dc71a610f Attempt fix GCPIO_Direct tests timeout (#25209)
add b2d500f8494 Fix pulling licenses (#25234)
add f77366a8115 Ignore flags for beam_sql magic (#25210)
add cfe8005c502 Deterministic ordering of gbk outputs for testing.
add 3f825c39106 Increase timeouts for cross-langauge tests.
add 99999c6766b Merge pull request #25241 Fix typescript github action
failures.
add 9a6354879dd Stop publishing empty test-only artifacts (#25191)
add 957301519bb Fix Debezium expansion service fail to start (#25243)
add bc2895c99a2 Merge pull request #25094: Externalizing the StreamWriter
parameters for StorageWrites
add aae7ecd2441 Setup typedoc for doc generation.
add 9e373713cf8 Quick pass adding some typescript docs and pointers.
add 68b60deefa3 Add typescript doc gen to release process.
add d182f462a5d Split README into user-facing and dev-facing portions.
add 3883affd24e Merge branch 'master' into typescript-typedoc
add 8f7d2cc8eb9 Merge pull request #25069 Run typedoc to generate
typescript docs.
add b17c1c66921 Update/add torch versions to tox.ini (#25045)
add 7a6598f2c96 Add support for templates in task hints (#25214)
add af416a4cc4f Bump torch (#25057)
add fde7909c951 Fix XVR_Direct time out (#25247)
add f61c3fd32c2 Exclude changestream integration test on
DataflowLegacyWorker (#25239)
add 067e4db747b [Tour of Beam] [Task] Fix dependency management for 2.44
Playground java runner (#25230)
add d7264d4ba2a Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in
/sdks (#25270)
add 01aa470cf72 [Spark Runner] Add new experiment that provides concurrent
bounded output for SDFs (resolves #23852) (#24837)
add 10805a2d402 Add sideinputs to the RunInference Transform (#25200)
add 74de85513b9 Move changes to correct release in CHANGES.md (#25288)
add 3f8b8e383d7 Pass instead of raising an error (#25287)
add 24f40aab7aa [WebSite] Add new Python quickstart (#24804)
add 49f00ab5c8d [CdapIO] Implement windowed write (#25206)
add d50924d2ae9 [Spark Dataset runner] Fix collection encoder bug that may
lead to corrupted data due to naive usage of unsafe storage (fixes #25296).
(#25297)
add 72781cb089a [Spark Dataset runner] Break linage of dataset to reduce
Spark planning overhead in case of large complex query plans (relates to #24710
and #23845) (#25187)
add 86480f33397 Fix flaky test due to create bigquery dataset conflict
(#25266)
add b3be8b65aff 24472 Implement FileWriteSchemaTransformProvider (#24806)
add 0441d9d5823 Embed ML video to docs (#25302)
add d68b02804a5 skip automated expansion test (#25304)
add ed699f4504f Add banner highlighting beam ml (#25306)
add b1c9d8aec07 Optimize to use cached output receiver instead of creating
one on DoFn invocation #21250 (#25245)
add c65e777d6c6 Optimize PGBK table to only update cache when there is a
large enough size change. #21250 (#25219)
add 679d30256c6 Swap setting a context from being on the hot path when we
emit elements to only be done during bundle creation and teardown #21250
(#25291)
add 4e6a3e41aba Basic SchemaTransform implementation for SQLTransform.
(#25177)
add 16cb63be7e0 issue24170 google colab link added (#24820)
add aee2c844ad8 Clarify llm download/loading instructions (#25145)
add bb582d8db12 Allow for setMaxRetryJobs in BigQueryIO to be configurable
(#25224)
add fd4ea9b6bce delete file used in internal testing (#25339)
add bbe27e96de0 Don't discard output column names
add 9178456b38f Merge pull request #25290: Don't discard output column
names
add cd8eb0c9c68 Fix output timestamp for multi output receiver in
FnApiDoFnRunner #25344 (#25349)
add dc632029f0c Change UnboundedScheduledExecutorService to avoid creating
threads when (#25300)
add d20d0b01c3c Upgrading spring-expression to latest patch version
(#25348)
add e484b203991 Update upper bound for numpy.
add 9696df8bb84 Restore the lower bound.
add 035fbefa8d8 Add a note to update build-requirements.txt
add e878846cafd Require Pandas 1.2.0 or above.
add 0e24d973f41 Further restrict allowed Pandas versions.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a5193922220)
\
N -- N -- N refs/heads/tvalentyn-patch-3 (0e24d973f41)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/REVIEWERS.yml | 1 -
.github/workflows/build_playground_backend.yml | 9 -
.github/workflows/playground_deploy_backend.yml | 9 -
.../workflows/playground_examples_ci_reusable.yml | 6 +-
.github/workflows/typescript_tests.yml | 2 +-
.test-infra/jenkins/README.md | 1 +
...mmit_CrossLanguageValidatesRunner_Direct.groovy | 4 +-
...ommit_CrossLanguageValidatesRunner_Flink.groovy | 2 +
...ommit_CrossLanguageValidatesRunner_Samza.groovy | 2 +
...ommit_CrossLanguageValidatesRunner_Spark.groovy | 2 +
...mmit_CrossLanguageValidatesRunner_Spark3.groovy | 2 +
...mit_Java_ValidatesRunner_Dataflow_Java11.groovy | 2 +-
...mit_Java_ValidatesRunner_Dataflow_Java17.groovy | 2 +-
.test-infra/tools/stale_bq_datasets_cleaner.sh | 2 +-
CHANGES.md | 37 +-
build.gradle.kts | 3 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 34 +-
.../backend/internal/fs_content/load.go | 6 +-
.../backend/internal/fs_content/load_test.go | 2 +-
.../backend/samples/api/get_unit_content.json | 2 +-
.../module 1/group/unit-challenge/hint1.md | 2 +-
.../module 1/unit-challenge/hint1.md | 2 +-
.../module 2/unit-challenge/hint1.md | 2 +-
.../tour-of-beam/frontend/build.gradle.kts | 106 +-
.../beam/model/fn_execution/v1/beam_fn_api.proto | 64 +
.../beam/model/pipeline/v1/beam_runner_api.proto | 7 +
playground/backend/containers/java/Dockerfile | 55 +-
playground/backend/containers/java/build.gradle | 8 +-
.../frontend/assets/{github.svg => colab.svg} | 10 +-
playground/frontend/assets/github.svg | 4 +-
playground/frontend/assets/translations/en.yaml | 3 +
playground/frontend/build.gradle | 6 +-
.../lib/components/link_button/colab_button.dart | 37 +-
.../lib/components/link_button/dataset_button.dart | 35 +-
.../lib/components/link_button/github_button.dart | 28 +-
.../lib/components/link_button/link_button.dart | 63 +
playground/frontend/lib/l10n/app_en.arb | 4 -
.../description_popover/description_popover.dart | 30 +-
.../examples/components/example_actions.dart | 52 +
.../widgets/embedded_actions.dart | 58 +-
playground/frontend/lib/src/assets/assets.gen.dart | 2 +
.../lib/playground_components.dart | 1 +
.../lib/src/models/example.dart | 6 +-
.../lib/src/models/example_base.dart | 6 +-
.../example_client/grpc_example_client.dart | 5 +-
.../src/main/scripts/build_release_candidate.sh | 15 +
.../construction/SplittableParDoNaiveBounded.java | 3 +-
runners/flink/flink_runner.gradle | 1 -
.../utils/FlinkPortableRunnerUtils.java | 15 +
.../runners/flink/translation/utils/Locker.java} | 25 +-
.../wrappers/streaming/DoFnOperator.java | 100 +-
.../streaming/ExecutableStageDoFnOperator.java | 79 +-
.../streaming/stableinput/BufferedElements.java | 6 +
.../streaming/stableinput/BufferingDoFnRunner.java | 127 +-
.../flink/FlinkRequiresStableInputTest.java | 279 ++--
.../apache/beam/runners/flink/FlinkRunnerTest.java | 4 +-
.../beam/runners/flink/PortableExecutionTest.java | 1 +
.../wrappers/streaming/DoFnOperatorTest.java | 5 +-
.../streaming/ExecutableStageDoFnOperatorTest.java | 80 +-
runners/google-cloud-dataflow-java/build.gradle | 19 +-
.../examples-streaming/build.gradle | 10 +-
.../examples/build.gradle | 17 +-
.../google-cloud-dataflow-java/worker/build.gradle | 285 ++--
.../worker/legacy-worker/build.gradle | 276 ----
.../dataflow/worker/BatchDataflowWorker.java | 107 +-
.../worker/BeamFnMapTaskExecutorFactory.java | 799 -----------
.../worker/DataflowBatchWorkerHarness.java | 7 +
.../worker/DataflowMapTaskExecutorFactory.java | 9 -
.../dataflow/worker/DataflowRunnerHarness.java | 248 ----
...FetchAndFilterStreamingSideInputsOperation.java | 280 ----
.../dataflow/worker/FnApiWindowMappingFn.java | 314 ----
.../worker/GroupAlsoByWindowParDoFnFactory.java | 21 +-
.../worker/IntrinsicMapTaskExecutorFactory.java | 19 -
.../worker/NoOpSourceOperationExecutor.java | 68 -
.../dataflow/worker/SdkHarnessRegistries.java | 286 ----
.../dataflow/worker/SdkHarnessRegistry.java | 68 -
.../worker/SourceOperationExecutorFactory.java | 26 +-
.../dataflow/worker/StreamingDataflowWorker.java | 213 +--
.../worker/counters/CounterUpdateAggregators.java | 75 -
.../DistributionCounterUpdateAggregator.java | 68 -
.../counters/MeanCounterUpdateAggregator.java | 58 -
.../counters/SumCounterUpdateAggregator.java | 50 -
.../dataflow/worker/fn/BeamFnControlService.java | 125 --
.../worker/fn/control/BeamFnMapTaskExecutor.java | 646 ---------
.../control/DataflowSideInputHandlerFactory.java | 163 ---
...ntMonitoringInfoToCounterUpdateTransformer.java | 131 --
...meMonitoringInfoToCounterUpdateTransformer.java | 148 --
...piMonitoringInfoToCounterUpdateTransformer.java | 89 --
...ntMonitoringInfoToCounterUpdateTransformer.java | 136 --
.../fn/control/ProcessRemoteBundleOperation.java | 165 ---
.../control/RegisterAndProcessBundleOperation.java | 693 ---------
...onMonitoringInfoToCounterUpdateTransformer.java | 144 --
...erMonitoringInfoToCounterUpdateTransformer.java | 137 --
.../worker/fn/data/BeamFnDataGrpcService.java | 256 ----
.../fn/data/RemoteGrpcPortReadOperation.java | 118 --
.../fn/data/RemoteGrpcPortWriteOperation.java | 255 ----
.../worker/fn/logging/BeamFnLoggingService.java | 155 --
.../fn/stream/ServerStreamObserverFactory.java | 103 --
.../graph/CloneAmbiguousFlattensFunction.java | 149 --
.../graph/CreateExecutableStageNodeFunction.java | 602 --------
.../graph/CreateRegisterFnOperationFunction.java | 318 -----
.../graph/DeduceFlattenLocationsFunction.java | 328 -----
.../worker/graph/DeduceNodeLocationsFunction.java | 124 --
...nsertFetchAndFilterStreamingSideInputNodes.java | 176 ---
.../worker/graph/LengthPrefixUnknownCoders.java | 64 -
.../beam/runners/dataflow/worker/graph/Nodes.java | 114 --
.../worker/graph/RegisterNodeFunction.java | 622 --------
.../graph/RemoveFlattenInstructionsFunction.java | 83 --
.../graph/ReplacePgbkWithPrecombineFunction.java | 87 --
.../dataflow/worker/BatchDataflowWorkerTest.java | 24 +-
.../worker/DataflowBatchWorkerHarnessTest.java | 21 +-
.../dataflow/worker/FnApiWindowMappingFnTest.java | 182 ---
.../IntrinsicMapTaskExecutorFactoryTest.java | 8 -
.../worker/NoOpSourceOperationExecutorTest.java | 61 -
.../dataflow/worker/SdkHarnessRegistryTest.java | 122 --
.../worker/SourceOperationExecutorFactoryTest.java | 16 -
.../worker/StreamingDataflowWorkerTest.java | 2 -
.../counters/CounterUpdateAggregatorsTest.java | 96 --
.../DistributionCounterUpdateAggregatorTest.java | 72 -
.../counters/MeanCounterUpdateAggregatorTest.java | 66 -
.../counters/SumCounterUpdateAggregatorTest.java | 62 -
.../worker/fn/BeamFnControlServiceTest.java | 174 ---
.../fn/control/BeamFnMapTaskExecutorTest.java | 295 ----
.../DataflowSideInputHandlerFactoryTest.java | 173 ---
...nitoringInfoToCounterUpdateTransformerTest.java | 124 --
...nitoringInfoToCounterUpdateTransformerTest.java | 167 ---
...nitoringInfoToCounterUpdateTransformerTest.java | 95 --
...nitoringInfoToCounterUpdateTransformerTest.java | 133 --
.../RegisterAndProcessBundleOperationTest.java | 770 ----------
.../SingularProcessBundleProgressTrackerTest.java | 148 --
...nitoringInfoToCounterUpdateTransformerTest.java | 143 --
...nitoringInfoToCounterUpdateTransformerTest.java | 133 --
.../worker/fn/data/BeamFnDataGrpcServiceTest.java | 293 ----
.../fn/data/RemoteGrpcPortReadOperationTest.java | 157 --
.../fn/data/RemoteGrpcPortWriteOperationTest.java | 228 ---
.../fn/logging/BeamFnLoggingServiceTest.java | 235 ---
.../fn/stream/ServerStreamObserverFactoryTest.java | 79 --
.../graph/CloneAmbiguousFlattensFunctionTest.java | 389 -----
.../CreateRegisterFnOperationFunctionTest.java | 559 --------
.../graph/DeduceFlattenLocationsFunctionTest.java | 394 -----
.../graph/DeduceNodeLocationsFunctionTest.java | 324 -----
...tFetchAndFilterStreamingSideInputNodesTest.java | 259 ----
.../graph/LengthPrefixUnknownCodersTest.java | 91 +-
.../runners/dataflow/worker/graph/NodesTest.java | 86 --
.../RemoveFlattenInstructionsFunctionTest.java | 382 -----
.../ReplacePgbkWithPrecombineFunctionTest.java | 153 --
.../fnexecution/control/SdkHarnessClient.java | 135 +-
.../runners/fnexecution/data/FnDataService.java | 53 +-
.../runners/fnexecution/data/GrpcDataService.java | 53 +-
.../control/DefaultJobBundleFactoryTest.java | 6 +-
.../fnexecution/control/SdkHarnessClientTest.java | 259 ++--
.../fnexecution/data/GrpcDataServiceTest.java | 37 +-
runners/samza/job-server/build.gradle | 1 +
.../beam/runners/samza/SamzaPipelineOptions.java | 7 +
.../runners/samza/runtime/SamzaDoFnRunners.java | 70 +-
.../runtime/SamzaMetricsBundleProgressHandler.java | 154 ++
.../SamzaMetricsBundleProgressHandlerTest.java | 187 +++
.../samza/runtime/SdkHarnessDoFnRunnerTest.java} | 44 +-
.../translation/EvaluationContext.java | 10 +-
.../translation/PipelineTranslator.java | 97 +-
.../translation/TransformTranslator.java | 13 +-
.../batch/CombineGloballyTranslatorBatch.java | 4 +
.../batch/CombineGroupedValuesTranslatorBatch.java | 4 +
.../batch/CombinePerKeyTranslatorBatch.java | 4 +
.../translation/batch/FlattenTranslatorBatch.java | 4 +
.../batch/GroupByKeyTranslatorBatch.java | 5 +-
.../translation/batch/ImpulseTranslatorBatch.java | 4 +
.../translation/batch/ParDoTranslatorBatch.java | 6 +-
.../batch/ReadSourceTranslatorBatch.java | 7 +-
.../batch/ReshuffleTranslatorBatch.java | 8 +
.../batch/WindowAssignTranslatorBatch.java | 4 +
.../translation/helpers/EncoderHelpers.java | 19 +-
runners/spark/spark_runner.gradle | 41 +-
.../spark/translation/MultiDoFnFunction.java | 66 +-
.../spark/translation/SparkInputDataProcessor.java | 374 +++++
.../spark/translation/SparkProcessContext.java | 149 +-
.../spark/translation/TransformTranslator.java | 16 +-
.../streaming/StreamingTransformTranslator.java | 3 +-
.../translation/SparkInputDataProcessorTest.java | 278 ++++
sdks/go.mod | 14 +-
sdks/go.sum | 28 +-
.../go/pkg/beam/core/runtime/xlangx/expand_test.go | 39 -
sdks/go/pkg/beam/io/mongodbio/coder.go | 25 +-
sdks/go/pkg/beam/io/mongodbio/coder_test.go | 135 +-
sdks/go/pkg/beam/io/mongodbio/common.go | 38 +-
.../pkg/beam/io/mongodbio/id_range_restriction.go | 206 +++
.../beam/io/mongodbio/id_range_restriction_test.go | 179 +++
sdks/go/pkg/beam/io/mongodbio/id_range_split.go | 248 ++++
.../pkg/beam/io/mongodbio/id_range_split_test.go | 275 ++++
sdks/go/pkg/beam/io/mongodbio/id_range_tracker.go | 194 +++
.../pkg/beam/io/mongodbio/id_range_tracker_test.go | 461 ++++++
sdks/go/pkg/beam/io/mongodbio/read.go | 402 ++----
sdks/go/pkg/beam/io/mongodbio/read_test.go | 323 -----
.../container/license_scripts/dep_urls_java.yaml | 2 +-
.../java/org/apache/beam/sdk/schemas/Schema.java | 65 +
.../beam/sdk/transforms/DoFnOutputReceivers.java | 2 +-
.../util/UnboundedScheduledExecutorService.java | 28 +-
.../org/apache/beam/sdk/RequiresStableInputIT.java | 6 +-
.../UnboundedScheduledExecutorServiceTest.java | 54 +
.../extensions/python/PythonExternalTransform.java | 5 +-
.../python/PythonExternalTransformTest.java | 1 +
.../extensions/sql/expansion-service/build.gradle | 1 +
.../SqlTransformSchemaTransformProvider.java | 232 +++
sdks/java/extensions/sql/perf-tests/build.gradle | 1 +
.../beam/sdk/extensions/sql/SqlTransform.java | 3 +-
.../extensions/sql/impl/planner/BeamRuleSets.java | 4 -
.../sql/impl/rel/AbstractBeamCalcRel.java | 10 +
.../sql/impl/rule/LogicalCalcMergeRule.java | 42 -
.../extensions/sql/BeamSqlMultipleSchemasTest.java | 21 +
.../sql/impl/rule/IOPushDownRuleTest.java | 2 -
...tTableProviderWithFilterAndProjectPushDown.java | 2 -
.../test/TestTableProviderWithFilterPushDown.java | 2 -
.../test/TestTableProviderWithProjectPushDown.java | 2 -
.../sql/zetasql/ZetaSQLQueryPlanner.java | 15 +-
sdks/java/extensions/zetasketch/build.gradle | 6 +-
.../sdk/fn/data/BeamFnDataGrpcMultiplexer.java | 224 +--
.../sdk/fn/data/BeamFnDataGrpcMultiplexer2.java | 267 ----
.../sdk/fn/data/BeamFnDataInboundObserver.java | 248 +++-
.../sdk/fn/data/BeamFnDataInboundObserver2.java | 241 ----
.../sdk/fn/data/BeamFnDataOutboundObserver.java | 87 --
.../data/CompletableFutureInboundDataClient.java | 78 -
.../apache/beam/sdk/fn/data/InboundDataClient.java | 60 -
.../fn/data/BeamFnDataGrpcMultiplexer2Test.java | 363 -----
.../sdk/fn/data/BeamFnDataGrpcMultiplexerTest.java | 316 ++++-
.../fn/data/BeamFnDataInboundObserver2Test.java | 247 ----
.../sdk/fn/data/BeamFnDataInboundObserverTest.java | 217 ++-
.../CompletableFutureInboundDataClientTest.java | 163 ---
.../jmh/PrecombineGroupingTableBenchmark.java | 87 +-
.../java/org/apache/beam/fn/harness/Caches.java | 43 +-
.../apache/beam/fn/harness/FnApiDoFnRunner.java | 309 +++-
.../beam/fn/harness/PrecombineGroupingTable.java | 10 +-
.../fn/harness/control/ExecutionStateSampler.java | 109 +-
.../fn/harness/control/ProcessBundleHandler.java | 67 +-
.../beam/fn/harness/data/BeamFnDataClient.java | 9 +-
.../beam/fn/harness/data/BeamFnDataGrpcClient.java | 12 +-
.../harness/data/PCollectionConsumerRegistry.java | 74 +-
.../harness/data/PTransformFunctionRegistry.java | 26 +-
.../fn/harness/PrecombineGroupingTableTest.java | 33 +-
.../harness/control/ExecutionStateSamplerTest.java | 144 ++
.../harness/control/ProcessBundleHandlerTest.java | 11 +-
.../fn/harness/data/BeamFnDataGrpcClientTest.java | 14 +-
.../data/PCollectionConsumerRegistryTest.java | 145 +-
.../data/PTransformFunctionRegistryTest.java | 81 +-
.../java/org/apache/beam/sdk/io/cdap/CdapIO.java | 44 +-
.../org/apache/beam/sdk/io/cdap/CdapIOTest.java | 36 +
.../beam/sdk/io/common/SchemaAwareJavaBeans.java | 494 +++++++
.../sdk/io/common/SchemaAwareJavaBeansTest.java | 175 +++
.../io/{fileschematransform => csv}/build.gradle | 26 +-
.../java/org/apache/beam/sdk/io/csv/CsvIO.java | 458 ++++++
.../apache/beam/sdk/io/csv/CsvRowConversions.java | 152 ++
.../org/apache/beam/sdk/io/csv}/package-info.java | 4 +-
.../org/apache/beam/sdk/io/csv/CsvIOTestData.java | 106 ++
.../apache/beam/sdk/io/csv/CsvIOTestJavaBeans.java | 265 ++++
.../org/apache/beam/sdk/io/csv/CsvIOWriteTest.java | 308 ++++
.../beam/sdk/io/csv/RowToCsvCSVFormatTest.java | 353 +++++
.../io/csv/RowToCsvPredefinedCSVFormatsTest.java | 284 ++++
.../DebeziumReadSchemaTransformProvider.java | 5 +-
.../OWNERS | 3 +
.../build.gradle | 20 +-
.../AvroWriteSchemaTransformFormatProvider.java | 88 ++
.../CsvWriteSchemaTransformFormatProvider.java} | 31 +-
.../FileWriteSchemaTransformConfiguration.java | 66 +-
.../FileWriteSchemaTransformFormatProvider.java | 16 +-
.../FileWriteSchemaTransformFormatProviders.java | 148 ++
.../FileWriteSchemaTransformProvider.java | 191 +++
.../JsonWriteSchemaTransformFormatProvider.java | 93 ++
.../ParquetWriteSchemaTransformFormatProvider.java | 117 ++
.../fileschematransform/XmlDateTimeAdapter.java} | 36 +-
.../sdk/io/fileschematransform/XmlRowAdapter.java | 93 ++
.../sdk/io/fileschematransform/XmlRowValue.java | 268 ++++
.../XmlWriteSchemaTransformFormatProvider.java | 115 ++
.../sdk/io/fileschematransform/package-info.java | 0
...FileWriteSchemaTransformFormatProviderTest.java | 104 ++
...FileWriteSchemaTransformFormatProviderTest.java | 434 ++++++
...WriteSchemaTransformFormatProviderTestData.java | 231 +++
...ileWriteSchemaTransformFormatProvidersTest.java | 17 +-
.../FileWriteSchemaTransformProviderTest.java | 127 ++
...FileWriteSchemaTransformFormatProviderTest.java | 104 ++
...FileWriteSchemaTransformFormatProviderTest.java | 112 ++
.../XmlDateTimeAdapterTest.java} | 32 +-
...FileWriteSchemaTransformFormatProviderTest.java | 120 ++
.../io/fileschematransform/XmlRowAdapterTest.java | 190 +++
.../io/fileschematransform/XmlRowValueTest.java | 391 +++++
.../FileWriteSchemaTransformFormatProviders.java | 124 --
.../beam/sdk/io/gcp/bigquery/AppendClientInfo.java | 120 +-
.../beam/sdk/io/gcp/bigquery/BigQueryHelpers.java | 9 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 30 +-
.../beam/sdk/io/gcp/bigquery/BigQueryOptions.java | 13 +
.../beam/sdk/io/gcp/bigquery/BigQueryServices.java | 8 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 21 +
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 29 -
.../sdk/io/gcp/bigquery/SplittingIterable.java | 61 +-
.../bigquery/StorageApiDynamicDestinations.java | 2 +
.../StorageApiDynamicDestinationsBeamRow.java | 8 +-
.../StorageApiDynamicDestinationsTableRow.java | 24 +-
.../beam/sdk/io/gcp/bigquery/StorageApiLoads.java | 20 +-
.../io/gcp/bigquery/StorageApiWritePayload.java | 25 +-
.../StorageApiWriteRecordsInconsistent.java | 12 +-
.../bigquery/StorageApiWriteUnshardedRecords.java | 146 +-
.../bigquery/StorageApiWritesShardedRecords.java | 173 ++-
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 110 +-
.../gcp/bigquery/WriteGroupedRecordsToFiles.java | 3 +-
.../BigQueryDirectReadSchemaTransformProvider.java | 2 +-
.../beam/sdk/io/gcp/bigtable/BigtableConfig.java | 38 +-
.../beam/sdk/io/gcp/bigtable/BigtableIO.java | 355 +++++
.../changestreams/ByteStringRangeHelper.java | 26 +-
.../changestreams/ChangeStreamMetrics.java | 12 +-
.../changestreams/ChangeStreamMutation.java} | 8 +-
.../changestreams/TimestampConverter.java} | 28 +-
.../bigtable/changestreams/UniqueIdGenerator.java | 30 +-
.../changestreams/action/ActionFactory.java | 121 ++
.../changestreams/action/ChangeStreamAction.java | 105 ++
.../action/DetectNewPartitionsAction.java | 118 ++
.../action/GenerateInitialPartitionsAction.java | 58 +
.../action/ReadChangeStreamPartitionAction.java | 123 ++
.../changestreams/action}/package-info.java | 6 +-
.../changestreams/dao/ChangeStreamDao.java | 18 +-
.../gcp/bigtable/changestreams/dao/DaoFactory.java | 113 ++
.../changestreams/dao/MetadataTableAdminDao.java | 82 ++
.../changestreams/dao/MetadataTableDao.java | 77 +
.../bigtable/changestreams/dao}/package-info.java | 6 +-
.../dofn/DetectNewPartitionsDoFn.java | 112 ++
.../changestreams/dofn/InitializeDoFn.java | 59 +
.../dofn/ReadChangeStreamPartitionDoFn.java | 118 ++
.../bigtable/changestreams/dofn}/package-info.java | 6 +-
.../changestreams/encoder}/package-info.java | 10 +-
.../changestreams/model/PartitionRecord.java | 135 ++
.../changestreams/model}/package-info.java | 6 +-
.../gcp/bigtable/changestreams}/package-info.java | 6 +-
.../ReadChangeStreamPartitionProgressTracker.java | 160 +++
.../changestreams/restriction/StreamProgress.java | 24 +-
.../changestreams/restriction}/package-info.java | 6 +-
...=> PubsubReadSchemaTransformConfiguration.java} | 10 +-
...java => PubsubReadSchemaTransformProvider.java} | 38 +-
.../PubsubSchemaTransformMessageToRowFactory.java | 18 +-
.../pubsub/PubsubWriteSchemaTransformProvider.java | 2 +
.../PubsubLiteReadSchemaTransformProvider.java | 3 +
.../PubsubLiteWriteSchemaTransformProvider.java | 3 +
.../apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 1 +
.../sdk/io/gcp/testing/FakeDatasetService.java | 63 +-
.../sdk/io/gcp/bigquery/BigQueryIOReadTest.java | 3 +
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 257 +++-
.../bigquery/TableRowToStorageApiProtoTest.java | 95 +-
.../changestreams/TimestampConverterTest.java} | 37 +-
.../beam/sdk/io/gcp/pubsub/PubsubIOTest.java | 2 +
... => PubsubReadSchemaTransformProviderTest.java} | 62 +-
...bsubSchemaTransformMessageToRowFactoryTest.java | 28 +-
.../SpannerChangeStreamErrorTest.java | 3 +
.../it/SpannerChangeStreamPostgresIT.java | 2 -
.../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 21 +-
sdks/java/io/kafka/build.gradle | 2 +-
.../kafka/KafkaWriteSchemaTransformProvider.java | 3 +
.../SingleStoreSchemaTransformReadProvider.java | 3 +
.../SingleStoreSchemaTransformWriteProvider.java | 3 +
sdks/java/testing/expansion-service/build.gradle | 1 +
sdks/java/testing/kafka-service/build.gradle | 4 +-
sdks/java/testing/load-tests/build.gradle | 6 +-
sdks/java/testing/nexmark/build.gradle | 6 +-
sdks/java/testing/tpcds/build.gradle | 6 +-
sdks/java/testing/watermarks/build.gradle | 6 +-
.../inference/large_language_modeling/main.py | 4 +-
.../online_clustering/clustering_pipeline/setup.py | 2 +-
.../inference/run_inference_side_inputs.py | 165 +++
.../kfp/components/preprocessing/requirements.txt | 2 +-
.../kfp/components/train/requirements.txt | 2 +-
.../apache_beam/examples/per_entity_training.py | 152 ++
.../transforms/elementwise/runinference_test.py | 45 +-
.../io/gcp/big_query_query_to_table_it_test.py | 6 +-
.../apache_beam/io/gcp/bigquery_file_loads.py | 2 +-
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 49 +-
.../apache_beam/io/gcp/bigquery_read_it_test.py | 10 +-
sdks/python/apache_beam/io/gcp/bigquery_test.py | 14 +-
.../apache_beam/io/gcp/bigquery_write_it_test.py | 8 +-
sdks/python/apache_beam/io/gcp/tests/utils.py | 6 +-
sdks/python/apache_beam/ml/inference/base.py | 161 ++-
sdks/python/apache_beam/ml/inference/base_test.py | 177 ++-
.../apache_beam/ml/inference/pytorch_inference.py | 77 +-
.../ml/inference/pytorch_inference_test.py | 5 +-
.../apache_beam/ml/inference/sklearn_inference.py | 37 +-
.../apache_beam/ml/inference/tensorrt_inference.py | 13 +-
sdks/python/apache_beam/ml/inference/utils.py | 48 +
.../runners/dataflow/dataflow_runner.py | 2 +-
.../apache_beam/runners/dataflow/internal/names.py | 4 +-
.../apache_beam/runners/interactive/sql/utils.py | 5 +-
.../runners/portability/flink_runner_test.py | 20 +-
.../apache_beam/transforms/combiners_test.py | 18 +
sdks/python/apache_beam/transforms/external.py | 3 +
sdks/python/apache_beam/transforms/util.py | 14 +
.../typehints/native_type_compatibility_test.py | 24 +-
sdks/python/apache_beam/typehints/typehints.py | 12 +-
.../python/apache_beam/typehints/typehints_test.py | 42 +-
.../train/requirements.txt => yaml/__init__.py} | 6 +-
sdks/python/apache_beam/yaml/main.py | 63 +
sdks/python/apache_beam/yaml/pipeline.schema.yaml | 130 ++
.../standard_providers.yaml} | 13 +-
sdks/python/apache_beam/yaml/yaml_provider.py | 437 ++++++
sdks/python/apache_beam/yaml/yaml_transform.py | 450 ++++++
.../python/apache_beam/yaml/yaml_transform_test.py | 90 ++
.../container/license_scripts/dep_urls_py.yaml | 2 +-
.../container/py310/base_image_requirements.txt | 32 +-
.../container/py37/base_image_requirements.txt | 35 +-
.../container/py38/base_image_requirements.txt | 39 +-
.../container/py39/base_image_requirements.txt | 39 +-
sdks/python/setup.py | 6 +-
sdks/python/test-suites/direct/common.gradle | 2 +-
sdks/python/test-suites/tox/py38/build.gradle | 50 +-
sdks/python/tox.ini | 5 +-
sdks/typescript/{README.md => README-dev.md} | 89 +-
sdks/typescript/README.md | 152 +-
sdks/typescript/package-lock.json | 177 ++-
sdks/typescript/package.json | 4 +-
sdks/typescript/src/apache_beam/coders/coders.ts | 1 +
.../typescript/src/apache_beam/coders/js_coders.ts | 7 +
.../typescript/src/apache_beam/coders/row_coder.ts | 20 +-
.../src/apache_beam/examples/wordcount_textio.ts | 2 +-
sdks/typescript/src/apache_beam/pvalue.ts | 9 +-
.../src/apache_beam/runners/direct_runner.ts | 2 +-
sdks/typescript/src/apache_beam/runners/runner.ts | 16 +-
sdks/typescript/src/apache_beam/testing/assert.ts | 23 +
.../apache_beam/testing/multi_pipeline_runner.ts | 19 +
.../src/apache_beam/transforms/external.ts | 9 +
.../src/apache_beam/transforms/flatten.ts | 7 +
.../apache_beam/transforms/group_and_combine.ts | 40 +-
.../src/apache_beam/transforms/internal.ts | 14 +
.../typescript/src/apache_beam/transforms/pardo.ts | 62 +-
.../src/apache_beam/transforms/python.ts | 35 +
.../src/apache_beam/transforms/transform.ts | 50 +
.../src/apache_beam/transforms/window.ts | 8 +
sdks/typescript/src/apache_beam/utils/utils.ts | 2 +
sdks/typescript/test/primitives_test.ts | 14 +-
sdks/typescript/test/wordcount.ts | 6 +-
sdks/typescript/typedoc.json | 24 +
settings.gradle.kts | 7 +-
website/www/site/assets/scss/_banner.sass | 43 +
website/www/site/assets/scss/_hero.scss | 3 +-
website/www/site/assets/scss/_navbar-desktop.scss | 3 +-
website/www/site/assets/scss/main.scss | 1 +
.../www/site/content/en/case-studies/163netease.md | 19 +
.../www/site/content/en/case-studies/accenture.md | 19 +
website/www/site/content/en/case-studies/align.md | 19 +
website/www/site/content/en/case-studies/aliz.md | 19 +
.../www/site/content/en/case-studies/arquivei.md | 19 +
.../site/content/en/case-studies/bahwancybertek.md | 19 +
website/www/site/content/en/case-studies/bbc.md | 19 +
website/www/site/content/en/case-studies/behalf.md | 19 +
.../www/site/content/en/case-studies/belllabs.md | 19 +
.../www/site/content/en/case-studies/benchsci.md | 19 +
.../www/site/content/en/case-studies/betterup.md | 19 +
.../content/en/case-studies/bigdatainstitute.md | 19 +
.../site/content/en/case-studies/carfinance247.md | 19 +
.../www/site/content/en/case-studies/citibank.md | 19 +
.../www/site/content/en/case-studies/cognite.md | 19 +
website/www/site/content/en/case-studies/cruise.md | 19 +
.../www/site/content/en/case-studies/datatonic.md | 19 +
.../site/content/en/case-studies/dunbradstreet.md | 19 +
.../www/site/content/en/case-studies/ericsson.md | 19 +
.../www/site/content/en/case-studies/evolve24.md | 19 +
.../site/content/en/case-studies/graalsystems.md | 21 +
.../www/site/content/en/case-studies/hazelcast.md | 19 +
.../www/site/content/en/case-studies/hoxtonai.md | 19 +
.../www/site/content/en/case-studies/ibiblio.md | 19 +
.../industrialtechnologyresearchinstitute.md | 19 +
website/www/site/content/en/case-studies/ineat.md | 19 +
.../www/site/content/en/case-studies/lumaapps.md | 19 +
.../www/site/content/en/case-studies/mindvalley.md | 19 +
.../www/site/content/en/case-studies/natwest.md | 19 +
website/www/site/content/en/case-studies/nokia.md | 19 +
website/www/site/content/en/case-studies/oracle.md | 19 +
.../site/content/en/case-studies/orielresearch.md | 19 +
.../en/case-studies/oxfordbigdatainstitute.md | 19 +
.../www/site/content/en/case-studies/plantix.md | 19 +
.../content/en/case-studies/portaltelemedicina.md | 19 +
.../www/site/content/en/case-studies/primerai.md | 19 +
.../www/site/content/en/case-studies/prosimo.md | 19 +
.../www/site/content/en/case-studies/spicule.md | 19 +
.../www/site/content/en/case-studies/tencent.md | 19 +
.../site/content/en/case-studies/theworkshop.md | 19 +
.../www/site/content/en/case-studies/tutanota.md | 19 +
website/www/site/content/en/case-studies/unity.md | 19 +
.../content/en/case-studies/universityofweimar.md | 19 +
website/www/site/content/en/case-studies/veepee.md | 19 +
.../site/content/en/case-studies/venturedive.md | 19 +
website/www/site/content/en/case-studies/veolia.md | 19 +
.../site/content/en/case-studies/xiaohongshu.md | 19 +
website/www/site/content/en/case-studies/yelp.md | 19 +
.../site/content/en/documentation/io/connectors.md | 96 +-
.../content/en/documentation/io/io-standards.md | 1500 ++++++++++++++++++++
.../en/documentation/ml/large-language-modeling.md | 26 +-
.../site/content/en/documentation/ml/overview.md | 5 +-
.../en/documentation/ml/per-entity-training.md | 64 +
.../www/site/content/en/documentation/sdks/go.md | 2 +-
.../documentation/sdks/python-machine-learning.md | 18 +-
.../transforms/java/elementwise/withkeys.md | 2 +-
.../site/content/en/get-started/quickstart-py.md | 31 +-
.../content/en/get-started/quickstart/python.md | 226 +++
.../get-started/resources/videos-and-podcasts.md | 7 +
website/www/site/layouts/partials/header.html | 6 +
.../partials/section-menu/en/documentation.html | 2 +
.../partials/section-menu/en/get-started.html | 8 +-
website/www/site/static/images/banner_desktop.jpg | Bin 0 -> 111671 bytes
website/www/site/static/images/banner_mobile.jpg | Bin 0 -> 59518 bytes
.../io-connector-guides-screenshot.png | Bin 0 -> 66976 bytes
...ector-performance-test-dashboard-screenshot.png | Bin 0 -> 262230 bytes
.../io-connector-relational-features-table.png | Bin 0 -> 36427 bytes
.../io-standards/io-supported-via-screenshot.png | Bin 0 -> 23580 bytes
.../io-standards/website-built-in-screenshot.png | Bin 0 -> 38992 bytes
.../static/images/logos/powered-by/163netease.png | Bin 0 -> 24509 bytes
.../static/images/logos/powered-by/accenture.png | Bin 0 -> 23000 bytes
.../site/static/images/logos/powered-by/align.png | Bin 0 -> 11914 bytes
.../site/static/images/logos/powered-by/aliz.png | Bin 0 -> 3789 bytes
.../static/images/logos/powered-by/arquivei.png | Bin 0 -> 8031 bytes
.../images/logos/powered-by/bahwancybertek.png | Bin 0 -> 29002 bytes
.../site/static/images/logos/powered-by/bbc.png | Bin 0 -> 8261 bytes
.../site/static/images/logos/powered-by/behalf.png | Bin 0 -> 4748 bytes
.../static/images/logos/powered-by/belllabs.png | Bin 0 -> 22337 bytes
.../static/images/logos/powered-by/benchsci.png | Bin 0 -> 12922 bytes
.../static/images/logos/powered-by/betterup.png | Bin 0 -> 3449 bytes
.../images/logos/powered-by/bigdatainstitute.png | Bin 0 -> 47223 bytes
.../images/logos/powered-by/carfinance247.png | Bin 0 -> 42883 bytes
.../static/images/logos/powered-by/citibank.png | Bin 0 -> 26340 bytes
.../static/images/logos/powered-by/cognite.png | Bin 0 -> 3470 bytes
.../site/static/images/logos/powered-by/cruise.png | Bin 0 -> 49411 bytes
.../static/images/logos/powered-by/datatonic.png | Bin 0 -> 3262 bytes
.../images/logos/powered-by/dunbradstreet.png | Bin 0 -> 6396 bytes
.../static/images/logos/powered-by/ericsson.png | Bin 0 -> 32216 bytes
.../static/images/logos/powered-by/evolve24.png | Bin 0 -> 18350 bytes
.../images/logos/powered-by/graalsystems.png | Bin 0 -> 11402 bytes
.../static/images/logos/powered-by/hazelcast.png | Bin 0 -> 9274 bytes
.../static/images/logos/powered-by/hoxtonai.png | Bin 0 -> 6177 bytes
.../static/images/logos/powered-by/ibiblio.png | Bin 0 -> 2887 bytes
.../industrialtechnologyresearchinstitute.png | Bin 0 -> 14001 bytes
.../site/static/images/logos/powered-by/ineat.png | Bin 0 -> 11809 bytes
.../static/images/logos/powered-by/lumaapps.png | Bin 0 -> 17104 bytes
.../static/images/logos/powered-by/mindvalley.png | Bin 0 -> 51538 bytes
.../static/images/logos/powered-by/natwest.png | Bin 0 -> 14297 bytes
.../site/static/images/logos/powered-by/nokia.png | Bin 0 -> 43679 bytes
.../site/static/images/logos/powered-by/oracle.png | Bin 0 -> 1220 bytes
.../images/logos/powered-by/orielresearch.png | Bin 0 -> 10109 bytes
.../logos/powered-by/oxfordbigdatainstitute.png | Bin 0 -> 47223 bytes
.../static/images/logos/powered-by/plantix.png | Bin 0 -> 67805 bytes
.../images/logos/powered-by/portaltelemedicina.png | Bin 0 -> 26046 bytes
.../static/images/logos/powered-by/primerai.png | Bin 0 -> 30330 bytes
.../static/images/logos/powered-by/prosimo.png | Bin 0 -> 12322 bytes
.../static/images/logos/powered-by/spicule.png | Bin 0 -> 18340 bytes
.../static/images/logos/powered-by/tencent.png | Bin 0 -> 108931 bytes
.../static/images/logos/powered-by/theworkshop.png | Bin 0 -> 7525 bytes
.../static/images/logos/powered-by/tutanota.png | Bin 0 -> 13434 bytes
.../site/static/images/logos/powered-by/unity.png | Bin 0 -> 17667 bytes
.../images/logos/powered-by/universityofweimar.png | Bin 0 -> 30417 bytes
.../site/static/images/logos/powered-by/veepee.png | Bin 0 -> 20971 bytes
.../static/images/logos/powered-by/venturedive.png | Bin 0 -> 9611 bytes
.../site/static/images/logos/powered-by/veolia.png | Bin 0 -> 15877 bytes
.../static/images/logos/powered-by/xiaohongshu.png | Bin 0 -> 5764 bytes
.../site/static/images/logos/powered-by/yelp.png | Bin 0 -> 150390 bytes
554 files changed, 21167 insertions(+), 21084 deletions(-)
copy playground/frontend/build.gradle =>
learning/tour-of-beam/frontend/build.gradle.kts (51%)
copy playground/frontend/assets/{github.svg => colab.svg} (53%)
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/MonitoringInfoToCounterUpdateTransformer.java
=> playground/frontend/lib/components/link_button/colab_button.dart (57%)
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritePayload.java
=> playground/frontend/lib/components/link_button/dataset_button.dart (50%)
copy
runners/google-cloud-dataflow-java/worker/src/main/java/com/google/cloud/dataflow/worker/DataflowRunnerHarness.java
=> playground/frontend/lib/components/link_button/github_button.dart (59%)
create mode 100644
playground/frontend/lib/components/link_button/link_button.dart
create mode 100644
playground/frontend/lib/modules/examples/components/example_actions.dart
copy
runners/{google-cloud-dataflow-java/worker/src/main/java/com/google/cloud/dataflow/worker/DataflowRunnerHarness.java
=>
flink/src/main/java/org/apache/beam/runners/flink/translation/utils/Locker.java}
(67%)
delete mode 100644
runners/google-cloud-dataflow-java/worker/legacy-worker/build.gradle
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/BeamFnMapTaskExecutorFactory.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/DataflowRunnerHarness.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/FetchAndFilterStreamingSideInputsOperation.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/FnApiWindowMappingFn.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/NoOpSourceOperationExecutor.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/SdkHarnessRegistries.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/SdkHarnessRegistry.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/counters/CounterUpdateAggregators.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/counters/DistributionCounterUpdateAggregator.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/counters/MeanCounterUpdateAggregator.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/counters/SumCounterUpdateAggregator.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/BeamFnControlService.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/BeamFnMapTaskExecutor.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/DataflowSideInputHandlerFactory.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/ElementCountMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/ExecutionTimeMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/FnApiMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/MeanByteCountMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/ProcessRemoteBundleOperation.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/RegisterAndProcessBundleOperation.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/UserDistributionMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/UserMonitoringInfoToCounterUpdateTransformer.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/data/BeamFnDataGrpcService.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/data/RemoteGrpcPortReadOperation.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/data/RemoteGrpcPortWriteOperation.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/logging/BeamFnLoggingService.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/stream/ServerStreamObserverFactory.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/CloneAmbiguousFlattensFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/CreateExecutableStageNodeFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/CreateRegisterFnOperationFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/DeduceFlattenLocationsFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/DeduceNodeLocationsFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/InsertFetchAndFilterStreamingSideInputNodes.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/RegisterNodeFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/RemoveFlattenInstructionsFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/graph/ReplacePgbkWithPrecombineFunction.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/FnApiWindowMappingFnTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/NoOpSourceOperationExecutorTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/SdkHarnessRegistryTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/counters/CounterUpdateAggregatorsTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/counters/DistributionCounterUpdateAggregatorTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/counters/MeanCounterUpdateAggregatorTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/counters/SumCounterUpdateAggregatorTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/BeamFnControlServiceTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/BeamFnMapTaskExecutorTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/DataflowSideInputHandlerFactoryTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/ElementCountMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/ExecutionTimeMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/FnApiMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/MeanByteCountMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/RegisterAndProcessBundleOperationTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/SingularProcessBundleProgressTrackerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/UserDistributionMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/control/UserMonitoringInfoToCounterUpdateTransformerTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/data/BeamFnDataGrpcServiceTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/data/RemoteGrpcPortReadOperationTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/data/RemoteGrpcPortWriteOperationTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/logging/BeamFnLoggingServiceTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/fn/stream/ServerStreamObserverFactoryTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/CloneAmbiguousFlattensFunctionTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/CreateRegisterFnOperationFunctionTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/DeduceFlattenLocationsFunctionTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/DeduceNodeLocationsFunctionTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/InsertFetchAndFilterStreamingSideInputNodesTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/RemoveFlattenInstructionsFunctionTest.java
delete mode 100644
runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/graph/ReplacePgbkWithPrecombineFunctionTest.java
create mode 100644
runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/SamzaMetricsBundleProgressHandler.java
create mode 100644
runners/samza/src/test/java/org/apache/beam/runners/samza/runtime/SamzaMetricsBundleProgressHandlerTest.java
rename
runners/{google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/counters/CounterUpdateAggregator.java
=>
samza/src/test/java/org/apache/beam/runners/samza/runtime/SdkHarnessDoFnRunnerTest.java}
(51%)
create mode 100644
runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkInputDataProcessor.java
create mode 100644
runners/spark/src/test/java/org/apache/beam/runners/spark/translation/SparkInputDataProcessorTest.java
delete mode 100644 sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_restriction.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_restriction_test.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_split.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_split_test.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_tracker.go
create mode 100644 sdks/go/pkg/beam/io/mongodbio/id_range_tracker_test.go
create mode 100644
sdks/java/extensions/sql/expansion-service/src/main/java/org/apache/beam/sdk/extensions/sql/expansion/SqlTransformSchemaTransformProvider.java
delete mode 100644
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/LogicalCalcMergeRule.java
delete mode 100644
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataGrpcMultiplexer2.java
delete mode 100644
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataInboundObserver2.java
delete mode 100644
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataOutboundObserver.java
delete mode 100644
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/CompletableFutureInboundDataClient.java
delete mode 100644
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/InboundDataClient.java
delete mode 100644
sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/data/BeamFnDataGrpcMultiplexer2Test.java
delete mode 100644
sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/data/BeamFnDataInboundObserver2Test.java
delete mode 100644
sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/data/CompletableFutureInboundDataClientTest.java
create mode 100644
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeans.java
create mode 100644
sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/SchemaAwareJavaBeansTest.java
copy sdks/java/io/{fileschematransform => csv}/build.gradle (62%)
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvIO.java
create mode 100644
sdks/java/io/csv/src/main/java/org/apache/beam/sdk/io/csv/CsvRowConversions.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=> csv/src/main/java/org/apache/beam/sdk/io/csv}/package-info.java (85%)
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOTestData.java
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOTestJavaBeans.java
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/CsvIOWriteTest.java
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/RowToCsvCSVFormatTest.java
create mode 100644
sdks/java/io/csv/src/test/java/org/apache/beam/sdk/io/csv/RowToCsvPredefinedCSVFormatsTest.java
rename sdks/java/io/{fileschematransform => file-schema-transform}/OWNERS (55%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/build.gradle (70%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/AvroWriteSchemaTransformFormatProvider.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvider.java
=>
file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/CsvWriteSchemaTransformFormatProvider.java}
(56%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformConfiguration.java
(77%)
rename sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvider.java
(66%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviders.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProvider.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/JsonWriteSchemaTransformFormatProvider.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/ParquetWriteSchemaTransformFormatProvider.java
copy
sdks/java/io/{fileschematransform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
=>
file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlDateTimeAdapter.java}
(51%)
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapter.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowValue.java
create mode 100644
sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlWriteSchemaTransformFormatProvider.java
copy sdks/java/io/{fileschematransform =>
file-schema-transform}/src/main/java/org/apache/beam/sdk/io/fileschematransform/package-info.java
(100%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/AvroFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviderTestData.java
copy sdks/java/io/{fileschematransform =>
file-schema-transform}/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
(64%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/JsonFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/ParquetFileWriteSchemaTransformFormatProviderTest.java
copy
sdks/java/io/{fileschematransform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
=>
file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlDateTimeAdapterTest.java}
(56%)
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlFileWriteSchemaTransformFormatProviderTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapterTest.java
create mode 100644
sdks/java/io/file-schema-transform/src/test/java/org/apache/beam/sdk/io/fileschematransform/XmlRowValueTest.java
delete mode 100644
sdks/java/io/fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProviders.java
rename
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/MonitoringInfoToCounterUpdateTransformer.java
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ByteStringRangeHelper.java
(54%)
copy
runners/google-cloud-dataflow-java/worker/src/main/java/com/google/cloud/dataflow/worker/DataflowRunnerHarness.java
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamMetrics.java
(72%)
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform/package-info.java
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/ChangeStreamMutation.java}
(74%)
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/{bigquery/StorageApiWritePayload.java
=> bigtable/changestreams/TimestampConverter.java} (53%)
copy
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/grpc/BeamFnService.java
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/UniqueIdGenerator.java
(55%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/ActionFactory.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/ChangeStreamAction.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/DetectNewPartitionsAction.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/GenerateInitialPartitionsAction.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action/ReadChangeStreamPartitionAction.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/action}/package-info.java
(79%)
rename
runners/google-cloud-dataflow-java/worker/src/main/java/com/google/cloud/dataflow/worker/DataflowRunnerHarness.java
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/ChangeStreamDao.java
(63%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/DaoFactory.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/MetadataTableAdminDao.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao/MetadataTableDao.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dao}/package-info.java
(80%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/DetectNewPartitionsDoFn.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/InitializeDoFn.java
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/ReadChangeStreamPartitionDoFn.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn}/package-info.java
(79%)
rename
{runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/harness/util
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/encoder}/package-info.java
(72%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/model/PartitionRecord.java
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/model}/package-info.java
(80%)
copy
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams}/package-info.java
(82%)
create mode 100644
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/restriction/ReadChangeStreamPartitionProgressTracker.java
rename
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/grpc/BeamFnService.java
=>
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/restriction/StreamProgress.java
(53%)
rename
sdks/java/io/{fileschematransform/src/main/java/org/apache/beam/sdk/io/fileschematransform
=>
google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/restriction}/package-info.java
(79%)
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/{PubsubSchemaTransformReadConfiguration.java
=> PubsubReadSchemaTransformConfiguration.java} (94%)
rename
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/{PubsubSchemaTransformReadProvider.java
=> PubsubReadSchemaTransformProvider.java} (86%)
rename
sdks/java/io/{fileschematransform/src/test/java/org/apache/beam/sdk/io/fileschematransform/FileWriteSchemaTransformFormatProvidersTest.java
=>
google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/TimestampConverterTest.java}
(50%)
rename
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/{PubsubSchemaTransformReadProviderTest.java
=> PubsubReadSchemaTransformProviderTest.java} (87%)
create mode 100644
sdks/python/apache_beam/examples/inference/run_inference_side_inputs.py
create mode 100644 sdks/python/apache_beam/examples/per_entity_training.py
create mode 100644 sdks/python/apache_beam/ml/inference/utils.py
copy
sdks/python/apache_beam/{examples/ml-orchestration/kfp/components/train/requirements.txt
=> yaml/__init__.py} (93%)
create mode 100644 sdks/python/apache_beam/yaml/main.py
create mode 100644 sdks/python/apache_beam/yaml/pipeline.schema.yaml
copy
sdks/python/apache_beam/{examples/ml-orchestration/kfp/components/train/requirements.txt
=> yaml/standard_providers.yaml} (75%)
create mode 100644 sdks/python/apache_beam/yaml/yaml_provider.py
create mode 100644 sdks/python/apache_beam/yaml/yaml_transform.py
create mode 100644 sdks/python/apache_beam/yaml/yaml_transform_test.py
copy sdks/typescript/{README.md => README-dev.md} (51%)
create mode 100644 sdks/typescript/typedoc.json
create mode 100644 website/www/site/assets/scss/_banner.sass
create mode 100644 website/www/site/content/en/case-studies/163netease.md
create mode 100644 website/www/site/content/en/case-studies/accenture.md
create mode 100644 website/www/site/content/en/case-studies/align.md
create mode 100644 website/www/site/content/en/case-studies/aliz.md
create mode 100644 website/www/site/content/en/case-studies/arquivei.md
create mode 100644 website/www/site/content/en/case-studies/bahwancybertek.md
create mode 100644 website/www/site/content/en/case-studies/bbc.md
create mode 100644 website/www/site/content/en/case-studies/behalf.md
create mode 100644 website/www/site/content/en/case-studies/belllabs.md
create mode 100644 website/www/site/content/en/case-studies/benchsci.md
create mode 100644 website/www/site/content/en/case-studies/betterup.md
create mode 100644 website/www/site/content/en/case-studies/bigdatainstitute.md
create mode 100644 website/www/site/content/en/case-studies/carfinance247.md
create mode 100644 website/www/site/content/en/case-studies/citibank.md
create mode 100644 website/www/site/content/en/case-studies/cognite.md
create mode 100644 website/www/site/content/en/case-studies/cruise.md
create mode 100644 website/www/site/content/en/case-studies/datatonic.md
create mode 100644 website/www/site/content/en/case-studies/dunbradstreet.md
create mode 100644 website/www/site/content/en/case-studies/ericsson.md
create mode 100644 website/www/site/content/en/case-studies/evolve24.md
create mode 100755 website/www/site/content/en/case-studies/graalsystems.md
create mode 100644 website/www/site/content/en/case-studies/hazelcast.md
create mode 100644 website/www/site/content/en/case-studies/hoxtonai.md
create mode 100644 website/www/site/content/en/case-studies/ibiblio.md
create mode 100644
website/www/site/content/en/case-studies/industrialtechnologyresearchinstitute.md
create mode 100644 website/www/site/content/en/case-studies/ineat.md
create mode 100644 website/www/site/content/en/case-studies/lumaapps.md
create mode 100644 website/www/site/content/en/case-studies/mindvalley.md
create mode 100644 website/www/site/content/en/case-studies/natwest.md
create mode 100644 website/www/site/content/en/case-studies/nokia.md
create mode 100644 website/www/site/content/en/case-studies/oracle.md
create mode 100644 website/www/site/content/en/case-studies/orielresearch.md
create mode 100644
website/www/site/content/en/case-studies/oxfordbigdatainstitute.md
create mode 100644 website/www/site/content/en/case-studies/plantix.md
create mode 100644
website/www/site/content/en/case-studies/portaltelemedicina.md
create mode 100644 website/www/site/content/en/case-studies/primerai.md
create mode 100644 website/www/site/content/en/case-studies/prosimo.md
create mode 100644 website/www/site/content/en/case-studies/spicule.md
create mode 100644 website/www/site/content/en/case-studies/tencent.md
create mode 100644 website/www/site/content/en/case-studies/theworkshop.md
create mode 100644 website/www/site/content/en/case-studies/tutanota.md
create mode 100644 website/www/site/content/en/case-studies/unity.md
create mode 100644
website/www/site/content/en/case-studies/universityofweimar.md
create mode 100644 website/www/site/content/en/case-studies/veepee.md
create mode 100644 website/www/site/content/en/case-studies/venturedive.md
create mode 100644 website/www/site/content/en/case-studies/veolia.md
create mode 100644 website/www/site/content/en/case-studies/xiaohongshu.md
create mode 100644 website/www/site/content/en/case-studies/yelp.md
create mode 100644 website/www/site/content/en/documentation/io/io-standards.md
create mode 100644
website/www/site/content/en/documentation/ml/per-entity-training.md
create mode 100644 website/www/site/content/en/get-started/quickstart/python.md
create mode 100644 website/www/site/static/images/banner_desktop.jpg
create mode 100644 website/www/site/static/images/banner_mobile.jpg
create mode 100644
website/www/site/static/images/io-standards/io-connector-guides-screenshot.png
create mode 100644
website/www/site/static/images/io-standards/io-connector-performance-test-dashboard-screenshot.png
create mode 100644
website/www/site/static/images/io-standards/io-connector-relational-features-table.png
create mode 100644
website/www/site/static/images/io-standards/io-supported-via-screenshot.png
create mode 100644
website/www/site/static/images/io-standards/website-built-in-screenshot.png
create mode 100644
website/www/site/static/images/logos/powered-by/163netease.png
create mode 100644
website/www/site/static/images/logos/powered-by/accenture.png
create mode 100644 website/www/site/static/images/logos/powered-by/align.png
create mode 100644 website/www/site/static/images/logos/powered-by/aliz.png
create mode 100644 website/www/site/static/images/logos/powered-by/arquivei.png
create mode 100644
website/www/site/static/images/logos/powered-by/bahwancybertek.png
create mode 100644 website/www/site/static/images/logos/powered-by/bbc.png
create mode 100644 website/www/site/static/images/logos/powered-by/behalf.png
create mode 100644 website/www/site/static/images/logos/powered-by/belllabs.png
create mode 100644 website/www/site/static/images/logos/powered-by/benchsci.png
create mode 100644 website/www/site/static/images/logos/powered-by/betterup.png
create mode 100644
website/www/site/static/images/logos/powered-by/bigdatainstitute.png
create mode 100644
website/www/site/static/images/logos/powered-by/carfinance247.png
create mode 100644 website/www/site/static/images/logos/powered-by/citibank.png
create mode 100644 website/www/site/static/images/logos/powered-by/cognite.png
create mode 100644 website/www/site/static/images/logos/powered-by/cruise.png
create mode 100644
website/www/site/static/images/logos/powered-by/datatonic.png
create mode 100644
website/www/site/static/images/logos/powered-by/dunbradstreet.png
create mode 100644 website/www/site/static/images/logos/powered-by/ericsson.png
create mode 100644 website/www/site/static/images/logos/powered-by/evolve24.png
create mode 100755
website/www/site/static/images/logos/powered-by/graalsystems.png
create mode 100644
website/www/site/static/images/logos/powered-by/hazelcast.png
create mode 100644 website/www/site/static/images/logos/powered-by/hoxtonai.png
create mode 100644 website/www/site/static/images/logos/powered-by/ibiblio.png
create mode 100644
website/www/site/static/images/logos/powered-by/industrialtechnologyresearchinstitute.png
create mode 100644 website/www/site/static/images/logos/powered-by/ineat.png
create mode 100644 website/www/site/static/images/logos/powered-by/lumaapps.png
create mode 100644
website/www/site/static/images/logos/powered-by/mindvalley.png
create mode 100644 website/www/site/static/images/logos/powered-by/natwest.png
create mode 100644 website/www/site/static/images/logos/powered-by/nokia.png
create mode 100644 website/www/site/static/images/logos/powered-by/oracle.png
create mode 100644
website/www/site/static/images/logos/powered-by/orielresearch.png
create mode 100644
website/www/site/static/images/logos/powered-by/oxfordbigdatainstitute.png
create mode 100644 website/www/site/static/images/logos/powered-by/plantix.png
create mode 100644
website/www/site/static/images/logos/powered-by/portaltelemedicina.png
create mode 100644 website/www/site/static/images/logos/powered-by/primerai.png
create mode 100644 website/www/site/static/images/logos/powered-by/prosimo.png
create mode 100644 website/www/site/static/images/logos/powered-by/spicule.png
create mode 100644 website/www/site/static/images/logos/powered-by/tencent.png
create mode 100644
website/www/site/static/images/logos/powered-by/theworkshop.png
create mode 100644 website/www/site/static/images/logos/powered-by/tutanota.png
create mode 100644 website/www/site/static/images/logos/powered-by/unity.png
create mode 100644
website/www/site/static/images/logos/powered-by/universityofweimar.png
create mode 100644 website/www/site/static/images/logos/powered-by/veepee.png
create mode 100644
website/www/site/static/images/logos/powered-by/venturedive.png
create mode 100644 website/www/site/static/images/logos/powered-by/veolia.png
create mode 100644
website/www/site/static/images/logos/powered-by/xiaohongshu.png
create mode 100644 website/www/site/static/images/logos/powered-by/yelp.png
