This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/gradle/com.diffplug.spotless-6.22.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 1a66308a98f Bump com.diffplug.spotless from 5.6.1 to 6.22.0
add fbde88dca3e Update flogger-system-backend to 0.7.4 (#28585)
add 47ba976b82d Add GitHub Workflow Replacements for Jenkins
job_PerformanceTests_TextIOIT (#28586)
add e7ec5fe8daf Add Load Tests GBK Flink Batch Go workflow (#28494)
add c5b75ed496f Create documentation page for Python SDK unrecoverable
errors (#28702)
add 5d4c524cf3e Add LoadTests Go ParDo Flink Batch workflow (#28723)
add e8342b00ef2 Add Load Tests Combine Flink Batch Go workflow (#28661)
add eae46458d92 Github Workflow Replacement for Jenkins Jobs,
beam_LoadTests_Java_GBK_SparkStructuredStreaming_Batch and Cron Fix for other
workflows (#28737)
add 65de726ce36 Beam post commit java influx db io it (#28530)
add 75d7c70a9a0 Fix Jpms test (#28726)
add 8a31ffe09ba replace comment.body with comment.id (#28739)
add 4204ba34a5d postcommit java singlestoreIO_IO (#28741)
add 0a6b29da878 Add missing access modifiers to SimpleRateLimitPolicy
(#28389)
add b10546cee25 Add Load Tests SideInput Flink Batch Go workflow (#28662)
add 5d6dceeba01 Add Load Tests CoGBK Flink Batch Go workflow (#28660)
add 5e38decf9e7 Add callout of PIP failures as unrecoverable, link to new
documentation (#28743)
add 80311628c27 Remove warning from catch in table exists validation
(#28288)
add 01197c42319 Add GBK, CoGBK, SideInputs and Combine python batch load
tests to perf alert tool (#28746)
add 2b20e26bb3b Restrict build_runner_image to pull_request instead of
pull_request_target (#28745)
add f2194f6b4c8 Temporarily Ignore 2 FhirIO Integration Tests (#28717)
add bf1e8297582 Report total active threads (#28513)
add 5fd870e1023 Fix dockerhub task
add 282d027f162 Merge pull request #28766: Fix dockerhub task
add 3bbfa7ec29a runner image to 2b20e26bb3b99d8e4f41a3d1d9d2e7080043de5c
(#28769)
add 4a9608dfc82 Github Workflow Replacement for Jenkins Jobs,
beam_PerformanceTests_xlang_KafkaIO_Python (#28730)
add 11e2bae4cbe Fix input detection. (#28775)
add a7bf59436a6 Removing Depreciation warning when using pipeline.options
(#28774)
add bd1c95cb0de Better flag documentation. (#28753)
add 71c8459633e [Blog Post] Apache Beam for a content discovery platform
(#28734)
add 1c5982567b3 Bump cloud.google.com/go/bigtable from 1.19.0 to 1.20.0 in
/sdks (#28789)
add 25024064ed2 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28790)
add b556aee9177 Bump urllib3 from 1.26.16 to 1.26.17 in
/sdks/python/container/py38 (#28787)
add c7d7896533b Github Workflow Replacement for Jenkins Jobs,
beam_LoadTests_Java_GBK_Dataflow_Batch_* (#28738)
add 2e0521162f6 [Java BQ] Storage API streaming load test (#28264)
add 3655b1ea3fb Update dyi-content-discovery-platform-genai-beam.md
add f3df03d8fa9 Merge pull request #28788 from apache/aaltay-patch-1
add 12b07140203 Add notebook for per key models (#28327)
add c01b41f9758 Bump urllib3 from 1.26.16 to 1.26.17 in
/sdks/python/container/py310 (#28785)
add be805372238 BigQuery testing suite that runs against BQ's day 0 region
(#28397)
add f6b43ce1736 Revert "assign highmem runner to beam_PostCommit_Python
and to beam_PreCommit_Java_GCP_IO_Direct (#28719)" (#28804)
add 95af465f0df yaml label (#28813)
add 9efdf6be61e Fix SpannerChangeStreamErrorTest.java and stop disabling
tests (#28751)
add a9684c5563d TPC-DS: Fix CSV format
add 0a560fc09d8 Merge pull request #28819: TPC-DS: Fix CSV format
add 3cae5ada4f6 Fix dtype usage (#28797)
add 9ed9c059fec Fix method argument handling for pandas 2.1 (#28816)
add 659b22089e6 Fix arm tests on Python PostCommit (#28740)
add f30f6c5e220 [Blog] Quest updated dates (#28824)
add b221d804998 Add job name and worker id to traceId of
AppendRowsRequest. (#28729)
add 3e22be4d310 Revert "Bump com.avast.gradle:gradle-docker-compose-plugin
(#28465)" (#28828)
add c2666e11497 Add num_workers and save_main_session flag to
auto_model_refresh notebook (#28777)
add 6f4e2852311 Fix spotless on master (#28831)
add 161cd6b1c8d Gradle 8 support (#28756)
add cc0a7c76641 Bump com.diffplug.spotless from 5.6.1 to 6.22.0
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1a66308a98f)
\
N -- N -- N
refs/heads/dependabot/gradle/com.diffplug.spotless-6.22.0 (cc0a7c76641)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/setup-k8s-access/action.yml | 2 +-
.github/autolabeler.yml | 1 +
.../arc/environments/beam.env | 6 +-
.github/workflows/README.md | 6 +-
.github/workflows/beam_CancelStaleDataflowJobs.yml | 2 +-
.github/workflows/beam_CleanUpGCPResources.yml | 2 +-
.../workflows/beam_CleanUpPrebuiltSDKImages.yml | 2 +-
.../beam_LoadTests_Go_CoGBK_Flink_batch.yml | 129 +++
.../beam_LoadTests_Go_Combine_Flink_Batch.yml | 134 +++
.../beam_LoadTests_Go_GBK_Flink_Batch.yml | 164 +++
.../beam_LoadTests_Go_ParDo_Flink_Batch.yml | 139 +++
.../beam_LoadTests_Go_SideInput_Flink_Batch.yml | 119 ++
.../beam_LoadTests_Java_GBK_Dataflow_Batch.yml | 141 +++
.../beam_LoadTests_Java_GBK_Dataflow_Streaming.yml | 141 +++
...LoadTests_Java_GBK_Dataflow_V2_Batch_Java11.yml | 166 +++
...LoadTests_Java_GBK_Dataflow_V2_Batch_Java17.yml | 168 +++
...Tests_Java_GBK_Dataflow_V2_Streaming_Java11.yml | 166 +++
...Tests_Java_GBK_Dataflow_V2_Streaming_Java17.yml | 168 +++
...sts_Java_GBK_SparkStructuredStreaming_Batch.yml | 141 +++
.github/workflows/beam_PerformanceTests_Cdap.yml | 2 +-
.../beam_PerformanceTests_HadoopFormat.yml | 2 +-
.../beam_PerformanceTests_MongoDBIO_IT.yml | 2 +-
.../beam_PerformanceTests_SparkReceiver_IO.yml | 2 +-
...Cdap.yml => beam_PerformanceTests_TextIOIT.yml} | 47 +-
...yml => beam_PerformanceTests_TextIOIT_HDFS.yml} | 46 +-
...l => beam_PerformanceTests_TextIOIT_Python.yml} | 59 +-
.../beam_PerformanceTests_xlang_KafkaIO_Python.yml | 121 +++
.../beam_PostCommit_BeamMetrics_Publish.yml | 2 +-
.../workflows/beam_PostCommit_Go_Dataflow_ARM.yml | 2 +-
.github/workflows/beam_PostCommit_Go_VR_Samza.yml | 2 +-
.github/workflows/beam_PostCommit_Go_VR_Spark.yml | 2 +-
.github/workflows/beam_PostCommit_Java.yml | 2 +-
.../beam_PostCommit_Java_Avro_Versions.yml | 2 +-
... beam_PostCommit_Java_BigQueryEarlyRollout.yml} | 25 +-
.../workflows/beam_PostCommit_Java_DataflowV1.yml | 2 +-
.../workflows/beam_PostCommit_Java_DataflowV2.yml | 2 +-
.../beam_PostCommit_Java_Examples_Dataflow.yml | 2 +-
.../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 2 +-
...beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 +-
.../beam_PostCommit_Java_Examples_Dataflow_V2.yml | 2 +-
...m_PostCommit_Java_Examples_Dataflow_V2_Java.yml | 2 +-
.../beam_PostCommit_Java_Examples_Direct.yml | 2 +-
.../beam_PostCommit_Java_Examples_Flink.yml | 2 +-
.../beam_PostCommit_Java_Examples_Spark.yml | 2 +-
.../beam_PostCommit_Java_Hadoop_Versions.yml | 2 +-
.../beam_PostCommit_Java_IO_Performance_Tests.yml | 2 +-
....yml => beam_PostCommit_Java_InfluxDbIO_IT.yml} | 29 +-
.../beam_PostCommit_Java_Jpms_Dataflow_Java11.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Dataflow_Java17.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Direct_Java11.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Direct_Java17.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Flink_Java11.yml | 2 +-
.../beam_PostCommit_Java_Jpms_Spark_Java11.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Dataflow.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Dataflow_V2.yml | 2 +-
...am_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Direct.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Flink.yml | 2 +-
.../beam_PostCommit_Java_Nexmark_Spark.yml | 2 +-
.../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 +-
.../workflows/beam_PostCommit_Java_PVR_Samza.yml | 2 +-
.../beam_PostCommit_Java_PVR_Spark3_Streaming.yml | 2 +-
.../beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 +-
.github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 +-
...l => beam_PostCommit_Java_SingleStoreIO_IT.yml} | 60 +-
.../beam_PostCommit_Java_Tpcds_Dataflow.yml | 2 +-
.../workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 +-
.../workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 +-
...am_PostCommit_Java_ValidatesRunner_Dataflow.yml | 2 +-
..._Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 +-
...mit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 +-
...PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 +-
..._Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 +-
...beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 +-
...it_Java_ValidatesRunner_Direct_JavaVersions.yml | 2 +-
.../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 2 +-
...ostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 +-
.../beam_PostCommit_Java_ValidatesRunner_Samza.yml | 2 +-
.../beam_PostCommit_Java_ValidatesRunner_Spark.yml | 2 +-
...va_ValidatesRunner_SparkStructuredStreaming.yml | 2 +-
...ostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 +-
...am_PostCommit_Java_ValidatesRunner_Twister2.yml | 2 +-
.../beam_PostCommit_Java_ValidatesRunner_ULR.yml | 2 +-
.github/workflows/beam_PostCommit_Javadoc.yml | 2 +-
.../beam_PostCommit_PortableJar_Flink.yml | 2 +-
.../beam_PostCommit_PortableJar_Spark.yml | 2 +-
.github/workflows/beam_PostCommit_Python.yml | 4 +-
.github/workflows/beam_PostCommit_Python_Arm.yml | 24 +-
.../beam_PostCommit_Python_Examples_Dataflow.yml | 2 +-
.../beam_PostCommit_Python_Examples_Direct.yml | 2 +-
.../beam_PostCommit_Python_Examples_Flink.yml | 2 +-
.../beam_PostCommit_Python_Examples_Spark.yml | 2 +-
.../beam_PostCommit_Python_MongoDBIO_IT.yml | 2 +-
.../beam_PostCommit_Python_Nexmark_Direct.yml | 2 +-
...stCommit_Python_ValidatesContainer_Dataflow.yml | 2 +-
..._Python_ValidatesContainer_Dataflow_With_RC.yml | 2 +-
..._PostCommit_Python_ValidatesRunner_Dataflow.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Flink.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Samza.yml | 2 +-
...eam_PostCommit_Python_ValidatesRunner_Spark.yml | 2 +-
.../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml | 2 +-
.../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 2 +-
.../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 2 +-
.github/workflows/beam_PostCommit_SQL.yml | 2 +-
.../workflows/beam_PostCommit_Sickbay_Python.yml | 2 +-
.../beam_PostCommit_TransformService_Direct.yml | 2 +-
.../workflows/beam_PostCommit_Website_Publish.yml | 2 +-
.github/workflows/beam_PostCommit_Website_Test.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Direct.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Flink.yml | 2 +-
.../beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 2 +-
...eam_PostCommit_XVR_JavaUsingPython_Dataflow.yml | 2 +-
..._PostCommit_XVR_PythonUsingJavaSQL_Dataflow.yml | 2 +-
...eam_PostCommit_XVR_PythonUsingJava_Dataflow.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Samza.yml | 2 +-
.github/workflows/beam_PostCommit_XVR_Spark3.yml | 2 +-
.../workflows/beam_PreCommit_CommunityMetrics.yml | 6 +-
.github/workflows/beam_PreCommit_Go.yml | 2 +-
.github/workflows/beam_PreCommit_GoPortable.yml | 2 +-
.github/workflows/beam_PreCommit_GoPrism.yml | 2 +-
.github/workflows/beam_PreCommit_ItFramework.yml | 2 +-
.github/workflows/beam_PreCommit_Java.yml | 2 +-
...eCommit_Java_Amazon-Web-Services2_IO_Direct.yml | 2 +-
...reCommit_Java_Amazon-Web-Services_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Amqp_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Azure_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Cassandra_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Cdap_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Clickhouse_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Csv_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Debezium_IO_Direct.yml | 2 +-
...beam_PreCommit_Java_ElasticSearch_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Examples_Dataflow.yml | 2 +-
...eam_PreCommit_Java_Examples_Dataflow_Java11.yml | 2 +-
...eam_PreCommit_Java_Examples_Dataflow_Java17.yml | 2 +-
...Commit_Java_File-schema-transform_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Flink_Versions.yml | 2 +-
.../beam_PreCommit_Java_GCP_IO_Direct.yml | 4 +-
.../beam_PreCommit_Java_HBase_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_HCatalog_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Hadoop_IO_Direct.yml | 2 +-
.../workflows/beam_PreCommit_Java_IOs_Direct.yml | 2 +-
.../beam_PreCommit_Java_InfluxDb_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_JDBC_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Jms_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Kafka_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Kinesis_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Kudu_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_MongoDb_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Mqtt_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Neo4j_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_PVR_Flink_Batch.yml | 2 +-
.../beam_PreCommit_Java_PVR_Flink_Docker.yml | 2 +-
.../beam_PreCommit_Java_Parquet_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Pulsar_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_RabbitMq_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Redis_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_SingleStore_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Snowflake_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Solr_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Spark3_Versions.yml | 2 +-
.../beam_PreCommit_Java_Splunk_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Thrift_IO_Direct.yml | 2 +-
.../beam_PreCommit_Java_Tika_IO_Direct.yml | 2 +-
.../workflows/beam_PreCommit_Kotlin_Examples.yml | 2 +-
.../workflows/beam_PreCommit_Portable_Python.yml | 2 +-
.github/workflows/beam_PreCommit_Python.yml | 2 +-
.github/workflows/beam_PreCommit_PythonDocker.yml | 2 +-
.github/workflows/beam_PreCommit_PythonDocs.yml | 2 +-
.../workflows/beam_PreCommit_PythonFormatter.yml | 2 +-
.github/workflows/beam_PreCommit_PythonLint.yml | 2 +-
.../workflows/beam_PreCommit_Python_Coverage.yml | 2 +-
.../workflows/beam_PreCommit_Python_Dataframes.yml | 2 +-
.../workflows/beam_PreCommit_Python_Examples.yml | 2 +-
.../beam_PreCommit_Python_Integration.yml | 2 +-
.../workflows/beam_PreCommit_Python_PVR_Flink.yml | 2 +-
.../workflows/beam_PreCommit_Python_Runners.yml | 2 +-
.../workflows/beam_PreCommit_Python_Transforms.yml | 2 +-
.github/workflows/beam_PreCommit_RAT.yml | 2 +-
.github/workflows/beam_PreCommit_SQL.yml | 2 +-
.github/workflows/beam_PreCommit_SQL_Java11.yml | 2 +-
.github/workflows/beam_PreCommit_SQL_Java17.yml | 2 +-
.github/workflows/beam_PreCommit_Spotless.yml | 2 +-
.github/workflows/beam_PreCommit_Typescript.yml | 2 +-
.github/workflows/beam_PreCommit_Website.yml | 2 +-
.../workflows/beam_PreCommit_Website_Stage_GCS.yml | 2 +-
.github/workflows/beam_PreCommit_Whitespace.yml | 2 +-
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 2 +-
.github/workflows/build_release_candidate.yml | 2 +-
.github/workflows/build_runner_image.yml | 2 +-
.github/workflows/build_wheels.yml | 2 +-
.github/workflows/dask_runner_tests.yml | 2 +-
.github/workflows/go_tests.yml | 2 +-
.github/workflows/java_tests.yml | 2 +-
...ructuredStreaming_Batch_2GB_of_100B_records.txt | 25 +
...ucturedStreaming_Batch_2GB_of_100kB_records.txt | 25 +
...tructuredStreaming_Batch_2GB_of_10B_records.txt | 25 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 25 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 25 +
...reaming_Batch_reiterate_4_times_10kB_values.txt | 25 +
...treaming_Batch_reiterate_4_times_2MB_values.txt | 25 +
.../go_CoGBK_Flink_Batch_MultipleKey.txt | 26 +
.../go_CoGBK_Flink_Batch_Reiteration_10KB.txt | 26 +
.../go_CoGBK_Flink_Batch_Reiteration_2MB.txt | 26 +
.../go_Combine_Flink_Batch_10b.txt | 26 +
.../go_Combine_Flink_Batch_Fanout_4.txt | 26 +
.../go_Combine_Flink_Batch_Fanout_8.txt | 26 +
.../go_GBK_Flink_Batch_100b.txt | 26 +
.../go_GBK_Flink_Batch_100kb.txt | 26 +
.../go_GBK_Flink_Batch_10b.txt | 26 +
.../go_GBK_Flink_Batch_Fanout_4.txt | 26 +
.../go_GBK_Flink_Batch_Fanout_8.txt | 26 +
.../go_GBK_Flink_Batch_Reiteration_10KB.txt | 26 +
.../go_ParDo_Flink_Batch_100_counters.txt | 27 +
.../go_ParDo_Flink_Batch_10_counters.txt | 27 +
.../go_ParDo_Flink_Batch_10_times.txt | 27 +
.../go_ParDo_Flink_Batch_200_times.txt | 27 +
.../go_SideInput_Flink_Batch_First_Iterable.txt | 25 +
.../go_SideInput_Flink_Batch_Iterable.txt | 24 +
...java_GBK_Dataflow_Batch_2GB_of_100B_records.txt | 28 +
...ava_GBK_Dataflow_Batch_2GB_of_100kB_records.txt | 28 +
.../java_GBK_Dataflow_Batch_2GB_of_10B_records.txt | 28 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 28 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 28 +
...ataflow_Batch_reiterate_4_times_10kB_values.txt | 28 +
...Dataflow_Batch_reiterate_4_times_2MB_values.txt | 28 +
..._GBK_Dataflow_Streaming_2GB_of_100B_records.txt | 29 +
...GBK_Dataflow_Streaming_2GB_of_100kB_records.txt | 29 +
...a_GBK_Dataflow_Streaming_2GB_of_10B_records.txt | 29 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 29 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 29 +
...low_Streaming_reiterate_4_times_10kB_values.txt | 29 +
...flow_Streaming_reiterate_4_times_2MB_values.txt | 29 +
...ataflow_V2_Batch_Java11_2GB_of_100B_records.txt | 29 +
...taflow_V2_Batch_Java11_2GB_of_100kB_records.txt | 29 +
...Dataflow_V2_Batch_Java11_2GB_of_10B_records.txt | 29 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 29 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 29 +
..._Batch_Java11_reiterate_4_times_10kB_values.txt | 29 +
...2_Batch_Java11_reiterate_4_times_2MB_values.txt | 29 +
...ataflow_V2_Batch_Java17_2GB_of_100B_records.txt | 29 +
...taflow_V2_Batch_Java17_2GB_of_100kB_records.txt | 29 +
...Dataflow_V2_Batch_Java17_2GB_of_10B_records.txt | 29 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 29 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 29 +
..._Batch_Java17_reiterate_4_times_10kB_values.txt | 29 +
...2_Batch_Java17_reiterate_4_times_2MB_values.txt | 29 +
...low_V2_Streaming_Java11_2GB_of_100B_records.txt | 30 +
...ow_V2_Streaming_Java11_2GB_of_100kB_records.txt | 30 +
...flow_V2_Streaming_Java11_2GB_of_10B_records.txt | 30 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 30 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 30 +
...eaming_Java11_reiterate_4_times_10kB_values.txt | 30 +
...reaming_Java11_reiterate_4_times_2MB_values.txt | 30 +
...low_V2_Streaming_Java17_2GB_of_100B_records.txt | 30 +
...ow_V2_Streaming_Java17_2GB_of_100kB_records.txt | 30 +
...flow_V2_Streaming_Java17_2GB_of_10B_records.txt | 30 +
...nout_4_times_with_2GB_10-byte_records_total.txt | 30 +
...nout_8_times_with_2GB_10-byte_records_total.txt | 30 +
...eaming_Java17_reiterate_4_times_10kB_values.txt | 30 +
...reaming_Java17_reiterate_4_times_2MB_values.txt | 30 +
.github/workflows/local_env_tests.yml | 2 +-
.../performance-tests-job-configs/textIOIT.txt | 26 +
.../textIOIT_HDFS.txt | 26 +
.../textIOIT_Python.txt | 28 +
.../xlang_KafkaIO_Python.txt | 30 +
.github/workflows/playground_frontend_test.yml | 2 +-
.github/workflows/python_dependency_tests.yml | 2 +-
.github/workflows/python_tests.yml | 2 +-
.github/workflows/tour_of_beam_backend.yml | 2 +-
.../workflows/tour_of_beam_backend_integration.yml | 2 +-
.github/workflows/tour_of_beam_frontend_test.yml | 2 +-
.github/workflows/typescript_tests.yml | 2 +-
...ob_PerformanceTests_xlang_KafkaIO_Python.groovy | 124 ---
.../04-outside-services/outside-0.yml | 1 -
.../04-outside-services/outside-1.yml | 3 +-
.../04-outside-services/outside-2.yml | 3 +-
.../kubernetes/singlestore/sdb-operator.yaml | 2 +-
.test-infra/metrics/build.gradle | 2 +-
buildSrc/build.gradle.kts | 17 +-
.../org/apache/beam/gradle/BeamDockerPlugin.groovy | 325 ++++++
.../apache/beam/gradle/BeamDockerRunPlugin.groovy | 143 +++
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 31 +-
.../org/apache/beam/gradle/VendorJavaPlugin.groovy | 2 +-
examples/notebooks/beam-ml/README.md | 1 +
.../beam-ml/automatic_model_refresh.ipynb | 1133 +++++++++-----------
examples/notebooks/beam-ml/per_key_models.ipynb | 608 +++++++++++
gradle/wrapper/gradle-wrapper.jar | Bin 61624 -> 63721 bytes
gradle/wrapper/gradle-wrapper.properties | 3 +-
gradlew | 19 +-
it/build.gradle | 4 +
it/google-cloud-platform/build.gradle | 5 +-
.../beam/it/gcp/bigquery/BigQueryStreamingLT.java | 643 +++++++++++
playground/kafka-emulator/build.gradle | 4 +-
.../dataflow/worker/DataflowSystemMetrics.java | 2 +
.../dataflow/worker/StreamingDataflowWorker.java | 11 +
.../dataflow/worker/util/BoundedQueueExecutor.java | 4 +
.../worker/StreamingDataflowWorkerTest.java | 75 +-
sdks/go.mod | 10 +-
sdks/go.sum | 16 +-
sdks/java/container/common.gradle | 2 +
.../sdk/extensions/gcp/util/GceMetadataUtil.java | 50 +-
.../apache/beam/sdk/io/googleads/GoogleAdsV14.java | 4 +-
sdks/java/io/google-cloud-platform/build.gradle | 48 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 6 +-
.../sdk/io/gcp/bigquery/BigQueryIOMetadata.java | 34 +-
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 21 +-
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 29 +-
.../sdk/io/gcp/bigquery/TestBigQueryOptions.java | 7 +
.../beam/sdk/io/gcp/bigtable/BigtableIO.java | 24 +-
.../gcp/common/GcpIoPipelineOptionsRegistrar.java | 2 +
.../beam/sdk/io/gcp/testing/BigqueryClient.java | 37 +-
.../io/gcp/bigquery/BigQueryIOStorageQueryIT.java | 10 +-
.../io/gcp/bigquery/BigQueryIOStorageReadIT.java | 9 +-
.../bigquery/BigQueryIOStorageReadTableRowIT.java | 43 +-
.../io/gcp/bigquery/BigQueryIOStorageWriteIT.java | 31 +-
.../bigquery/BigQuerySchemaUpdateOptionsIT.java | 6 +-
.../BigQueryTimePartitioningClusteringIT.java | 42 +-
.../sdk/io/gcp/bigquery/BigQueryToTableIT.java | 9 +-
.../sdk/io/gcp/bigquery/BigQueryUtilsTest.java | 37 +-
.../sdk/io/gcp/bigquery/FileLoadsStreamingIT.java | 9 +-
.../bigquery/StorageApiDirectWriteProtosIT.java | 9 +-
.../gcp/bigquery/StorageApiSinkFailedRowsIT.java | 13 +-
.../io/gcp/bigquery/StorageApiSinkRowUpdateIT.java | 9 +-
.../gcp/bigquery/StorageApiSinkSchemaUpdateIT.java | 17 +-
.../gcp/bigquery/TableRowToStorageApiProtoIT.java | 11 +-
.../beam/sdk/io/gcp/healthcare/FhirIOSearchIT.java | 3 +
.../SpannerChangeStreamErrorTest.java | 181 +++-
sdks/java/testing/jpms-tests/build.gradle | 40 +-
.../apache/beam/sdk/tpcds/SqlTransformRunner.java | 3 +-
sdks/python/apache_beam/dataframe/frames.py | 14 +-
sdks/python/apache_beam/pipeline.py | 4 -
.../runners/dataflow/dataflow_runner.py | 20 +-
.../runners/interactive/interactive_beam.py | 6 -
.../runners/interactive/interactive_environment.py | 6 -
.../runners/interactive/recording_manager.py | 5 -
.../testing/analyzers/tests_config.yaml | 189 ++++
sdks/python/apache_beam/yaml/yaml_provider.py | 4 +-
sdks/python/container/piputil.go | 10 +-
.../container/py310/base_image_requirements.txt | 2 +-
.../container/py38/base_image_requirements.txt | 2 +-
sdks/python/scripts/run_integration_test.sh | 9 +
sdks/python/test-suites/dataflow/common.gradle | 3 +
website/www/site/content/en/blog/beamquest.md | 2 +-
.../dyi-content-discovery-platform-genai-beam.md | 338 ++++++
.../sdks/python-unrecoverable-errors.md | 61 ++
.../site/content/en/documentation/sdks/python.md | 4 +
website/www/site/data/authors.yml | 6 +
.../layouts/partials/section-menu/en/sdks.html | 1 +
.../images/blog/dyi-cdp-genai-beam/cdp-arch.png | Bin 0 -> 271543 bytes
.../blog/dyi-cdp-genai-beam/cdp-highlevel.png | Bin 0 -> 31242 bytes
.../images/blog/dyi-cdp-genai-beam/pipeline-1.png | Bin 0 -> 146525 bytes
.../pipeline-2-extractcontent.png | Bin 0 -> 130427 bytes
.../pipeline-3-errorhandling.png | Bin 0 -> 112800 bytes
.../pipeline-4-processembeddings1.png | Bin 0 -> 49246 bytes
.../pipeline-4-processembeddings2.png | Bin 0 -> 58035 bytes
.../dyi-cdp-genai-beam/pipeline-5-storecontent.png | Bin 0 -> 74751 bytes
.../dyi-cdp-genai-beam/pipeline-6-refresh1.png | Bin 0 -> 74889 bytes
.../dyi-cdp-genai-beam/pipeline-6-refresh2.png | Bin 0 -> 72757 bytes
.../dyi-cdp-genai-beam/pipeline-6-refresh3.png | Bin 0 -> 53972 bytes
360 files changed, 7841 insertions(+), 1327 deletions(-)
create mode 100644 .github/workflows/beam_LoadTests_Go_CoGBK_Flink_batch.yml
create mode 100644 .github/workflows/beam_LoadTests_Go_Combine_Flink_Batch.yml
create mode 100644 .github/workflows/beam_LoadTests_Go_GBK_Flink_Batch.yml
create mode 100644 .github/workflows/beam_LoadTests_Go_ParDo_Flink_Batch.yml
create mode 100644
.github/workflows/beam_LoadTests_Go_SideInput_Flink_Batch.yml
create mode 100644 .github/workflows/beam_LoadTests_Java_GBK_Dataflow_Batch.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_Dataflow_Streaming.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_Dataflow_V2_Batch_Java11.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_Dataflow_V2_Batch_Java17.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java11.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java17.yml
create mode 100644
.github/workflows/beam_LoadTests_Java_GBK_SparkStructuredStreaming_Batch.yml
copy .github/workflows/{beam_PerformanceTests_Cdap.yml =>
beam_PerformanceTests_TextIOIT.yml} (63%)
copy .github/workflows/{beam_PerformanceTests_HadoopFormat.yml =>
beam_PerformanceTests_TextIOIT_HDFS.yml} (68%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_TextIOIT_Python.yml} (59%)
create mode 100644
.github/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml
copy .github/workflows/{beam_PostCommit_Java.yml =>
beam_PostCommit_Java_BigQueryEarlyRollout.yml} (77%)
copy .github/workflows/{beam_CleanUpGCPResources.yml =>
beam_PostCommit_Java_InfluxDbIO_IT.yml} (66%)
copy .github/workflows/{beam_CleanUpGCPResources.yml =>
beam_PostCommit_Java_SingleStoreIO_IT.yml} (54%)
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/Java_GBK_SparkStructuredStreaming_Batch_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_CoGBK_Flink_Batch_MultipleKey.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_CoGBK_Flink_Batch_Reiteration_10KB.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_CoGBK_Flink_Batch_Reiteration_2MB.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_Combine_Flink_Batch_10b.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_Combine_Flink_Batch_Fanout_4.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_Combine_Flink_Batch_Fanout_8.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_100b.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_100kb.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_10b.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_Fanout_4.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_Fanout_8.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_GBK_Flink_Batch_Reiteration_10KB.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_ParDo_Flink_Batch_100_counters.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_ParDo_Flink_Batch_10_counters.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_ParDo_Flink_Batch_10_times.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_ParDo_Flink_Batch_200_times.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_SideInput_Flink_Batch_First_Iterable.txt
create mode 100644
.github/workflows/load-tests-job-configs/go_SideInput_Flink_Batch_Iterable.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Batch_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_Streaming_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java11_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Batch_Java17_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java11_reiterate_4_times_2MB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_2GB_of_100B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_2GB_of_100kB_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_2GB_of_10B_records.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_fanout_4_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_fanout_8_times_with_2GB_10-byte_records_total.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_reiterate_4_times_10kB_values.txt
create mode 100644
.github/workflows/load-tests-job-configs/java_GBK_Dataflow_V2_Streaming_Java17_reiterate_4_times_2MB_values.txt
create mode 100644 .github/workflows/performance-tests-job-configs/textIOIT.txt
create mode 100644
.github/workflows/performance-tests-job-configs/textIOIT_HDFS.txt
create mode 100644
.github/workflows/performance-tests-job-configs/textIOIT_Python.txt
create mode 100644
.github/workflows/performance-tests-job-configs/xlang_KafkaIO_Python.txt
delete mode 100644
.test-infra/jenkins/job_PerformanceTests_xlang_KafkaIO_Python.groovy
create mode 100644
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamDockerPlugin.groovy
create mode 100644
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamDockerRunPlugin.groovy
create mode 100644 examples/notebooks/beam-ml/per_key_models.ipynb
create mode 100644
it/google-cloud-platform/src/test/java/org/apache/beam/it/gcp/bigquery/BigQueryStreamingLT.java
create mode 100644
website/www/site/content/en/blog/dyi-content-discovery-platform-genai-beam.md
create mode 100644
website/www/site/content/en/documentation/sdks/python-unrecoverable-errors.md
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/cdp-arch.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/cdp-highlevel.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-1.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-2-extractcontent.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-3-errorhandling.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-4-processembeddings1.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-4-processembeddings2.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-5-storecontent.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-6-refresh1.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-6-refresh2.png
create mode 100644
website/www/site/static/images/blog/dyi-cdp-genai-beam/pipeline-6-refresh3.png
