This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git.
from 05cbe24 Merge pull request #16293 from ibzib/flink-upgrade add c0921f7 Use a patched shadow 6.1.0 plugin using Log4j 2.16.0 (#16269) add 3fb40fe [BEAM-12830] Replace GoGradle plugin with Shell Scripts. (#16291) add a915161 [BEAM-13430] Start the process of upgrading the Gradle 7. (#16292) add ea47b1f [BEAM-13502]: fix loading example on embedded version add c970da3 Merge pull request #16305 from [BEAM-13502] [Playground] [Bugfix] showing a selected example on the embedded version of the playground add 4a356da [BEAM-13430] Introduce new testRuntimeMigration configuration that mirrors what testRuntime used to do. (#16307) add c03b4b5e [BEAM-13498] Sickbay portable Flink testProcessElementSkew. add c4dd8f4 Merge pull request #16295 from ibzib/BEAM-13498 add 97a1556 [BEAM-13499] Add warning about hcatalog to release notes and javadoc (#16297) add d845a00 [BEAM-13397] Bump numpy to 1.21 for M1 chip add 37428b3 re-generate requirements.txt add 59bc389 [BEAM-13397] Bump numpy to 1.21 for M1 chip #16227 add efa8983 Update flink cluster to use a supported dataproc version (1.2 -> 1.5) add b3ba7b5 Merge pull request #16310 from apache/aaltay-patch-1 add 2e72638 [BEAM-13430] Remove propdeps and replace with compileOnly (#16308) add c5fbe47 Merge pull request #16304 from [BEAM-13491] [Playground] Examples' catalog doesn't close after selecting example add 402bb56 Merge pull request #16283 from [BEAM-13448] [Playground] track run code time to the analytics add ec4c7ee Merge pull request #16244 from [BEAM-13463] [Playground] add retries to check status for run code add 38deb59 Merge pull request #16241 from [BEAM-13440] [Playground] Implement initialization of Cloud Logger add 45df0ac6 [BEAM-13399] Add integration test for Go SDK expansion service JAR pull/start-up (#16313) add a4336c9 [BEAM-13421] Fix bug with xs called with non-tuple key (#16258) No new revisions were added by this update. Summary of changes: .github/workflows/local_env_tests.yml | 9 +- .test-infra/dataproc/flink_cluster.sh | 4 +- .test-infra/metrics/build.gradle | 4 +- .test-infra/validate-runner/build.gradle | 28 ++-- CHANGES.md | 4 +- buildSrc/build.gradle.kts | 44 ++--- .../jarlibs/shadow-6.1.0-log4jfix-SNAPSHOT.jar | Bin 0 -> 5628292 bytes .../org/apache/beam/gradle/BeamModulePlugin.groovy | 174 ++++++++++--------- .../apache/beam/gradle/GrpcVendoring_1_36_0.groovy | 4 +- .../org/apache/beam/gradle/VendorJavaPlugin.groovy | 2 +- examples/java/build.gradle | 92 +++++------ examples/java/twitter/build.gradle | 32 ++-- examples/kotlin/build.gradle | 46 +++--- learning/katas/java/build.gradle | 30 ++-- learning/katas/kotlin/build.gradle | 32 ++-- model/fn-execution/build.gradle | 4 +- model/interactive/build.gradle | 4 +- model/job-management/build.gradle | 4 +- ownership/JAVA_DEPENDENCY_OWNERS.yaml | 5 - playground/backend/cmd/server/server.go | 3 + .../backend/internal/environment/application.go | 20 ++- .../internal/environment/application_test.go | 82 +++++++++ .../internal/environment/environment_service.go | 7 +- .../environment/environment_service_test.go | 21 ++- .../internal/logger/cloud_logging_handler.go | 1 + playground/backend/internal/logger/logger.go | 51 +++--- playground/backend/internal/logger/logger_test.go | 4 +- playground/backend/internal/logger/std_handler.go | 78 +++++++++ .../lib/modules/analytics/analytics_events.dart | 1 + .../lib/modules/analytics/analytics_service.dart | 16 +- .../components/pipeline_options_text_field.dart | 3 +- .../code_client/grpc_code_client.dart | 21 ++- .../code_repository/code_repository.dart | 12 +- .../code_repository/run_code_result.dart | 2 +- .../example_list/category_expansion_panel.dart | 10 +- .../components/example_list/example_list.dart | 6 + .../example_list/expansion_panel_item.dart | 16 +- .../lib/modules/examples/example_selector.dart | 18 +- .../components/embedded_editor.dart | 1 + .../components/editor_textarea_wrapper.dart | 12 +- .../components/playground_page_providers.dart | 8 +- .../pages/playground/states/playground_state.dart | 5 +- playground/frontend/lib/utils/run_with_retry.dart | 52 ++++++ .../frontend/test/utils/run_with_retry_test.dart | 95 +++++++++++ release/build.gradle.kts | 4 +- runners/core-construction-java/build.gradle | 40 ++--- runners/core-java/build.gradle | 26 +-- runners/direct-java/build.gradle | 16 +- runners/extensions-java/metrics/build.gradle | 14 +- runners/flink/flink_runner.gradle | 83 +++++----- runners/flink/job-server/flink_job_server.gradle | 20 ++- runners/google-cloud-dataflow-java/build.gradle | 78 ++++----- .../examples-streaming/build.gradle | 2 +- .../examples/build.gradle | 2 +- .../google-cloud-dataflow-java/worker/build.gradle | 70 ++++---- .../worker/legacy-worker/build.gradle | 60 ++++--- .../dataflow/worker/status/DebugCapture.java | 4 +- runners/java-fn-execution/build.gradle | 37 +++-- runners/java-job-service/build.gradle | 36 ++-- runners/jet/build.gradle | 42 ++--- runners/local-java/build.gradle | 6 +- runners/portability/java/build.gradle | 42 ++--- runners/samza/build.gradle | 58 +++---- runners/samza/job-server/build.gradle | 12 +- runners/spark/job-server/spark_job_server.gradle | 16 +- runners/spark/spark_runner.gradle | 87 +++++----- runners/twister2/build.gradle | 48 +++--- sdks/go/build.gradle | 40 ----- sdks/go/container/build.gradle | 39 +---- sdks/go/examples/build.gradle | 82 +-------- .../beam/core/runtime/xlangx/expansionx/process.go | 4 + sdks/go/run_with_go_version.sh | 75 +++++++++ sdks/go/test/build.gradle | 42 ----- sdks/go/test/integration/xlang/expansion_test.go | 68 ++++++++ sdks/go/test/load/build.gradle | 54 +----- sdks/go/test/run_validatesrunner_tests.sh | 2 +- sdks/java/container/build.gradle | 36 +--- sdks/java/core/build.gradle | 18 +- sdks/java/expansion-service/app/build.gradle | 2 +- sdks/java/expansion-service/build.gradle | 25 +-- sdks/java/extensions/arrow/build.gradle | 14 +- sdks/java/extensions/euphoria/build.gradle | 22 +-- .../google-cloud-platform-core/build.gradle | 46 +++--- sdks/java/extensions/jackson/build.gradle | 10 +- sdks/java/extensions/join-library/build.gradle | 6 +- sdks/java/extensions/kryo/build.gradle | 10 +- sdks/java/extensions/ml/build.gradle | 61 +++---- sdks/java/extensions/protobuf/build.gradle | 12 +- sdks/java/extensions/sbe/build.gradle | 6 +- .../schemaio-expansion-service/build.gradle | 18 +- sdks/java/extensions/sketching/build.gradle | 14 +- sdks/java/extensions/sorter/build.gradle | 16 +- sdks/java/extensions/sql/build.gradle | 89 ++++++---- sdks/java/extensions/sql/datacatalog/build.gradle | 36 ++-- .../extensions/sql/expansion-service/build.gradle | 10 +- sdks/java/extensions/sql/hcatalog/build.gradle | 11 +- .../sql/meta/provider/hcatalog/package-info.java | 11 +- .../provider/hcatalog/BeamSqlHiveSchemaTest.java | 5 + sdks/java/extensions/sql/jdbc/build.gradle | 23 ++- sdks/java/extensions/sql/perf-tests/build.gradle | 6 +- sdks/java/extensions/sql/shell/build.gradle | 4 +- .../extensions/sql/udf-test-provider/build.gradle | 4 +- sdks/java/extensions/sql/zetasql/build.gradle | 50 +++--- sdks/java/extensions/zetasketch/build.gradle | 19 ++- sdks/java/fn-execution/build.gradle | 26 +-- sdks/java/harness/build.gradle | 28 ++-- sdks/java/io/amazon-web-services/build.gradle | 57 +++---- sdks/java/io/amazon-web-services2/build.gradle | 75 ++++----- sdks/java/io/amqp/build.gradle | 18 +- sdks/java/io/azure/build.gradle | 30 ++-- sdks/java/io/bigquery-io-perf-tests/build.gradle | 20 ++- sdks/java/io/cassandra/build.gradle | 26 +-- sdks/java/io/clickhouse/build.gradle | 22 +-- sdks/java/io/common/build.gradle | 8 +- sdks/java/io/contextualtextio/build.gradle | 20 +-- sdks/java/io/debezium/build.gradle | 36 ++-- .../io/debezium/expansion-service/build.gradle | 6 +- .../elasticsearch-tests-5/build.gradle | 26 +-- .../elasticsearch-tests-6/build.gradle | 26 +-- .../elasticsearch-tests-7/build.gradle | 24 +-- .../elasticsearch-tests-common/build.gradle | 18 +- sdks/java/io/elasticsearch/build.gradle | 26 +-- sdks/java/io/expansion-service/build.gradle | 10 +- sdks/java/io/file-based-io-tests/build.gradle | 19 ++- sdks/java/io/google-cloud-platform/build.gradle | 184 ++++++++++----------- .../expansion-service/build.gradle | 6 +- sdks/java/io/hadoop-common/build.gradle | 13 +- sdks/java/io/hadoop-file-system/build.gradle | 26 +-- sdks/java/io/hadoop-format/build.gradle | 73 ++++---- sdks/java/io/hbase/build.gradle | 36 ++-- sdks/java/io/hcatalog/build.gradle | 41 ++--- .../apache/beam/sdk/io/hcatalog/HCatalogIO.java | 7 + .../apache/beam/sdk/io/hcatalog/package-info.java | 11 +- sdks/java/io/influxdb/build.gradle | 18 +- sdks/java/io/jdbc/build.gradle | 31 ++-- sdks/java/io/jms/build.gradle | 22 +-- sdks/java/io/kafka/build.gradle | 56 ++++--- sdks/java/io/kinesis/build.gradle | 46 +++--- .../java/io/kinesis/expansion-service/build.gradle | 6 +- sdks/java/io/kudu/build.gradle | 14 +- sdks/java/io/mongodb/build.gradle | 21 +-- sdks/java/io/mqtt/build.gradle | 22 +-- sdks/java/io/parquet/build.gradle | 25 +-- sdks/java/io/rabbitmq/build.gradle | 22 +-- sdks/java/io/redis/build.gradle | 12 +- sdks/java/io/snowflake/build.gradle | 30 ++-- .../io/snowflake/expansion-service/build.gradle | 6 +- sdks/java/io/solr/build.gradle | 26 +-- sdks/java/io/splunk/build.gradle | 30 ++-- sdks/java/io/synthetic/build.gradle | 20 +-- sdks/java/io/thrift/build.gradle | 10 +- sdks/java/io/tika/build.gradle | 12 +- sdks/java/io/xml/build.gradle | 18 +- sdks/java/testing/expansion-service/build.gradle | 9 +- sdks/java/testing/jpms-tests/build.gradle | 10 +- sdks/java/testing/kafka-service/build.gradle | 6 +- sdks/java/testing/load-tests/build.gradle | 26 +-- sdks/java/testing/nexmark/build.gradle | 46 +++--- sdks/java/testing/test-utils/build.gradle | 24 +-- sdks/java/testing/tpcds/build.gradle | 34 ++-- sdks/java/testing/watermarks/build.gradle | 6 +- sdks/python/apache_beam/dataframe/frames.py | 14 +- sdks/python/apache_beam/dataframe/frames_test.py | 15 ++ sdks/python/container/build.gradle | 34 +--- .../container/py36/base_image_requirements.txt | 26 +-- .../container/py37/base_image_requirements.txt | 34 ++-- .../container/py38/base_image_requirements.txt | 34 ++-- sdks/python/setup.py | 2 +- 168 files changed, 2498 insertions(+), 2047 deletions(-) create mode 100644 buildSrc/jarlibs/shadow-6.1.0-log4jfix-SNAPSHOT.jar create mode 100644 playground/backend/internal/logger/std_handler.go create mode 100644 playground/frontend/lib/utils/run_with_retry.dart create mode 100644 playground/frontend/test/utils/run_with_retry_test.dart create mode 100755 sdks/go/run_with_go_version.sh create mode 100644 sdks/go/test/integration/xlang/expansion_test.go