This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git.
from 92ba229 [BEAM-12730] Handle corner case for TextIO with custom delimiter #15864 add 2c3e979 [BEAM-13105] playground - add shorcut hint tooltip add b97a1c3 Merge branch 'master' into BEAM-13105 add 7db9d2c Merge pull request #15814 from [BEAM-13105] [Playground] - add shortcut hint tooltip add 9109354 Merge pull request #15854 from [BEAM-13046][Playground] protobuf supporting examples API add 9bca51f [BEAM-12550] Parallelizable skew Implementation (#15809) add 48318dd Merge pull request #15782 from [BEAM-13034] [Playground] add semantics to playground app add 07aaec3 [BEAM-13099] Use vendored Calcite 1.28.0 in SQL extensions (#15836) add 6a63eb9 Use BigQuieryIO.loadProjectId in WriteRename class to create JobRefernce config and Fix side input for WriteRename add 735db24 [BEAM-13140] Fixes bug in WriteRename class add 0d172c3 Performance improvement to PubSubLiteIO to not use a streaming committer to send one commit message. add beac28d Merge pull request #15880: Performance improvement to PubSubLiteIO to not use a streaming committer to send one commit message add f41cdf4 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. add 4af1e52 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. add 0690953 Merge pull request #15839 from [BEAM-13041][Playground] Prepare files with java code add 0f94f96 [BEAM-13001] updated CHANGES.md to include msec counter for Go (#15872) add 4f3b5c0 Add window mapping to CHANGES.md (#15871) add 95e9de7 Merge pull request #15784 from [BEAM-8135] - Removing BigQueryResultsPublisher and its references add d5cd15a [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 0d2b05b Merge pull request #15856: [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 82d94b3 [BEAM-13119] Subdirectory prefix tag for Go SDK (#15881) add 156bfc2 Merge pull request #15852 from [BEAM-13102] [Playground] update playground according with the latest design changes add 66ef316 JdbcIO has a single WriteFn underlying all implementations add 2cfb6a7 Addressing comments add bee60cd Merge pull request #15862 from [BEAM-13165] JdbcIO has a single WriteFn underlying all implementations add d81ad7a Add missing parentheses for Python test example add 3c3bfa3 Update test input and imports add c79eaf8 Merge pull request #15630 Fix Python test example in documentation add 013da55 Fix typo: s/spark/twister2 add 0faa3b9 Merge pull request #15885 from ibzib/wc-twister2 No new revisions were added by this update. Summary of changes: .test-infra/jenkins/LoadTestConfig.groovy | 20 +- .../job_LoadTests_CoGBK_Dataflow_V2_Java11.groovy | 20 +- .../jenkins/job_LoadTests_CoGBK_Java.groovy | 21 +- ...ts_CoGBK_Java_spark_structured_streaming.groovy | 17 +- .../jenkins/job_LoadTests_Combine_Java.groovy | 17 +- .../job_LoadTests_Combine_Java_Smoke.groovy | 8 +- ..._Combine_Java_spark_structured_streaming.groovy | 14 +- .../job_LoadTests_GBK_Dataflow_V2_Java11.groovy | 29 +- .test-infra/jenkins/job_LoadTests_GBK_Java.groovy | 29 +- .../jenkins/job_LoadTests_GBK_Java_Smoke.groovy | 17 +- ...ests_GBK_Java_spark_structured_streaming.groovy | 26 +- .../job_LoadTests_ParDo_Dataflow_V2_Java11.groovy | 20 +- .../jenkins/job_LoadTests_ParDo_Java.groovy | 20 +- ...ts_ParDo_Java_spark_structured_streaming.groovy | 17 +- CHANGES.md | 4 + build.gradle.kts | 1 - .../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +- playground/api/v1/api.proto | 54 ++ playground/backend/cmd/server/controller.go | 34 + playground/backend/internal/api/v1/api.pb.go | 711 +++++++++++++++++++-- playground/backend/internal/api/v1/api_grpc.pb.go | 148 ++++- playground/backend/internal/executors/executor.go | 17 +- .../backend/internal/executors/executor_builder.go | 19 + .../backend/internal/executors/executor_test.go | 36 +- .../internal/preparators/java_preparators.go | 142 ++++ .../internal/preparators/java_preparators_test.go | 102 +++ .../validator.go => preparators/preparator.go} | 9 +- playground/frontend/assets/beam_lg.png | Bin 0 -> 1000 bytes playground/frontend/assets/github.svg | 19 + playground/frontend/assets/output_bottom.svg | 19 + playground/frontend/assets/output_left.svg | 19 + playground/frontend/assets/output_right.svg | 19 + playground/frontend/assets/reset.svg | 19 + playground/frontend/assets/send_feedback.svg | 19 + playground/frontend/assets/shortcuts.svg | 19 + playground/frontend/assets/theme.svg | 19 + .../frontend/assets/thumb_down.svg | 10 +- .../frontend/assets/thumb_up.svg | 16 +- playground/frontend/lib/api/v1/api.pb.dart | 397 +++++++++++- playground/frontend/lib/api/v1/api.pbenum.dart | 33 +- playground/frontend/lib/api/v1/api.pbgrpc.dart | 84 +++ playground/frontend/lib/api/v1/api.pbjson.dart | 100 ++- .../dropdown_button/dropdown_button.dart} | 132 ++-- .../lib/components/logo/logo_component.dart | 40 +- playground/frontend/lib/config/theme.dart | 48 +- playground/frontend/lib/constants/assets.dart | 3 + playground/frontend/lib/constants/colors.dart | 6 + playground/frontend/lib/constants/font_weight.dart | 2 + .../fonts.dart} | 18 +- playground/frontend/lib/constants/sizes.dart | 5 +- .../actions/components/new_example_action.dart | 19 +- .../modules/actions/components/reset_action.dart | 17 +- .../modules/editor/components/editor_textarea.dart | 36 +- .../modules/editor/components/editor_themes.dart | 60 ++ .../lib/modules/editor/components/run_button.dart | 29 +- .../code_client/grpc_code_client.dart | 5 + .../lib/modules/examples/example_selector.dart | 146 +---- .../lib/modules/output/components/output_area.dart | 8 +- .../components/output_header/output_header.dart | 2 +- .../output_header/output_placements.dart | 21 +- .../components/output_result.dart} | 24 +- .../modules/output/models/output_placement.dart | 21 +- .../lib/modules/sdk/components/sdk_selector.dart | 61 +- .../components/sdk_selector_row.dart} | 40 +- .../modules/shortcuts/components/shortcut_row.dart | 20 +- .../components/shortcut_tooltip.dart} | 27 +- .../shortcuts/components/shortcuts_modal.dart | 2 +- .../shortcuts/constants/global_shortcuts.dart | 111 ++-- .../shortcuts/utils/shortcuts_display_name.dart | 22 +- .../components/editor_textarea_wrapper.dart | 22 +- .../pages/playground/components/more_actions.dart | 3 +- .../components/playground_feedback.dart} | 42 +- .../components/playground_page_footer.dart | 20 +- .../components/playground_page_providers.dart | 4 +- .../lib/pages/playground/playground_page.dart | 16 +- .../pages/playground/states/examples_state.dart | 6 - .../pages/playground/states/playground_state.dart | 3 +- playground/frontend/pubspec.lock | 77 +++ playground/frontend/pubspec.yaml | 1 + .../playground/states/playground_state_test.dart | 10 +- playground/frontend/web/index.html | 2 +- .../main/groovy/quickstart-java-twister2.groovy | 2 +- release/src/main/scripts/choose_rc_commit.sh | 8 + sdks/java/extensions/sql/build.gradle | 12 +- .../sql/example/BeamSqlDataCatalogExample.java | 2 +- .../datacatalog/DataCatalogTableProvider.java | 2 +- .../sql/meta/provider/datacatalog/SchemaUtils.java | 4 +- .../provider/bigquery/BigQueryIOPushDownIT.java | 7 +- .../extensions/sql/src/main/codegen/config.fmpp | 8 +- .../beam/sdk/extensions/sql/SqlTransform.java | 4 +- .../extensions/sql/TableNameExtractionUtils.java | 17 +- .../sdk/extensions/sql/impl/BeamCalciteSchema.java | 16 +- .../sql/impl/BeamCalciteSchemaFactory.java | 20 +- .../sdk/extensions/sql/impl/BeamCalciteTable.java | 30 +- .../beam/sdk/extensions/sql/impl/BeamSqlEnv.java | 14 +- .../sql/impl/BeamSqlPipelineOptionsRegistrar.java | 2 +- .../extensions/sql/impl/BeamTableStatistics.java | 14 +- .../sql/impl/CalciteConnectionWrapper.java | 16 +- .../extensions/sql/impl/CalciteFactoryWrapper.java | 29 +- .../extensions/sql/impl/CalciteQueryPlanner.java | 74 +-- .../sdk/extensions/sql/impl/JavaUdfLoader.java | 2 +- .../sdk/extensions/sql/impl/JdbcConnection.java | 8 +- .../beam/sdk/extensions/sql/impl/JdbcDriver.java | 28 +- .../beam/sdk/extensions/sql/impl/JdbcFactory.java | 37 +- .../beam/sdk/extensions/sql/impl/QueryPlanner.java | 8 +- .../extensions/sql/impl/ScalarFunctionImpl.java | 46 +- .../extensions/sql/impl/TableResolutionUtils.java | 6 +- .../beam/sdk/extensions/sql/impl/UdafImpl.java | 12 +- .../beam/sdk/extensions/sql/impl/UdfImpl.java | 10 +- .../sql/impl/UdfImplReflectiveFunctionBase.java | 16 +- ...SqlUserDefinedSQLNativeTableValuedFunction.java | 14 +- .../beam/sdk/extensions/sql/impl/cep/CEPCall.java | 10 +- .../sdk/extensions/sql/impl/cep/CEPFieldRef.java | 2 +- .../sdk/extensions/sql/impl/cep/CEPLiteral.java | 2 +- .../sdk/extensions/sql/impl/cep/CEPOperation.java | 8 +- .../sdk/extensions/sql/impl/cep/CEPOperator.java | 4 +- .../sdk/extensions/sql/impl/cep/CEPPattern.java | 2 +- .../beam/sdk/extensions/sql/impl/cep/CEPUtils.java | 16 +- .../beam/sdk/extensions/sql/impl/cep/OrderKey.java | 2 +- .../extensions/sql/impl/parser/BeamSqlParser.java | 8 +- .../sql/impl/parser/SqlCheckConstraint.java | 18 +- .../sql/impl/parser/SqlColumnDeclaration.java | 23 +- .../sql/impl/parser/SqlCreateExternalTable.java | 28 +- .../sql/impl/parser/SqlCreateFunction.java | 32 +- .../extensions/sql/impl/parser/SqlDdlNodes.java | 27 +- .../extensions/sql/impl/parser/SqlDropObject.java | 25 +- .../extensions/sql/impl/parser/SqlDropTable.java | 10 +- .../sql/impl/parser/SqlSetOptionBeam.java | 18 +- .../extensions/sql/impl/planner/BeamCostModel.java | 8 +- .../sql/impl/planner/BeamJavaTypeFactory.java | 12 +- .../sql/impl/planner/BeamRelDataTypeSystem.java | 6 +- .../sql/impl/planner/BeamRelMetadataQuery.java | 6 +- .../extensions/sql/impl/planner/BeamRuleSets.java | 14 +- .../sql/impl/planner/NodeStatsMetadata.java | 12 +- .../sql/impl/planner/RelMdNodeStats.java | 14 +- .../sql/impl/rel/AbstractBeamCalcRel.java | 18 +- .../sql/impl/rel/BeamAggregationRel.java | 18 +- .../sdk/extensions/sql/impl/rel/BeamCalcRel.java | 68 +- .../extensions/sql/impl/rel/BeamCoGBKJoinRel.java | 16 +- .../sql/impl/rel/BeamEnumerableConverter.java | 38 +- .../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java | 25 +- .../extensions/sql/impl/rel/BeamIOSourceRel.java | 20 +- .../extensions/sql/impl/rel/BeamIntersectRel.java | 12 +- .../sdk/extensions/sql/impl/rel/BeamJoinRel.java | 39 +- .../sql/impl/rel/BeamLogicalConvention.java | 12 +- .../sdk/extensions/sql/impl/rel/BeamMatchRel.java | 27 +- .../sdk/extensions/sql/impl/rel/BeamMinusRel.java | 12 +- .../sql/impl/rel/BeamPushDownIOSourceRel.java | 14 +- .../sdk/extensions/sql/impl/rel/BeamRelNode.java | 8 +- .../sql/impl/rel/BeamSetOperatorRelBase.java | 2 +- .../sql/impl/rel/BeamSideInputJoinRel.java | 16 +- .../sql/impl/rel/BeamSideInputLookupJoinRel.java | 14 +- .../sdk/extensions/sql/impl/rel/BeamSortRel.java | 33 +- .../extensions/sql/impl/rel/BeamSqlRelUtils.java | 23 +- .../sql/impl/rel/BeamTableFunctionScanRel.java | 24 +- .../extensions/sql/impl/rel/BeamUncollectRel.java | 12 +- .../sdk/extensions/sql/impl/rel/BeamUnionRel.java | 12 +- .../sdk/extensions/sql/impl/rel/BeamUnnestRel.java | 22 +- .../sdk/extensions/sql/impl/rel/BeamValuesRel.java | 18 +- .../sdk/extensions/sql/impl/rel/BeamWindowRel.java | 22 +- .../extensions/sql/impl/rel/CalcRelSplitter.java | 54 +- .../sdk/extensions/sql/impl/rel/package-info.java | 2 +- .../impl/rule/BeamAggregateProjectMergeRule.java | 20 +- .../sql/impl/rule/BeamAggregationRule.java | 24 +- .../sql/impl/rule/BeamBasicAggregationRule.java | 26 +- .../sql/impl/rule/BeamCalcMergeRule.java | 10 +- .../sdk/extensions/sql/impl/rule/BeamCalcRule.java | 18 +- .../sql/impl/rule/BeamCalcSplittingRule.java | 14 +- .../sql/impl/rule/BeamCoGBKJoinRule.java | 12 +- .../sql/impl/rule/BeamEnumerableConverterRule.java | 8 +- .../sql/impl/rule/BeamIOPushDownRule.java | 38 +- .../extensions/sql/impl/rule/BeamIOSinkRule.java | 6 +- .../sql/impl/rule/BeamIntersectRule.java | 10 +- .../sql/impl/rule/BeamJoinAssociateRule.java | 12 +- .../sql/impl/rule/BeamJoinPushThroughJoinRule.java | 16 +- .../extensions/sql/impl/rule/BeamMatchRule.java | 10 +- .../extensions/sql/impl/rule/BeamMinusRule.java | 10 +- .../sql/impl/rule/BeamSideInputJoinRule.java | 12 +- .../sql/impl/rule/BeamSideInputLookupJoinRule.java | 12 +- .../sdk/extensions/sql/impl/rule/BeamSortRule.java | 10 +- .../sql/impl/rule/BeamTableFunctionScanRule.java | 12 +- .../sql/impl/rule/BeamUncollectRule.java | 8 +- .../extensions/sql/impl/rule/BeamUnionRule.java | 12 +- .../extensions/sql/impl/rule/BeamUnnestRule.java | 26 +- .../extensions/sql/impl/rule/BeamValuesRule.java | 10 +- .../extensions/sql/impl/rule/BeamWindowRule.java | 10 +- .../sql/impl/rule/JoinRelOptRuleCall.java | 16 +- .../sql/impl/rule/LogicalCalcMergeRule.java | 12 +- .../sdk/extensions/sql/impl/rule/package-info.java | 2 +- .../extensions/sql/impl/schema/BeamTableUtils.java | 7 +- .../impl/transform/BeamBuiltinAggregations.java | 2 +- .../transform/BeamBuiltinAnalyticFunctions.java | 2 +- .../sql/impl/transform/BeamJoinTransforms.java | 8 +- .../transform/agg/AggregationCombineFnAdapter.java | 4 +- .../sql/impl/transform/agg/CovarianceFn.java | 4 +- .../sql/impl/transform/agg/VarianceFn.java | 2 +- .../sql/impl/udf/BuiltinHashFunctions.java | 2 +- .../sql/impl/udf/BuiltinStringFunctions.java | 2 +- .../sql/impl/utils/BigDecimalConverter.java | 2 +- .../extensions/sql/impl/utils/CalciteUtils.java | 37 +- .../sql/impl/utils/SerializableRexFieldAccess.java | 4 +- .../sql/impl/utils/SerializableRexInputRef.java | 2 +- .../sql/impl/utils/SerializableRexNode.java | 6 +- .../sdk/extensions/sql/meta/BaseBeamTable.java | 2 +- .../beam/sdk/extensions/sql/meta/BeamSqlTable.java | 2 +- .../extensions/sql/meta/BeamSqlTableFilter.java | 4 +- .../extensions/sql/meta/DefaultTableFilter.java | 2 +- .../sql/meta/provider/ReadOnlyTableProvider.java | 2 +- .../provider/bigquery/BeamBigQuerySqlDialect.java | 22 +- .../provider/bigquery/BeamSqlUnparseContext.java | 44 +- .../sql/meta/provider/bigquery/BigQueryFilter.java | 34 +- .../sql/meta/provider/bigquery/BigQueryTable.java | 14 +- .../provider/bigquery/BigQueryTableProvider.java | 2 +- .../sql/meta/provider/bigtable/BigtableFilter.java | 14 +- .../sql/meta/provider/bigtable/BigtableTable.java | 2 +- .../sql/meta/provider/kafka/BeamKafkaTable.java | 2 +- .../sql/meta/provider/mongodb/MongoDbTable.java | 20 +- .../sql/meta/provider/test/TestTableFilter.java | 14 +- .../sql/meta/provider/test/TestTableProvider.java | 12 +- .../sql/meta/provider/test/TestTableUtils.java | 2 +- .../sql/meta/provider/test/TestUnboundedTable.java | 2 +- .../sql/meta/provider/text/TextTableProvider.java | 6 +- .../sql/meta/store/InMemoryMetaStore.java | 2 +- ..._28_0.org.codehaus.commons.compiler.properties} | 2 +- .../sdk/extensions/sql/BeamComplexTypeTest.java | 4 +- .../sql/BeamSqlDslSqlStdOperatorsTest.java | 28 +- .../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java | 10 +- .../sdk/extensions/sql/BeamSqlExplainTest.java | 6 +- .../beam/sdk/extensions/sql/BeamSqlMapTest.java | 2 +- .../extensions/sql/BeamSqlMultipleSchemasTest.java | 2 +- .../sdk/extensions/sql/PubsubToBigqueryIT.java | 4 +- .../apache/beam/sdk/extensions/sql/TestUtils.java | 2 +- .../extensions/sql/TypedCombineFnDelegateTest.java | 10 +- .../sdk/extensions/sql/impl/JdbcDriverTest.java | 8 +- .../sql/impl/LazyAggregateCombineFnTest.java | 12 +- .../sql/impl/parser/BeamDDLNestedTypesTest.java | 2 +- .../extensions/sql/impl/parser/BeamDDLTest.java | 12 +- .../extensions/sql/impl/planner/NodeStatsTest.java | 10 +- .../sql/impl/rel/BeamAggregationRelTest.java | 2 +- .../extensions/sql/impl/rel/BeamCalcRelTest.java | 2 +- .../rel/BeamCoGBKJoinRelBoundedVsBoundedTest.java | 2 +- .../BeamCoGBKJoinRelUnboundedVsUnboundedTest.java | 2 +- .../sql/impl/rel/BeamEnumerableConverterTest.java | 26 +- .../sql/impl/rel/BeamIOSourceRelTest.java | 4 +- .../sql/impl/rel/BeamIntersectRelTest.java | 2 +- .../extensions/sql/impl/rel/BeamMinusRelTest.java | 2 +- .../sql/impl/rel/BeamSideInputJoinRelTest.java | 2 +- .../extensions/sql/impl/rel/BeamSortRelTest.java | 2 +- .../sql/impl/rel/BeamUncollectRelTest.java | 2 +- .../extensions/sql/impl/rel/BeamUnionRelTest.java | 2 +- .../extensions/sql/impl/rel/BeamValuesRelTest.java | 2 +- .../sql/impl/rule/IOPushDownRuleTest.java | 14 +- .../sql/impl/rule/JoinReorderingTest.java | 76 +-- .../sql/impl/schema/BeamSqlRowCoderTest.java | 8 +- .../sql/impl/utils/CalciteUtilsTest.java | 10 +- ...BeamSqlBuiltinFunctionsIntegrationTestBase.java | 6 +- .../BeamSqlDateFunctionsIntegrationTest.java | 4 +- .../sql/meta/CustomTableResolverTest.java | 4 +- .../provider/bigquery/BigQueryReadWriteIT.java | 2 +- .../meta/provider/bigquery/BigQueryRowCountIT.java | 2 +- .../bigquery/BigQueryTestTableProvider.java | 2 +- .../provider/datastore/DataStoreReadWriteIT.java | 2 +- .../provider/mongodb/MongoDbTableProviderTest.java | 2 +- .../provider/pubsub/PubsubTableProviderIT.java | 8 +- .../meta/provider/test/TestTableProviderTest.java | 2 +- ...tTableProviderWithFilterAndProjectPushDown.java | 8 +- .../test/TestTableProviderWithFilterPushDown.java | 10 +- .../test/TestTableProviderWithProjectPushDown.java | 8 +- .../meta/provider/text/TextTableProviderTest.java | 2 +- .../beam/sdk/extensions/sql/utils/RowAsserts.java | 2 +- sdks/java/extensions/sql/zetasql/build.gradle | 6 +- .../extensions/sql/zetasql/BeamCalcRelType.java | 37 +- .../sql/zetasql/BeamJavaUdfCalcRule.java | 2 +- .../sql/zetasql/BeamZetaSqlCalcMergeRule.java | 10 +- .../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 26 +- .../sql/zetasql/BeamZetaSqlCalcRule.java | 2 +- .../extensions/sql/zetasql/BeamZetaSqlCatalog.java | 24 +- .../extensions/sql/zetasql/BeamZetaSqlRelType.java | 22 +- .../sdk/extensions/sql/zetasql/DateTimeUtils.java | 2 +- .../extensions/sql/zetasql/TableResolution.java | 8 +- .../extensions/sql/zetasql/ZetaSQLPlannerImpl.java | 30 +- .../sql/zetasql/ZetaSQLQueryPlanner.java | 56 +- .../zetasql/ZetaSqlCalciteTranslationUtils.java | 22 +- .../translation/AggregateScanConverter.java | 19 +- .../translation/ArrayScanColumnRefToUncollect.java | 22 +- .../ArrayScanLiteralToUncollectConverter.java | 6 +- .../translation/ArrayScanToJoinConverter.java | 18 +- .../sql/zetasql/translation/ConversionContext.java | 6 +- .../zetasql/translation/ExpressionConverter.java | 36 +- .../zetasql/translation/FilterScanConverter.java | 6 +- .../sql/zetasql/translation/JoinScanConverter.java | 10 +- .../LimitOffsetScanToLimitConverter.java | 14 +- .../LimitOffsetScanToOrderByLimitConverter.java | 22 +- .../OrderByScanUnsupportedConverter.java | 2 +- .../zetasql/translation/ProjectScanConverter.java | 8 +- .../translation/QueryStatementConverter.java | 2 +- .../sql/zetasql/translation/RelConverter.java | 16 +- .../translation/SetOperationScanConverter.java | 8 +- .../translation/SingleRowScanConverter.java | 2 +- .../SqlCaseWithValueOperatorRewriter.java | 8 +- .../translation/SqlCoalesceOperatorRewriter.java | 10 +- .../translation/SqlIfNullOperatorRewriter.java | 8 +- .../zetasql/translation/SqlInOperatorRewriter.java | 6 +- .../translation/SqlNullIfOperatorRewriter.java | 8 +- .../translation/SqlOperatorMappingTable.java | 4 +- .../zetasql/translation/SqlOperatorRewriter.java | 4 +- .../sql/zetasql/translation/SqlOperators.java | 54 +- .../translation/SqlWindowTableFunction.java | 28 +- .../sql/zetasql/translation/TVFScanConverter.java | 6 +- .../zetasql/translation/TableScanConverter.java | 36 +- .../zetasql/translation/WithRefScanConverter.java | 2 +- .../sql/zetasql/translation/WithScanConverter.java | 2 +- .../translation/ZetaSqlScalarFunctionImpl.java | 10 +- .../translation/impl/BeamBuiltinMethods.java | 2 +- .../zetasql/translation/impl/CastFunctionImpl.java | 24 +- .../zetasql/translation/impl/StringFunctions.java | 4 +- .../translation/impl/TimestampFunctions.java | 2 +- .../zetasql/unnest/BeamZetaSqlUncollectRel.java | 10 +- .../zetasql/unnest/BeamZetaSqlUncollectRule.java | 6 +- .../sql/zetasql/unnest/BeamZetaSqlUnnestRel.java | 20 +- .../sql/zetasql/unnest/BeamZetaSqlUnnestRule.java | 22 +- .../sql/zetasql/unnest/ZetaSqlUnnest.java | 52 +- .../sql/zetasql/BeamJavaUdfCalcRuleTest.java | 6 +- .../sql/zetasql/BeamZetaSqlCatalogTest.java | 2 +- .../sql/zetasql/TableResolutionTest.java | 4 +- .../sql/zetasql/ZetaSQLPushDownTest.java | 16 +- .../extensions/sql/zetasql/ZetaSqlJavaUdfTest.java | 2 +- .../sql/zetasql/ZetaSqlJavaUdfTypeTest.java | 4 +- .../extensions/sql/zetasql/ZetaSqlTestBase.java | 10 +- .../beam/fn/harness/ProcessBundleBenchmark.java | 8 +- .../beam/sdk/bigqueryioperftests/BigQueryIOIT.java | 1 - .../java/org/apache/beam/sdk/io/avro/AvroIOIT.java | 1 - .../apache/beam/sdk/io/parquet/ParquetIOIT.java | 1 - .../java/org/apache/beam/sdk/io/text/TextIOIT.java | 1 - .../apache/beam/sdk/io/tfrecord/TFRecordIOIT.java | 1 - .../java/org/apache/beam/sdk/io/xml/XmlIOIT.java | 1 - .../beam/sdk/io/gcp/bigquery/BatchLoads.java | 8 +- .../beam/sdk/io/gcp/bigquery/WriteRename.java | 17 +- .../beam/sdk/io/gcp/bigquery/WriteTables.java | 5 +- ...ialOffsetReader.java => BlockingCommitter.java} | 8 +- .../internal/PerSubscriptionPartitionSdf.java | 14 +- .../pubsublite/internal/SubscribeTransform.java | 3 +- .../pubsublite/internal/SubscriberAssembler.java | 19 +- .../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 3 +- .../internal/PerSubscriptionPartitionSdfTest.java | 16 +- .../sdk/io/hadoop/format/HadoopFormatIOIT.java | 2 - .../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 508 ++++++++------- .../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java | 2 - .../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 2 - .../apache/beam/sdk/io/mongodb/MongoDBIOIT.java | 2 - .../org/apache/beam/sdk/io/parquet/ParquetIO.java | 1 + .../org/apache/beam/sdk/loadtests/LoadTest.java | 24 - .../apache/beam/sdk/loadtests/LoadTestOptions.java | 16 - .../java/org/apache/beam/sdk/nexmark/Main.java | 27 - .../beam/sdk/nexmark/PerfsToBigQueryTest.java | 88 --- .../beam/sdk/testutils/metrics/IOITMetrics.java | 19 - .../publishing/BigQueryResultsPublisher.java | 65 -- .../sdk/testutils/fakes/FakeBigQueryClient.java | 68 -- .../fakes/FakeBigQueryResultsPublisher.java | 52 -- .../publishing/BigQueryResultsPublisherTest.java | 114 ---- sdks/java/testing/tpcds/build.gradle | 4 +- .../org/apache/beam/sdk/tpcds/QueryReader.java | 6 +- .../apache/beam/sdk/tpcds/SqlTransformRunner.java | 4 +- sdks/python/apache_beam/dataframe/frames.py | 73 ++- sdks/python/apache_beam/dataframe/frames_test.py | 5 +- settings.gradle.kts | 1 - vendor/calcite-1_26_0/build.gradle | 99 --- .../site/content/en/contribute/release-guide.md | 6 + .../documentation/pipelines/test-your-pipeline.md | 18 +- 369 files changed, 4664 insertions(+), 3300 deletions(-) create mode 100644 playground/backend/internal/preparators/java_preparators.go create mode 100644 playground/backend/internal/preparators/java_preparators_test.go copy playground/backend/internal/{validators/validator.go => preparators/preparator.go} (82%) create mode 100644 playground/frontend/assets/beam_lg.png copy .test-infra/metrics/sync/github/README.md => playground/frontend/assets/thumb_down.svg (57%) copy .test-infra/kubernetes/kafka-cluster/README.md => playground/frontend/assets/thumb_up.svg (58%) copy playground/frontend/lib/{modules/examples/example_selector.dart => components/dropdown_button/dropdown_button.dart} (51%) copy playground/frontend/lib/{pages/playground/states/example_dropdown_state.dart => constants/fonts.dart} (69%) create mode 100644 playground/frontend/lib/modules/editor/components/editor_themes.dart copy playground/frontend/lib/modules/{actions/components/header_icon_button.dart => output/components/output_result.dart} (69%) copy playground/frontend/lib/modules/{output/components/output_header/output_header.dart => sdk/components/sdk_selector_row.dart} (55%) copy playground/frontend/lib/modules/{actions/components/header_icon_button.dart => shortcuts/components/shortcut_tooltip.dart} (67%) copy runners/core-java/src/main/java/org/apache/beam/runners/core/GroupAlsoByWindowsAggregators.java => playground/frontend/lib/modules/shortcuts/utils/shortcuts_display_name.dart (66%) copy playground/frontend/lib/{components/toggle_theme_button/toggle_theme_button.dart => pages/playground/components/playground_feedback.dart} (56%) rename sdks/java/extensions/sql/src/main/resources/{org.apache.beam.vendor.calcite.v1_26_0.org.codehaus.commons.compiler.properties => org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.commons.compiler.properties} (93%) copy sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsublite/internal/{InitialOffsetReader.java => BlockingCommitter.java} (84%) delete mode 100644 sdks/java/testing/nexmark/src/test/java/org/apache/beam/sdk/nexmark/PerfsToBigQueryTest.java delete mode 100644 sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/publishing/BigQueryResultsPublisher.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/fakes/FakeBigQueryClient.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/fakes/FakeBigQueryResultsPublisher.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/publishing/BigQueryResultsPublisherTest.java delete mode 100644 vendor/calcite-1_26_0/build.gradle