[beam] branch master updated (667ec40 -> 985e2f0)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 667ec40 Merge pull request #14504: Refactor PR template to separate test types and label test variants add 3ea492d [BEAM-12172] Bump gradle to 6.8.3 add 985e2f0 Merge pull request #14543: [BEAM-12172] Bump gradle to 6.8.3 No new revisions were added by this update. Summary of changes: gradle/wrapper/gradle-wrapper.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch master updated (c54397e -> 667ec40)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from c54397e Merge pull request #14555: Fixes to deploying RC artifacts to pypi add 14bbf5f Refactor PR template to separate test types and label test variants add 667ec40 Merge pull request #14504: Refactor PR template to separate test types and label test variants No new revisions were added by this update. Summary of changes: .github/PULL_REQUEST_TEMPLATE.md | 186 +++ 1 file changed, 130 insertions(+), 56 deletions(-)
[beam] branch master updated (06f6050 -> c54397e)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 06f6050 Merge pull request #14561 from dmkozh/fake_coders add 0ad0bf3 Run release scripts from same directory, not temp clone add 4a7e281 Fix invocation of download_github_actions_artifacts.py from deploy_release_candidate_pypi.sh add bbd3cf3 Limit GitHub Actions artifact downloads to RC tag to avoid paging add 1e08700 More verbose output downloading GHA artifacts add c54397e Merge pull request #14555: Fixes to deploying RC artifacts to pypi No new revisions were added by this update. Summary of changes: release/src/main/scripts/deploy_release_candidate_pypi.sh | 5 +++-- release/src/main/scripts/download_github_actions_artifacts.py | 10 +++--- 2 files changed, 10 insertions(+), 5 deletions(-)
[beam] tag nightly-master updated (a86dc06 -> bb948c1)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/beam.git. *** WARNING: tag nightly-master was modified! *** from a86dc06 (commit) to bb948c1 (commit) from a86dc06 Bump container pandas version to 1.2.4 (#14524) add 992c378 [BEAM-2888] Added packages.confluent.io maven repo add b10ce99 Merge pull request #14545: [BEAM-2888] Added "packages.confluent.io" maven repo add 97af077 [BEAM-9547] DataFrame.corr cleanup (#14327) add f9d4805 [BEAM-12091] Make file staging uniform among runners add 8e66956 Merge pull request #14520: [BEAM-12091] Make file staging uniform among runners add 3a2dc20 Update pardo.md add 22a68fa Merge pull request #14542 from pkch/patch-1 add ca63e97 [BEAM-12160] Add TODO for fixing warning add ebe68fb Merge pull request #14516: [BEAM-12160] Add TODO for fixing the warning add 0d8f22d Merge pull request #14467 from [BEAM-11607] Add word count tasks add 735ba19 [BEAM-11903] Bump achilles to 6.1.0 add bda9067 Merge pull request #14519 from [BEAM-11903] Bump achilles to 6.1.0 add 1d8fbcc More tests for time types add f5021d2 [BEAM-9379] Output outside of codegen, support rows add e040608 Merge pull request #14518: [BEAM-9379] Simplify BeamCalcRel output conversion add e0d6fe7 Fix script location in build_release_candidate.sh add 15e1121 Build release candidate from tag add 74ebcf0 Run release scripts from PWD not cloned repo add 332c5ef Take build_release_candidate.sh parameters on command line for easy reruns add 5e601f5 Use SSH auth URL for pushing beam-site branch add d818b1d Merge pull request #14528: Improvements to build_release_candidate.sh to build from tag with shallow clone add 34b0184 Avoid sending zero msec counters. add 437bdee Merge pull request #14541 Avoid sending zero msec counters. add 8122b33 [BEAM-366] Populate display data in portable job representation (#14470) add d502185 [BEAM-12118] Modify QueuingBeamFnDataClient to avoid completion latency due to polling. (#14480) add 8bb55c6 [BEAM-7372] cleanup py2 codepath from apache_beam/testing (#14496) add 9c9b225 [BEAM-7372] cleanup py2 codepath from apache_beam/tool, apache_beam/utils and concerning files (#14497) add ef21f16 [BEAM-12074] Add @with_docs_from decorator for generating API docs (#14382) add de0fe2d [BEAM-12170] Handle duplicate metrics due to flatten unzipping. add 4e6bfd0 Merge pull request #14550 [BEAM-12170] Handle duplicate metrics due to flatten unzipping. add dc90a77 Add runner capabilities to Beam model proto. add 69a41e1 Runner Capabilities, go proto. add 1fa1bb9 Plumb runner capabilities to Beam SDKs. add 35759a7 Fix one more usage. add 8c9605f Merge pull request #14513 Allow the declaration of runner capabilities over FnAPI. add bb948c1 [BEAM-12029] Make WontImplementErrors more helpful (#14517) No new revisions were added by this update. Summary of changes: .test-infra/validate-runner/build.gradle | 3 + build.gradle.kts | 7 + examples/java/build.gradle | 9 + .../src/main/proto/beam_provision_api.proto| 4 + .../pipeline/src/main/proto/beam_runner_api.proto | 15 +- .../src/main/scripts/build_release_candidate.sh| 123 +++- .../scripts/download_github_actions_artifacts.py | 20 +- .../core/construction/DisplayDataTranslation.java | 13 +- .../construction/resources/PipelineResources.java | 37 +- .../construction/DisplayDataTranslationTest.java | 12 +- .../resources/PipelineResourcesTest.java | 78 ++- .../runners/core/metrics/SimpleExecutionState.java | 5 + .../runners/core/metrics/SimpleStateRegistry.java | 16 +- .../flink/FlinkPipelineExecutionEnvironment.java | 18 +- .../worker/fn/data/BeamFnDataGrpcService.java | 6 + .../environment/EmbeddedEnvironmentFactory.java| 2 + .../fnexecution/control/RemoteExecutionTest.java | 11 +- .../runners/portability/ExternalWorkerService.java | 2 + .../SparkStructuredStreamingRunner.java| 20 +- .../runners/spark/SparkCommonPipelineOptions.java | 27 +- .../beam/runners/spark/SparkPipelineRunner.java| 8 - .../org/apache/beam/runners/spark/SparkRunner.java | 18 +- .../beam/runners/twister2/Twister2Runner.java | 33 +- sdks/go/examples/build.gradle | 7 + .../model/fnexecution_v1/beam_provision_api.pb.go | 55 +- sdks/java/container/boot.go| 1 + .../beam/sdk/util/common/ReflectHelpers.java | 2 +- .../sdk/extensions/sql/impl/rel/BeamCalcRel.java | 329 +- .../beam/sdk/extensions/sql/BeamSqlDslBase.java| 28 + .../sdk/extensions/sql/BeamSqlDslUdfUdafTest
[beam] branch master updated (bb948c1 -> 06f6050)
This is an automated email from the ASF dual-hosted git repository. ningk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from bb948c1 [BEAM-12029] Make WontImplementErrors more helpful (#14517) new fd22c67 Don't use fake coders in interactive Beam. new 80d2364 Formatting fixes new 206ca1f Formatting fixes new 6e24d1d Ran yapf on changes. new 06f6050 Merge pull request #14561 from dmkozh/fake_coders The 31630 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../runners/interactive/background_caching_job.py | 5 ++-- .../runners/interactive/pipeline_fragment.py | 7 +++--- .../runners/interactive/pipeline_fragment_test.py | 4 ++-- .../runners/interactive/pipeline_instrument.py | 18 +++--- .../interactive/pipeline_instrument_test.py| 28 +- 5 files changed, 26 insertions(+), 36 deletions(-)
[beam] branch master updated (8c9605f -> bb948c1)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 8c9605f Merge pull request #14513 Allow the declaration of runner capabilities over FnAPI. add bb948c1 [BEAM-12029] Make WontImplementErrors more helpful (#14517) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/dataframe/frame_base.py| 86 +++- sdks/python/apache_beam/dataframe/frames.py| 449 +++-- sdks/python/apache_beam/dataframe/frames_test.py | 11 + sdks/python/apache_beam/dataframe/io.py| 14 +- .../dataframe/pandas_top_level_functions.py| 19 +- 5 files changed, 436 insertions(+), 143 deletions(-)
[beam] branch master updated (4e6bfd0 -> 8c9605f)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 4e6bfd0 Merge pull request #14550 [BEAM-12170] Handle duplicate metrics due to flatten unzipping. add dc90a77 Add runner capabilities to Beam model proto. add 69a41e1 Runner Capabilities, go proto. add 1fa1bb9 Plumb runner capabilities to Beam SDKs. add 35759a7 Fix one more usage. add 8c9605f Merge pull request #14513 Allow the declaration of runner capabilities over FnAPI. No new revisions were added by this update. Summary of changes: .../src/main/proto/beam_provision_api.proto| 4 ++ .../environment/EmbeddedEnvironmentFactory.java| 2 + .../fnexecution/control/RemoteExecutionTest.java | 1 + .../runners/portability/ExternalWorkerService.java | 2 + .../model/fnexecution_v1/beam_provision_api.pb.go | 55 +- sdks/java/container/boot.go| 1 + .../java/org/apache/beam/fn/harness/FnHarness.java | 16 +++ sdks/python/container/boot.go | 1 + 8 files changed, 61 insertions(+), 21 deletions(-)
[beam] branch master updated (ef21f16 -> 4e6bfd0)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from ef21f16 [BEAM-12074] Add @with_docs_from decorator for generating API docs (#14382) add de0fe2d [BEAM-12170] Handle duplicate metrics due to flatten unzipping. add 4e6bfd0 Merge pull request #14550 [BEAM-12170] Handle duplicate metrics due to flatten unzipping. No new revisions were added by this update. Summary of changes: .../beam/runners/core/metrics/SimpleExecutionState.java| 5 + .../beam/runners/core/metrics/SimpleStateRegistry.java | 14 ++ 2 files changed, 15 insertions(+), 4 deletions(-)
[beam] branch master updated (9c9b225 -> ef21f16)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 9c9b225 [BEAM-7372] cleanup py2 codepath from apache_beam/tool, apache_beam/utils and concerning files (#14497) add ef21f16 [BEAM-12074] Add @with_docs_from decorator for generating API docs (#14382) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/dataframe/__init__.py| 13 ++ sdks/python/apache_beam/dataframe/frame_base.py | 130 +++-- sdks/python/apache_beam/dataframe/frames.py | 226 +++ sdks/python/apache_beam/dataframe/frames_test.py | 32 sdks/python/apache_beam/dataframe/transforms.py | 61 +++--- sdks/python/scripts/generate_pydoc.sh| 12 +- 6 files changed, 362 insertions(+), 112 deletions(-)
[beam] branch master updated (8bb55c6 -> 9c9b225)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 8bb55c6 [BEAM-7372] cleanup py2 codepath from apache_beam/testing (#14496) add 9c9b225 [BEAM-7372] cleanup py2 codepath from apache_beam/tool, apache_beam/utils and concerning files (#14497) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/io/parquetio_test.py | 3 +-- sdks/python/apache_beam/tools/__init__.py| 1 - sdks/python/apache_beam/tools/coders_microbenchmark.py | 12 +++- .../tools/distribution_counter_microbenchmark.py | 5 - .../apache_beam/tools/fn_api_runner_microbenchmark.py| 5 - sdks/python/apache_beam/tools/map_fn_microbenchmark.py | 6 -- sdks/python/apache_beam/tools/microbenchmarks_test.py| 2 -- .../tools/runtime_type_check_microbenchmark.py | 5 - .../python/apache_beam/tools/sideinput_microbenchmark.py | 5 - .../apache_beam/tools/teststream_microbenchmark.py | 5 - sdks/python/apache_beam/tools/utils.py | 4 sdks/python/apache_beam/typehints/typecheck_test_py3.py | 4 +--- sdks/python/apache_beam/utils/__init__.py| 2 -- sdks/python/apache_beam/utils/annotations.py | 2 -- sdks/python/apache_beam/utils/annotations_test.py| 2 -- sdks/python/apache_beam/utils/counters.py| 4 sdks/python/apache_beam/utils/counters_test.py | 2 -- sdks/python/apache_beam/utils/histogram.py | 3 --- sdks/python/apache_beam/utils/histogram_test.py | 3 --- sdks/python/apache_beam/utils/interactive_utils.py | 2 -- sdks/python/apache_beam/utils/interactive_utils_test.py | 2 -- sdks/python/apache_beam/utils/plugin.py | 4 sdks/python/apache_beam/utils/processes.py | 2 -- sdks/python/apache_beam/utils/processes_test.py | 2 -- sdks/python/apache_beam/utils/profiler.py| 9 + sdks/python/apache_beam/utils/proto_utils.py | 3 --- sdks/python/apache_beam/utils/retry.py | 9 + sdks/python/apache_beam/utils/retry_test.py | 3 --- sdks/python/apache_beam/utils/sentinel.py| 2 -- sdks/python/apache_beam/utils/sharded_key.py | 2 -- sdks/python/apache_beam/utils/shared.py | 4 sdks/python/apache_beam/utils/shared_test.py | 4 sdks/python/apache_beam/utils/subprocess_server.py | 6 ++ sdks/python/apache_beam/utils/subprocess_server_test.py | 16 +--- sdks/python/apache_beam/utils/thread_pool_executor.py| 8 +--- .../apache_beam/utils/thread_pool_executor_test.py | 5 - sdks/python/apache_beam/utils/timestamp.py | 15 +-- sdks/python/apache_beam/utils/timestamp_test.py | 4 sdks/python/apache_beam/utils/urns.py| 3 --- sdks/python/apache_beam/utils/windowed_value.py | 3 --- sdks/python/apache_beam/utils/windowed_value_test.py | 2 -- 41 files changed, 16 insertions(+), 169 deletions(-)
[beam] branch master updated (d502185 -> 8bb55c6)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from d502185 [BEAM-12118] Modify QueuingBeamFnDataClient to avoid completion latency due to polling. (#14480) add 8bb55c6 [BEAM-7372] cleanup py2 codepath from apache_beam/testing (#14496) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/testing/__init__.py | 1 - sdks/python/apache_beam/testing/benchmarks/__init__.py | 1 - .../testing/benchmarks/chicago_taxi/preprocess.py | 2 -- .../testing/benchmarks/chicago_taxi/process_tfma.py | 2 -- .../apache_beam/testing/benchmarks/chicago_taxi/setup.py| 2 -- .../benchmarks/chicago_taxi/tfdv_analyze_and_validate.py| 4 .../testing/benchmarks/chicago_taxi/trainer/model.py| 4 .../testing/benchmarks/chicago_taxi/trainer/task.py | 4 .../testing/benchmarks/chicago_taxi/trainer/taxi.py | 4 .../apache_beam/testing/benchmarks/nexmark/__init__.py | 1 - .../testing/benchmarks/nexmark/models/__init__.py | 1 - .../testing/benchmarks/nexmark/models/auction_bid.py| 2 -- .../testing/benchmarks/nexmark/models/nexmark_model.py | 2 -- .../apache_beam/testing/benchmarks/nexmark/monitor.py | 2 -- .../testing/benchmarks/nexmark/nexmark_launcher.py | 4 .../apache_beam/testing/benchmarks/nexmark/nexmark_util.py | 3 --- .../benchmarks/nexmark/queries/nexmark_query_util.py| 2 -- .../testing/benchmarks/nexmark/queries/query0.py| 2 -- .../testing/benchmarks/nexmark/queries/query1.py| 2 -- .../testing/benchmarks/nexmark/queries/query10.py | 2 -- .../testing/benchmarks/nexmark/queries/query11.py | 2 -- .../testing/benchmarks/nexmark/queries/query12.py | 2 -- .../testing/benchmarks/nexmark/queries/query2.py| 2 -- .../testing/benchmarks/nexmark/queries/query3.py| 2 -- .../testing/benchmarks/nexmark/queries/query4.py| 2 -- .../testing/benchmarks/nexmark/queries/query5.py| 2 -- .../testing/benchmarks/nexmark/queries/query6.py| 3 --- .../testing/benchmarks/nexmark/queries/query7.py| 2 -- .../testing/benchmarks/nexmark/queries/query8.py| 2 -- .../testing/benchmarks/nexmark/queries/query9.py| 2 -- .../testing/benchmarks/nexmark/queries/winning_bids.py | 2 -- sdks/python/apache_beam/testing/datatype_inference.py | 12 +--- sdks/python/apache_beam/testing/datatype_inference_test.py | 5 + sdks/python/apache_beam/testing/extra_assertions.py | 13 - sdks/python/apache_beam/testing/extra_assertions_test.py| 2 -- .../apache_beam/testing/load_tests/co_group_by_key_test.py | 2 -- sdks/python/apache_beam/testing/load_tests/combine_test.py | 2 -- .../apache_beam/testing/load_tests/group_by_key_test.py | 2 -- sdks/python/apache_beam/testing/load_tests/load_test.py | 2 -- .../testing/load_tests/load_test_metrics_utils.py | 2 -- .../apache_beam/testing/load_tests/microbenchmarks_test.py | 2 -- sdks/python/apache_beam/testing/load_tests/pardo_test.py| 2 -- .../python/apache_beam/testing/load_tests/sideinput_test.py | 3 --- sdks/python/apache_beam/testing/metric_result_matchers.py | 2 -- .../apache_beam/testing/metric_result_matchers_test.py | 2 -- sdks/python/apache_beam/testing/pipeline_verifiers.py | 2 -- sdks/python/apache_beam/testing/pipeline_verifiers_test.py | 3 --- sdks/python/apache_beam/testing/synthetic_pipeline.py | 7 --- sdks/python/apache_beam/testing/synthetic_pipeline_test.py | 2 -- sdks/python/apache_beam/testing/test_pipeline.py| 2 -- sdks/python/apache_beam/testing/test_pipeline_test.py | 2 -- sdks/python/apache_beam/testing/test_stream.py | 7 +-- sdks/python/apache_beam/testing/test_stream_it_test.py | 2 -- sdks/python/apache_beam/testing/test_stream_service.py | 2 -- sdks/python/apache_beam/testing/test_stream_service_test.py | 2 -- sdks/python/apache_beam/testing/test_stream_test.py | 2 -- sdks/python/apache_beam/testing/test_utils.py | 3 --- sdks/python/apache_beam/testing/test_utils_test.py | 2 -- sdks/python/apache_beam/testing/util.py | 3 --- sdks/python/apache_beam/testing/util_test.py| 8 60 files changed, 3 insertions(+), 169 deletions(-)
[beam] branch master updated (8122b33 -> d502185)
This is an automated email from the ASF dual-hosted git repository. amaliujia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 8122b33 [BEAM-366] Populate display data in portable job representation (#14470) add d502185 [BEAM-12118] Modify QueuingBeamFnDataClient to avoid completion latency due to polling. (#14480) No new revisions were added by this update. Summary of changes: .../worker/fn/data/BeamFnDataGrpcService.java | 6 + .../sdk/fn/data/BeamFnDataInboundObserver.java | 5 + .../data/CompletableFutureInboundDataClient.java | 6 + .../apache/beam/sdk/fn/data/InboundDataClient.java | 3 + .../fn/harness/control/ProcessBundleHandler.java | 5 +- .../beam/fn/harness/data/BeamFnDataGrpcClient.java | 3 +- .../fn/harness/data/BeamFnTimerGrpcClient.java | 5 + .../fn/harness/data/QueueingBeamFnDataClient.java | 181 - .../fn/harness/data/FakeBeamFnTimerClient.java | 6 + .../harness/data/QueueingBeamFnDataClientTest.java | 121 +- 10 files changed, 296 insertions(+), 45 deletions(-)
[beam] branch master updated (437bdee -> 8122b33)
This is an automated email from the ASF dual-hosted git repository. goenka pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 437bdee Merge pull request #14541 Avoid sending zero msec counters. add 8122b33 [BEAM-366] Populate display data in portable job representation (#14470) No new revisions were added by this update. Summary of changes: .../pipeline/src/main/proto/beam_runner_api.proto | 15 +++-- .../core/construction/DisplayDataTranslation.java | 13 ++-- .../construction/DisplayDataTranslationTest.java | 12 ++-- sdks/python/apache_beam/io/gcp/bigquery.py | 3 +- sdks/python/apache_beam/pipeline.py| 4 +- sdks/python/apache_beam/pipeline_test.py | 74 ++ sdks/python/apache_beam/portability/common_urns.py | 3 + sdks/python/apache_beam/transforms/display.py | 42 8 files changed, 145 insertions(+), 21 deletions(-)
[beam] branch master updated (d818b1d -> 437bdee)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from d818b1d Merge pull request #14528: Improvements to build_release_candidate.sh to build from tag with shallow clone add 34b0184 Avoid sending zero msec counters. add 437bdee Merge pull request #14541 Avoid sending zero msec counters. No new revisions were added by this update. Summary of changes: .../apache/beam/runners/core/metrics/SimpleStateRegistry.java | 4 +++- .../beam/runners/fnexecution/control/RemoteExecutionTest.java | 10 +++--- 2 files changed, 10 insertions(+), 4 deletions(-)
[beam] branch master updated (e040608 -> d818b1d)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from e040608 Merge pull request #14518: [BEAM-9379] Simplify BeamCalcRel output conversion add e0d6fe7 Fix script location in build_release_candidate.sh add 15e1121 Build release candidate from tag add 74ebcf0 Run release scripts from PWD not cloned repo add 332c5ef Take build_release_candidate.sh parameters on command line for easy reruns add 5e601f5 Use SSH auth URL for pushing beam-site branch add d818b1d Merge pull request #14528: Improvements to build_release_candidate.sh to build from tag with shallow clone No new revisions were added by this update. Summary of changes: .../src/main/scripts/build_release_candidate.sh| 123 +++-- .../scripts/download_github_actions_artifacts.py | 20 ++-- 2 files changed, 101 insertions(+), 42 deletions(-)
[beam] annotated tag v2.29.0-RC1 updated (a467e2f -> 3748c7c)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to annotated tag v2.29.0-RC1 in repository https://gitbox.apache.org/repos/asf/beam.git. *** WARNING: tag v2.29.0-RC1 was modified! *** from a467e2f (commit) to 3748c7c (tag) tagging a467e2fc5e74a49fc3ae1fc8b371f818a4f715e9 (commit) replaces jupyterlab-sidepanel-v1.0.0 by Kenneth Knowles on Thu Apr 15 13:20:49 2021 -0700 - Log - v2.29.0-RC1 --- No new revisions were added by this update. Summary of changes:
[beam] branch master updated (bda9067 -> e040608)
This is an automated email from the ASF dual-hosted git repository. apilloud pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from bda9067 Merge pull request #14519 from [BEAM-11903] Bump achilles to 6.1.0 add 1d8fbcc More tests for time types add f5021d2 [BEAM-9379] Output outside of codegen, support rows add e040608 Merge pull request #14518: [BEAM-9379] Simplify BeamCalcRel output conversion No new revisions were added by this update. Summary of changes: .../sdk/extensions/sql/impl/rel/BeamCalcRel.java | 329 ++--- .../beam/sdk/extensions/sql/BeamSqlDslBase.java| 28 ++ .../sdk/extensions/sql/BeamSqlDslUdfUdafTest.java | 61 +++- 3 files changed, 240 insertions(+), 178 deletions(-)
[beam] branch release-2.29.0 updated (3b39b41 -> 12cde03)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch release-2.29.0 in repository https://gitbox.apache.org/repos/asf/beam.git. from 3b39b41 Merge pull request #14456: [BEAM-8696] Cherry pick #14412 to release 2.29: protobuf-java to be in-line with libraries-bom 16.3.0 add c921c1c [BEAM-9185] Publish pre-release python artifacts (RCs) to PyPI (#14325) new 12cde03 Merge pull request #14551: Cherrypick #14325 to 2.29.0 release branch: GitHub Actions changes to build RC wheels and deploy The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/build_wheels.yml | 93 +++ release/src/main/scripts/choose_rc_commit.sh | 0 .../main/scripts/deploy_release_candidate_pypi.sh | 172 + .../scripts/download_github_actions_artifacts.py | 30 ++-- .../site/content/en/contribute/release-guide.md| 31 +++- 5 files changed, 316 insertions(+), 10 deletions(-) mode change 100644 => 100755 release/src/main/scripts/choose_rc_commit.sh create mode 100755 release/src/main/scripts/deploy_release_candidate_pypi.sh
[beam] 01/01: Merge pull request #14551: Cherrypick #14325 to 2.29.0 release branch: GitHub Actions changes to build RC wheels and deploy
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch release-2.29.0 in repository https://gitbox.apache.org/repos/asf/beam.git commit 12cde03262d9b43dfad55b69c79ddb6014b8c898 Merge: 3b39b41 c921c1c Author: Kenn Knowles AuthorDate: Thu Apr 15 13:18:00 2021 -0700 Merge pull request #14551: Cherrypick #14325 to 2.29.0 release branch: GitHub Actions changes to build RC wheels and deploy .github/workflows/build_wheels.yml | 93 +++ release/src/main/scripts/choose_rc_commit.sh | 0 .../main/scripts/deploy_release_candidate_pypi.sh | 172 + .../scripts/download_github_actions_artifacts.py | 30 ++-- .../site/content/en/contribute/release-guide.md| 31 +++- 5 files changed, 316 insertions(+), 10 deletions(-)
[beam] branch master updated: [BEAM-11903] Bump achilles to 6.1.0
This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 735ba19 [BEAM-11903] Bump achilles to 6.1.0
new bda9067 Merge pull request #14519 from [BEAM-11903] Bump achilles to
6.1.0
735ba19 is described below
commit 735ba19c4de1448e6d276c0a4eb412a7e9ee92d5
Author: masahitojp
AuthorDate: Tue Apr 13 21:35:01 2021 +0900
[BEAM-11903] Bump achilles to 6.1.0
---
sdks/java/io/cassandra/build.gradle | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/sdks/java/io/cassandra/build.gradle
b/sdks/java/io/cassandra/build.gradle
index 17b693a..94b584b 100644
--- a/sdks/java/io/cassandra/build.gradle
+++ b/sdks/java/io/cassandra/build.gradle
@@ -28,8 +28,8 @@ enableJavaPerformanceTesting()
description = "Apache Beam :: SDKs :: Java :: IO :: Cassandra"
ext.summary = "IO to read and write with Apache Cassandra database"
-// compatible with all Cassandra versions up to 3.11.3
-def achilles_version = "6.0.2"
+// compatible with all Cassandra versions up to 3.11.10
+def achilles_version = "6.1.0"
dependencies {
compile library.java.vendored_guava_26_0_jre
[beam] branch master updated (ebe68fb -> 0d8f22d)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from ebe68fb Merge pull request #14516: [BEAM-12160] Add TODO for fixing the warning add 0d8f22d Merge pull request #14467 from [BEAM-11607] Add word count tasks No new revisions were added by this update. Summary of changes: build.gradle.kts | 7 +++ examples/java/build.gradle| 9 + sdks/go/examples/build.gradle | 7 +++ sdks/python/build.gradle | 12 4 files changed, 35 insertions(+)
[beam] branch master updated (22a68fa -> ebe68fb)
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 22a68fa Merge pull request #14542 from pkch/patch-1 add ca63e97 [BEAM-12160] Add TODO for fixing warning add ebe68fb Merge pull request #14516: [BEAM-12160] Add TODO for fixing the warning No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/beam/sdk/tpcds/TableSchemaJSONLoader.java | 1 + .../tpcds/src/main/java/org/apache/beam/sdk/tpcds/TpcdsSchemas.java | 1 + 2 files changed, 2 insertions(+)
[beam] branch asf-site updated: Publishing website 2021/04/15 18:01:32 at commit 22a68fa
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 400a6b7 Publishing website 2021/04/15 18:01:32 at commit 22a68fa 400a6b7 is described below commit 400a6b7119addb6effe1a1c7adb98859d39ba578 Author: jenkins AuthorDate: Thu Apr 15 18:01:33 2021 + Publishing website 2021/04/15 18:01:32 at commit 22a68fa --- .../documentation/transforms/python/elementwise/pardo/index.html | 7 --- website/generated-content/sitemap.xml | 2 +- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html index 886dc86..d5e7646 100644 --- a/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html +++ b/website/generated-content/documentation/transforms/python/elementwise/pardo/index.html @@ -109,8 +109,9 @@ You can customize what a worker does when it starts and shuts down with se You can also customize what to do when a https://beam.apache.org/documentation/runtime/model/#bundling-and-persistence>bundle of elements starts and finishes with start_bundle and finish_bundle.https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn.setup>DoFn.setup(): -Called once per DoFn instance when the DoFn instance is initialized. -setup need not to be cached, so it could be called more than once per worker. +Called whenever the DoFn instance is deserialized on the worker. This means it can be called more than once per worker because +multiple instances of a given DoFn subclass may be created (e.g., due to parallelization, or due to garbage collection after a period +of disuse). This is a good place to connect to database instances, open network connections or other resources.https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn.start_bundle>DoFn.start_bundle(): Called once per bundle of elements before calling process on the first element of the bundle. This is a good place to start keeping track of the bundle elements.https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn.process>DoFn.process(element, *args, **kwargs): @@ -176,7 +177,7 @@ teardownKnown issues:DoFn.teardown() metrics are lost.Related transformsMap behaves the same, but produces exactly one output for each input.FlatMap behaves the same as Map, but for each input it may produce zero or more outputs.Filter is useful if the function is just deciding whether to output an element or not.https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.Pardo>https://beam.apache.org/images/logos/sdks/python.png width=32px height=32px alt=Pydoc> -PydocLast updated on 2021/02/05Have you found everything you were looking for?Was it all useful and clear? Is there anything that you would like to change? Let us know!mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACKLast updated on 2021/04/14Have you found everything you were looking for?Was it all useful and clear? Is there anything that you would like to change? Let us know!mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACKhttp://www.apache.org>The Apache Software Foundation | Privacy Policy | RSS FeedApache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation. \ No newline at end of file diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 7fe2c1a..8775792 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.28.0/2021-02-22T11:40:20-08:00/categories/blog/2021-03-31T21:00:41-04:00/blog/2021-03-31T21:00:41-04:00/categories/2021-03-31T21:00:41-04:00/blog/k [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.28.0/2021-02-22T11:40:20-08:00/categories/blog/2021-03-31T21:00:41-04:00/blog/2021-03-31T21:00:41-04:00/categories/2021-03-31T21:00:41-04:00/blog/k [...] \ No newline at end of file
[beam] branch master updated: Update pardo.md
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 3a2dc20 Update pardo.md new 22a68fa Merge pull request #14542 from pkch/patch-1 3a2dc20 is described below commit 3a2dc20dddbb54abc16e9314b739045c044b5b43 Author: Max <1593648+p...@users.noreply.github.com> AuthorDate: Wed Apr 14 14:26:19 2021 -0700 Update pardo.md Fix docs for `setup` method: 1. It is not correct that it is called once per initialization; since it is not called at all when the initialization happens in the pipeline construction stage. I clarified that it's called during deserialization on the worker. 2. The phrase "setup need not to be cached, so it could be called more than once per worker." is completely unclear. Perhaps it is meant to be "The objects created in setup need not be cached by the user because it is called whenever the DoFn instance is initialized", but then the implication goes in the wrong direction. I removed it since it is hard to figure out what the intended meaning was. Instead I explained why this method may be called more than once per worker. --- .../content/en/documentation/transforms/python/elementwise/pardo.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/website/www/site/content/en/documentation/transforms/python/elementwise/pardo.md b/website/www/site/content/en/documentation/transforms/python/elementwise/pardo.md index 5021e57..efa64ab 100644 --- a/website/www/site/content/en/documentation/transforms/python/elementwise/pardo.md +++ b/website/www/site/content/en/documentation/transforms/python/elementwise/pardo.md @@ -90,8 +90,9 @@ You can also customize what to do when a starts and finishes with `start_bundle` and `finish_bundle`. * [`DoFn.setup()`](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn.setup): - Called *once per `DoFn` instance* when the `DoFn` instance is initialized. - `setup` need not to be cached, so it could be called more than once per worker. + Called whenever the `DoFn` instance is deserialized on the worker. This means it can be called more than once per worker because + multiple instances of a given `DoFn` subclass may be created (e.g., due to parallelization, or due to garbage collection after a period + of disuse). This is a good place to connect to database instances, open network connections or other resources. * [`DoFn.start_bundle()`](https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.core.html#apache_beam.transforms.core.DoFn.start_bundle):
[beam] branch master updated (97af077 -> 8e66956)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 97af077 [BEAM-9547] DataFrame.corr cleanup (#14327) add f9d4805 [BEAM-12091] Make file staging uniform among runners add 8e66956 Merge pull request #14520: [BEAM-12091] Make file staging uniform among runners No new revisions were added by this update. Summary of changes: .../construction/resources/PipelineResources.java | 37 +- .../resources/PipelineResourcesTest.java | 78 +++--- .../flink/FlinkPipelineExecutionEnvironment.java | 18 + .../SparkStructuredStreamingRunner.java| 20 +- .../runners/spark/SparkCommonPipelineOptions.java | 27 +--- .../beam/runners/spark/SparkPipelineRunner.java| 8 --- .../org/apache/beam/runners/spark/SparkRunner.java | 18 + .../beam/runners/twister2/Twister2Runner.java | 33 + .../beam/sdk/util/common/ReflectHelpers.java | 2 +- 9 files changed, 114 insertions(+), 127 deletions(-)
[beam] branch master updated (b10ce99 -> 97af077)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from b10ce99 Merge pull request #14545: [BEAM-2888] Added "packages.confluent.io" maven repo add 97af077 [BEAM-9547] DataFrame.corr cleanup (#14327) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/dataframe/frames.py | 12 +--- sdks/python/apache_beam/dataframe/frames_test.py | 13 ++--- 2 files changed, 11 insertions(+), 14 deletions(-)
[beam] branch master updated (a86dc06 -> b10ce99)
This is an automated email from the ASF dual-hosted git repository. iemejia pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from a86dc06 Bump container pandas version to 1.2.4 (#14524) add 992c378 [BEAM-2888] Added packages.confluent.io maven repo add b10ce99 Merge pull request #14545: [BEAM-2888] Added "packages.confluent.io" maven repo No new revisions were added by this update. Summary of changes: .test-infra/validate-runner/build.gradle | 3 +++ 1 file changed, 3 insertions(+)
