[beam] branch nightly-refs/heads/master updated (92ba229 -> 0faa3b9)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git. from 92ba229 [BEAM-12730] Handle corner case for TextIO with custom delimiter #15864 add 2c3e979 [BEAM-13105] playground - add shorcut hint tooltip add b97a1c3 Merge branch 'master' into BEAM-13105 add 7db9d2c Merge pull request #15814 from [BEAM-13105] [Playground] - add shortcut hint tooltip add 9109354 Merge pull request #15854 from [BEAM-13046][Playground] protobuf supporting examples API add 9bca51f [BEAM-12550] Parallelizable skew Implementation (#15809) add 48318dd Merge pull request #15782 from [BEAM-13034] [Playground] add semantics to playground app add 07aaec3 [BEAM-13099] Use vendored Calcite 1.28.0 in SQL extensions (#15836) add 6a63eb9 Use BigQuieryIO.loadProjectId in WriteRename class to create JobRefernce config and Fix side input for WriteRename add 735db24 [BEAM-13140] Fixes bug in WriteRename class add 0d172c3 Performance improvement to PubSubLiteIO to not use a streaming committer to send one commit message. add beac28d Merge pull request #15880: Performance improvement to PubSubLiteIO to not use a streaming committer to send one commit message add f41cdf4 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. add 4af1e52 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. add 0690953 Merge pull request #15839 from [BEAM-13041][Playground] Prepare files with java code add 0f94f96 [BEAM-13001] updated CHANGES.md to include msec counter for Go (#15872) add 4f3b5c0 Add window mapping to CHANGES.md (#15871) add 95e9de7 Merge pull request #15784 from [BEAM-8135] - Removing BigQueryResultsPublisher and its references add d5cd15a [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 0d2b05b Merge pull request #15856: [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 82d94b3 [BEAM-13119] Subdirectory prefix tag for Go SDK (#15881) add 156bfc2 Merge pull request #15852 from [BEAM-13102] [Playground] update playground according with the latest design changes add 66ef316 JdbcIO has a single WriteFn underlying all implementations add 2cfb6a7 Addressing comments add bee60cd Merge pull request #15862 from [BEAM-13165] JdbcIO has a single WriteFn underlying all implementations add d81ad7a Add missing parentheses for Python test example add 3c3bfa3 Update test input and imports add c79eaf8 Merge pull request #15630 Fix Python test example in documentation add 013da55 Fix typo: s/spark/twister2 add 0faa3b9 Merge pull request #15885 from ibzib/wc-twister2 No new revisions were added by this update. Summary of changes: .test-infra/jenkins/LoadTestConfig.groovy | 20 +- .../job_LoadTests_CoGBK_Dataflow_V2_Java11.groovy | 20 +- .../jenkins/job_LoadTests_CoGBK_Java.groovy| 21 +- ...ts_CoGBK_Java_spark_structured_streaming.groovy | 17 +- .../jenkins/job_LoadTests_Combine_Java.groovy | 17 +- .../job_LoadTests_Combine_Java_Smoke.groovy| 8 +- ..._Combine_Java_spark_structured_streaming.groovy | 14 +- .../job_LoadTests_GBK_Dataflow_V2_Java11.groovy| 29 +- .test-infra/jenkins/job_LoadTests_GBK_Java.groovy | 29 +- .../jenkins/job_LoadTests_GBK_Java_Smoke.groovy| 17 +- ...ests_GBK_Java_spark_structured_streaming.groovy | 26 +- .../job_LoadTests_ParDo_Dataflow_V2_Java11.groovy | 20 +- .../jenkins/job_LoadTests_ParDo_Java.groovy| 20 +- ...ts_ParDo_Java_spark_structured_streaming.groovy | 17 +- CHANGES.md | 4 + build.gradle.kts | 1 - .../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +- playground/api/v1/api.proto| 54 ++ playground/backend/cmd/server/controller.go| 34 + playground/backend/internal/api/v1/api.pb.go | 711 +++-- playground/backend/internal/api/v1/api_grpc.pb.go | 148 - playground/backend/internal/executors/executor.go | 17 +- .../backend/internal/executors/executor_builder.go | 19 + .../backend/internal/executors/executor_test.go| 36 +- .../internal/preparators/java_preparators.go | 142 .../internal/preparators/java_preparators_test.go | 102 +++ .../validator.go => preparators/preparator.go} | 9 +- playground/frontend/assets/beam_lg.png | Bin 0 -> 1000 bytes playground/frontend/assets/github.svg | 19 + playground/frontend/assets/output_bottom.svg | 19 + playground/frontend/assets/output_left.svg |
[beam] branch asf-site updated: Publishing website 2021/11/04 00:01:45 at commit 0faa3b9
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 5ac2b04 Publishing website 2021/11/04 00:01:45 at commit 0faa3b9 5ac2b04 is described below commit 5ac2b0476c83a5a685562a768bd878226f54a165 Author: jenkins AuthorDate: Thu Nov 4 00:01:45 2021 + Publishing website 2021/11/04 00:01:45 at commit 0faa3b9 --- website/generated-content/contribute/index.xml | 4 .../contribute/release-guide/index.html| 8 +++- .../pipelines/test-your-pipeline/index.html| 22 +++--- website/generated-content/sitemap.xml | 2 +- 4 files changed, 23 insertions(+), 13 deletions(-) diff --git a/website/generated-content/contribute/index.xml b/website/generated-content/contribute/index.xml index 77d4bae..14ea16e 100644 --- a/website/generated-content/contribute/index.xml +++ b/website/generated-content/contribute/index.xml @@ -1462,6 +1462,10 @@ All wheels should be published, in addition to the zip of the release source.# Optional: unlock the signing key by signing an arbitrary file. gpg --output ~/doc.sig --sign ~/.bashrc VERSION_TAG="v${RELEASE}" +# Tag for Go SDK +git tag -s "sdks/$VERSION_TAG" "$RC_TAG" +git push https://github.com/apache/beam "sdks/$VERSION_TAG" +# Tag for repo root. git tag -s "$VERSION_TAG" "$RC_TAG" git push https://github.com/apache/beam "$VERSION_TAG"After pushing the tag, the tag should be visible on Github’s Tags page.
diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html index 9115a11..89ef99f 100644 --- a/website/generated-content/contribute/release-guide/index.html +++ b/website/generated-content/contribute/release-guide/index.html @@ -399,6 +399,12 @@ All wheels should be published, in addition to the zip of the release source. gpg --output ~/doc.sig --sign ~/.bashrc VERSION_TAG="v${RELEASE}" + +# Tag for Go SDK +git tag -s "sdks/$VERSION_TAG" "$RC_TAG" +git push https://github.com/apache/beam "sdks/$VERSION_TAG" + +# Tag for repo root. git tag -s "$VERSION_TAG" "$RC_TAG" git push https://github.com/apache/beam "$VERSION_TAG" After pushing the tag, the tag should be visible on Github’s https://github.com/apache/beam/tags>Tags page.Publish release to GithubOnce the tag is uploaded, publish the release notes to Github, as follows:cd beam/release/src/main/scripts && ./publish_github_release_notes.sh @@ -413,7 +419,7 @@ If you end up getting permissions errors ask on the mailing list for assistance. Ask other contributors to do the same.Also, update https://en.wikipedia.org/wiki/Apache_Beam>the Wikipedia article on Apache Beam.Checklist to declare the process completedRelease announced on the user@ mailing list.Blog post published, if applicable.Release recorded in reporter.apache.org.Release announced on social media.Completion declared on the dev@ mailing list. [...] Once you’ve finished the release, please take a step back and look what areas of this process and be improved. Perhaps some part of the process can be simplified. Perhaps parts of this guide can be clarified.If we have specific ideas, please start a discussion on the dev@ mailing list and/or propose a pull request to update this guide. -Thanks!Last updated on 2021/10/07Have you found everything you were looking for?Was it all useful and clear? Is there anything that you would like to change? Let us know!mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACKmailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACKhttp://www.apache.org>The Apache Software Foundation | Privacy Policy | RSS FeedApache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation. \ No newline at end of file diff --git a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html index e486fb8..f162f7b 100644 --- a/website/generated-content/documentation/pipelines/test-your-pipeline/index.html +++ b/website/generated-content/documentation/pipelines/test-your-pipeline/index.html @@ -72,26 +72,26 @@ function openMenu(){addPlaceholder();blockScroll();}from apache_beam.testing.util import assert_that +}from apache_beam.testing.test_pipeline import TestPipeline +from apache_beam.testing.util import assert_that from apache_beam.testing.util import equal_to
[beam] branch master updated: Fix typo: s/spark/twister2
This is an automated email from the ASF dual-hosted git repository. ibzib pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 013da55 Fix typo: s/spark/twister2 new 0faa3b9 Merge pull request #15885 from ibzib/wc-twister2 013da55 is described below commit 013da5568a64e46204a33e58cc2233eae7e0c577 Author: Kyle Weaver AuthorDate: Wed Nov 3 15:21:13 2021 -0700 Fix typo: s/spark/twister2 --- release/src/main/groovy/quickstart-java-twister2.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release/src/main/groovy/quickstart-java-twister2.groovy b/release/src/main/groovy/quickstart-java-twister2.groovy index 7c7e382..b46538c 100644 --- a/release/src/main/groovy/quickstart-java-twister2.groovy +++ b/release/src/main/groovy/quickstart-java-twister2.groovy @@ -28,7 +28,7 @@ t.describe 'Run Apache Beam Java SDK Quickstart - Twister2' t.intent 'Gets the WordCount Example Code' QuickstartArchetype.generate(t) - t.intent 'Runs the WordCount Code with Spark runner' + t.intent 'Runs the WordCount Code with Twister2 runner' // Run the wordcount example with the Twister2 runner t.run """mvn compile exec:java -q \ -Dmaven.wagon.http.retryHandler.class=default \
[beam] branch master updated (bee60cd -> c79eaf8)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from bee60cd Merge pull request #15862 from [BEAM-13165] JdbcIO has a single WriteFn underlying all implementations add d81ad7a Add missing parentheses for Python test example add 3c3bfa3 Update test input and imports add c79eaf8 Merge pull request #15630 Fix Python test example in documentation No new revisions were added by this update. Summary of changes: .../en/documentation/pipelines/test-your-pipeline.md | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-)
[beam] branch master updated (156bfc2 -> bee60cd)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 156bfc2 Merge pull request #15852 from [BEAM-13102] [Playground] update playground according with the latest design changes new 66ef316 JdbcIO has a single WriteFn underlying all implementations new 2cfb6a7 Addressing comments new bee60cd Merge pull request #15862 from [BEAM-13165] JdbcIO has a single WriteFn underlying all implementations The 33427 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 508 +++-- 1 file changed, 268 insertions(+), 240 deletions(-)
[beam] branch master updated (82d94b3 -> 156bfc2)
This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 82d94b3 [BEAM-13119] Subdirectory prefix tag for Go SDK (#15881)
add 156bfc2 Merge pull request #15852 from [BEAM-13102] [Playground]
update playground according with the latest design changes
No new revisions were added by this update.
Summary of changes:
build.gradle.kts | 1 -
playground/frontend/assets/beam_lg.png | Bin 0 -> 1000 bytes
playground/frontend/assets/github.svg | 19 +++
playground/frontend/assets/output_bottom.svg | 19 +++
playground/frontend/assets/output_left.svg | 19 +++
playground/frontend/assets/output_right.svg| 19 +++
playground/frontend/assets/reset.svg | 19 +++
playground/frontend/assets/send_feedback.svg | 19 +++
playground/frontend/assets/shortcuts.svg | 19 +++
playground/frontend/assets/theme.svg | 19 +++
.../frontend/assets/thumb_down.svg | 10 +-
.../frontend/assets/thumb_up.svg | 16 +--
.../dropdown_button/dropdown_button.dart} | 132 +--
.../lib/components/logo/logo_component.dart| 40 --
playground/frontend/lib/config/theme.dart | 48 ---
playground/frontend/lib/constants/assets.dart | 3 +
playground/frontend/lib/constants/colors.dart | 6 +
playground/frontend/lib/constants/font_weight.dart | 2 +
.../fonts.dart}| 18 +--
playground/frontend/lib/constants/sizes.dart | 5 +-
.../modules/editor/components/editor_textarea.dart | 21 ++-
.../modules/editor/components/editor_themes.dart | 60 +
.../lib/modules/examples/example_selector.dart | 146 +++--
.../lib/modules/output/components/output_area.dart | 8 +-
.../components/output_header/output_header.dart| 2 +-
.../components/output_result.dart} | 24 ++--
.../modules/output/models/output_placement.dart| 4 +-
.../lib/modules/sdk/components/sdk_selector.dart | 53
.../components/sdk_selector_row.dart} | 40 +++---
.../shortcuts/components/shortcuts_modal.dart | 2 +-
.../shortcuts/constants/global_shortcuts.dart | 5 +-
.../components/editor_textarea_wrapper.dart| 22 ++--
.../pages/playground/components/more_actions.dart | 3 +-
.../components/playground_feedback.dart} | 42 +++---
.../components/playground_page_footer.dart | 20 ++-
.../components/playground_page_providers.dart | 4 +-
.../lib/pages/playground/playground_page.dart | 16 +--
.../pages/playground/states/examples_state.dart| 6 -
.../pages/playground/states/playground_state.dart | 3 +-
playground/frontend/pubspec.lock | 77 +++
playground/frontend/pubspec.yaml | 1 +
.../playground/states/playground_state_test.dart | 10 +-
42 files changed, 611 insertions(+), 391 deletions(-)
create mode 100644 playground/frontend/assets/beam_lg.png
copy .test-infra/metrics/sync/github/README.md =>
playground/frontend/assets/thumb_down.svg (57%)
copy .test-infra/kubernetes/kafka-cluster/README.md =>
playground/frontend/assets/thumb_up.svg (58%)
copy playground/frontend/lib/{modules/examples/example_selector.dart =>
components/dropdown_button/dropdown_button.dart} (51%)
copy
playground/frontend/lib/{pages/playground/states/example_dropdown_state.dart =>
constants/fonts.dart} (69%)
create mode 100644
playground/frontend/lib/modules/editor/components/editor_themes.dart
copy
playground/frontend/lib/modules/{actions/components/header_icon_button.dart =>
output/components/output_result.dart} (69%)
copy
playground/frontend/lib/modules/{output/components/output_header/output_header.dart
=> sdk/components/sdk_selector_row.dart} (55%)
copy
playground/frontend/lib/{components/toggle_theme_button/toggle_theme_button.dart
=> pages/playground/components/playground_feedback.dart} (56%)
[beam] branch master updated (0d2b05b -> 82d94b3)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0d2b05b Merge pull request #15856: [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 82d94b3 [BEAM-13119] Subdirectory prefix tag for Go SDK (#15881) No new revisions were added by this update. Summary of changes: release/src/main/scripts/choose_rc_commit.sh| 8 website/www/site/content/en/contribute/release-guide.md | 6 ++ 2 files changed, 14 insertions(+)
svn commit: r50785 - in /dev/beam/vendor: beam-vendor-calcite-1_28_0/ beam-vendor-grpc-1_36_0/
Author: lcwik Date: Wed Nov 3 19:47:13 2021 New Revision: 50785 Log: Remove artifacts now that those releases have been finalized. Removed: dev/beam/vendor/beam-vendor-calcite-1_28_0/ dev/beam/vendor/beam-vendor-grpc-1_36_0/
svn commit: r50784 - /release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/
Author: lcwik Date: Wed Nov 3 19:46:21 2021 New Revision: 50784 Log: Finalize beam-vendor-grpc-1_36_0:0.2 release. Added: release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/ release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip (with props) release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.asc release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.sha512 Added: release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip == Binary file - no diff available. Propchange: release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip -- svn:mime-type = application/octet-stream Added: release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.asc == --- release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.asc (added) +++ release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.asc Wed Nov 3 19:46:21 2021 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEE6tXeKT9KA90ud1ZVieaKVuNxzKIFAmDeQXwACgkQieaKVuNx +zKIogRAAtrxFa/lVrR9VESCbrx+c/4Uutyej+c+SKoBHs0tN+9VaSl/w9s5fNkAR +R808UOAAr3Lf+oUDAvnR+1zUyMo4ztY3X2actZgIjKlygQPNJ6SRYMsJMa9A/Er9 +hqsxAuxkooNPSWF6b+hAZ4PudoeTXkLq63JcBCiQ//zz/i6JP0zmY+gvX2DS3F0o +EdyQdrDVI1r3J5fetFuZTjkdjpMrw7cbni16eyASuizSqhGr42k+XiSXdw1qbetz +01CTkCjU2Zuwimeo4T2zTJSABQlWbPDMT/64GEEuMiIldozjgnmgyouwI5bC7Hd6 +StMcH31/H29sigGDhWeYmvSLr86wbw5zf4MKaJyqzLkmIEkOkZIq5IeqkOaBQGM1 ++iW+yk4MyWcSEiJsNqCcwVrWz3ZofQ/XcQK1ZUssjJ2E9KyYshYmuX5EcCCuzSAK +Y9fwCMxApY6qEh8JjoJ6pqha9NhZ+ph47yGuUJ9VzQ3NCMC5VKparPBAEWr0dI/L +IcPlrdJb3fApaByonJ2rRpRAtup1naWwoybRx5jtDRMKXZWg/bbzl10ajOr1yJ2T +SlC67iKFizChuXykEd3o1pvrf6kEAZtRWMddLAGNQY0HetxPPuACRU/IzSgxNAam +1iiSR8vD3mz/2T7wVrlSBu25n2eianfp+buNeBpSW2rHxWcAIAo= +=iQcx +-END PGP SIGNATURE- Added: release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.sha512 == --- release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.sha512 (added) +++ release/beam/vendor/beam-vendor-grpc-1_36_0/0.2/apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip.sha512 Wed Nov 3 19:46:21 2021 @@ -0,0 +1 @@ +84fa501df46f8b4de9c08d748d220751314f5b9a9722bdb5ff84097c6a66a3748ef1d8d4c0e301a12eba71788e86c2d00444ec87518ba16a1695ea2a5ebae73a apache-beam-37fd13e738185822af132fe2e457b618e766802f-source-release.zip
svn commit: r50783 - in /release/beam/vendor/beam-vendor-calcite-1_28_0: ./ 0.1/
Author: lcwik Date: Wed Nov 3 19:44:45 2021 New Revision: 50783 Log: Finalize beam-vendor-calcite-1_28_0:0.1 release Added: release/beam/vendor/beam-vendor-calcite-1_28_0/ release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/ release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip (with props) release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.asc release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.sha512 Added: release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip == Binary file - no diff available. Propchange: release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip -- svn:mime-type = application/octet-stream Added: release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.asc == --- release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.asc (added) +++ release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.asc Wed Nov 3 19:44:45 2021 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEHRkMhbj2tLoSdv/JRJtqPVWNILAFAmF8ao4ACgkQRJtqPVWN +ILCm8g/8DpFggX5w6ynQfY+JnP8g1QvVILE+su+Rs1VD7Gi0IthbYPU6InMe1DiB +QWiChrHKd3jFCP+/lEsnupiufBaM4Ugnyhdbw5QKOVnCHloFoIIr7xGR3Vk6vPko +agsOkB0SkOv/U3FHcGtbwfQVd/aGl7RzdLjBdjKcAD05QbY4ooFmB+ERj6oEilqV +0NxzKNZOWDYxTrSxI+t1V7WO/GIoVKwLMzjqzl0Z63qfj0aNheL6o2kHiRU5imW0 +lmQ+RmYN5HRRiaIlKN272AMgHSxMPsEE/XA8sPTTLDIvushbe5VW+0KhFiod0Z4X +gg+PCwF7ah30Z+lRXgbHgI5vtU3TZv/l9yodI1lF1p190HKoy9GnrdYaswJwieEn +d3HsytNXI3LBk/wDHd+RFxrvVrkw8Bb+aCSZ7vwVVXm91J/zbzxZAVhvUOPPKJVT +egLR8rhKVUXK/TxYbgoxa8443leff+o2r4e2n+a0MrkiAc0ppDsUx3nEMsAF+VJr +cSMWqiaDC3U9xliNFwD32LYBClAWDK6fQTmNT/P/dGFb+pR1RueQQhPlyvdZNpf7 +UXepau+dFppY0CUSz9g3tirT0m5drEkWZ3atJyeq6a7RUXODgX+XuYbzIwxPsQ5R +y5elbYCv1eupxusevty/eZHQKlaYmBB8wj8mJ44NhPkbq0vjlJc= +=OY+B +-END PGP SIGNATURE- Added: release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.sha512 == --- release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.sha512 (added) +++ release/beam/vendor/beam-vendor-calcite-1_28_0/0.1/apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip.sha512 Wed Nov 3 19:44:45 2021 @@ -0,0 +1 @@ +1d5070bc36c4928911fc0887966771f2e179b998edc1be0ee11aa66abd02aac747206fbe287e9e0c3f6d25aa99eb64dc37427d92213a0853b14401c8c9bbceca apache-beam-cb9fb5110aea50c3cbc2d7b49191b32d38a0307b-source-release.zip
[beam] branch master updated (95e9de7 -> 0d2b05b)
This is an automated email from the ASF dual-hosted git repository. michel pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 95e9de7 Merge pull request #15784 from [BEAM-8135] - Removing BigQueryResultsPublisher and its references add d5cd15a [BEAM-13157] support hadoop configuration on ParquetIO.Parse add 0d2b05b Merge pull request #15856: [BEAM-13157] support hadoop configuration on ParquetIO.Parse No new revisions were added by this update. Summary of changes: .../parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java | 1 + 1 file changed, 1 insertion(+)
[beam] branch master updated (4f3b5c0 -> 95e9de7)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 4f3b5c0 Add window mapping to CHANGES.md (#15871) add 95e9de7 Merge pull request #15784 from [BEAM-8135] - Removing BigQueryResultsPublisher and its references No new revisions were added by this update. Summary of changes: .test-infra/jenkins/LoadTestConfig.groovy | 20 +--- .../job_LoadTests_CoGBK_Dataflow_V2_Java11.groovy | 20 +--- .../jenkins/job_LoadTests_CoGBK_Java.groovy| 21 +--- ...ts_CoGBK_Java_spark_structured_streaming.groovy | 17 +-- .../jenkins/job_LoadTests_Combine_Java.groovy | 17 +-- .../job_LoadTests_Combine_Java_Smoke.groovy| 8 +- ..._Combine_Java_spark_structured_streaming.groovy | 14 +-- .../job_LoadTests_GBK_Dataflow_V2_Java11.groovy| 29 +- .test-infra/jenkins/job_LoadTests_GBK_Java.groovy | 29 +- .../jenkins/job_LoadTests_GBK_Java_Smoke.groovy| 17 +-- ...ests_GBK_Java_spark_structured_streaming.groovy | 26 + .../job_LoadTests_ParDo_Dataflow_V2_Java11.groovy | 20 +--- .../jenkins/job_LoadTests_ParDo_Java.groovy| 20 +--- ...ts_ParDo_Java_spark_structured_streaming.groovy | 17 +-- .../provider/bigquery/BigQueryIOPushDownIT.java| 1 - .../beam/sdk/bigqueryioperftests/BigQueryIOIT.java | 1 - .../java/org/apache/beam/sdk/io/avro/AvroIOIT.java | 1 - .../apache/beam/sdk/io/parquet/ParquetIOIT.java| 1 - .../java/org/apache/beam/sdk/io/text/TextIOIT.java | 1 - .../apache/beam/sdk/io/tfrecord/TFRecordIOIT.java | 1 - .../java/org/apache/beam/sdk/io/xml/XmlIOIT.java | 1 - .../sdk/io/hadoop/format/HadoopFormatIOIT.java | 2 - .../java/org/apache/beam/sdk/io/jdbc/JdbcIOIT.java | 2 - .../org/apache/beam/sdk/io/kafka/KafkaIOIT.java| 2 - .../apache/beam/sdk/io/mongodb/MongoDBIOIT.java| 2 - .../org/apache/beam/sdk/loadtests/LoadTest.java| 24 - .../apache/beam/sdk/loadtests/LoadTestOptions.java | 16 --- .../java/org/apache/beam/sdk/nexmark/Main.java | 27 - .../beam/sdk/nexmark/PerfsToBigQueryTest.java | 88 .../beam/sdk/testutils/metrics/IOITMetrics.java| 19 .../publishing/BigQueryResultsPublisher.java | 65 .../sdk/testutils/fakes/FakeBigQueryClient.java| 68 .../fakes/FakeBigQueryResultsPublisher.java| 52 -- .../publishing/BigQueryResultsPublisherTest.java | 114 - 34 files changed, 35 insertions(+), 728 deletions(-) delete mode 100644 sdks/java/testing/nexmark/src/test/java/org/apache/beam/sdk/nexmark/PerfsToBigQueryTest.java delete mode 100644 sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/publishing/BigQueryResultsPublisher.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/fakes/FakeBigQueryClient.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/fakes/FakeBigQueryResultsPublisher.java delete mode 100644 sdks/java/testing/test-utils/src/test/java/org/apache/beam/sdk/testutils/publishing/BigQueryResultsPublisherTest.java
[beam] branch master updated (0f94f96 -> 4f3b5c0)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0f94f96 [BEAM-13001] updated CHANGES.md to include msec counter for Go (#15872) add 4f3b5c0 Add window mapping to CHANGES.md (#15871) No new revisions were added by this update. Summary of changes: CHANGES.md | 2 ++ 1 file changed, 2 insertions(+)
[beam] branch master updated (0690953 -> 0f94f96)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 0690953 Merge pull request #15839 from [BEAM-13041][Playground] Prepare files with java code add 0f94f96 [BEAM-13001] updated CHANGES.md to include msec counter for Go (#15872) No new revisions were added by this update. Summary of changes: CHANGES.md | 1 + 1 file changed, 1 insertion(+)
[beam] branch master updated (4af1e52 -> 0690953)
This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 4af1e52 [BEAM-13015] Use a network based channel instead of an
inmemory one which allows us to more faithfully mirror performance/overhead of
the SDK harness.
add 0690953 Merge pull request #15839 from [BEAM-13041][Playground]
Prepare files with java code
No new revisions were added by this update.
Summary of changes:
playground/backend/internal/executors/executor.go | 17 ++-
.../backend/internal/executors/executor_builder.go | 19 +++
.../backend/internal/executors/executor_test.go| 36 --
.../internal/preparators/java_preparators.go | 142 +
.../internal/preparators/java_preparators_test.go | 102 +++
.../validator.go => preparators/preparator.go} | 9 +-
6 files changed, 308 insertions(+), 17 deletions(-)
create mode 100644 playground/backend/internal/preparators/java_preparators.go
create mode 100644
playground/backend/internal/preparators/java_preparators_test.go
copy playground/backend/internal/{validators/validator.go =>
preparators/preparator.go} (82%)
[beam] branch master updated (beac28d -> 4af1e52)
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from beac28d Merge pull request #15880: Performance improvement to PubSubLiteIO to not use a streaming committer to send one commit message add f41cdf4 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. add 4af1e52 [BEAM-13015] Use a network based channel instead of an inmemory one which allows us to more faithfully mirror performance/overhead of the SDK harness. No new revisions were added by this update. Summary of changes: .../java/org/apache/beam/fn/harness/ProcessBundleBenchmark.java | 8 1 file changed, 4 insertions(+), 4 deletions(-)
[beam] branch master updated (735db24 -> beac28d)
This is an automated email from the ASF dual-hosted git repository.
udim pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 735db24 [BEAM-13140] Fixes bug in WriteRename class
add 0d172c3 Performance improvement to PubSubLiteIO to not use a
streaming committer to send one commit message.
add beac28d Merge pull request #15880: Performance improvement to
PubSubLiteIO to not use a streaming committer to send one commit message
No new revisions were added by this update.
Summary of changes:
...nitialOffsetReader.java => BlockingCommitter.java} | 8 ++--
.../internal/PerSubscriptionPartitionSdf.java | 14 --
.../gcp/pubsublite/internal/SubscribeTransform.java | 3 +--
.../gcp/pubsublite/internal/SubscriberAssembler.java | 19 ++-
.../internal/PerSubscriptionPartitionSdfTest.java | 16 +++-
5 files changed, 20 insertions(+), 40 deletions(-)
copy
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsublite/internal/{InitialOffsetReader.java
=> BlockingCommitter.java} (84%)
[beam] branch master updated (07aaec3 -> 735db24)
This is an automated email from the ASF dual-hosted git repository. ningk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 07aaec3 [BEAM-13099] Use vendored Calcite 1.28.0 in SQL extensions (#15836) add 6a63eb9 Use BigQuieryIO.loadProjectId in WriteRename class to create JobRefernce config and Fix side input for WriteRename add 735db24 [BEAM-13140] Fixes bug in WriteRename class No new revisions were added by this update. Summary of changes: .../org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java | 8 +--- .../apache/beam/sdk/io/gcp/bigquery/WriteRename.java| 17 + .../apache/beam/sdk/io/gcp/bigquery/WriteTables.java| 5 - .../beam/sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 3 ++- 4 files changed, 24 insertions(+), 9 deletions(-)
[beam] branch master updated (48318dd -> 07aaec3)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 48318dd Merge pull request #15782 from [BEAM-13034] [Playground] add semantics to playground app add 07aaec3 [BEAM-13099] Use vendored Calcite 1.28.0 in SQL extensions (#15836) No new revisions were added by this update. Summary of changes: CHANGES.md | 1 + .../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +- sdks/java/extensions/sql/build.gradle | 12 +-- .../sql/example/BeamSqlDataCatalogExample.java | 2 +- .../datacatalog/DataCatalogTableProvider.java | 2 +- .../sql/meta/provider/datacatalog/SchemaUtils.java | 4 +- .../provider/bigquery/BigQueryIOPushDownIT.java| 6 +- .../extensions/sql/src/main/codegen/config.fmpp| 8 +- .../beam/sdk/extensions/sql/SqlTransform.java | 4 +- .../extensions/sql/TableNameExtractionUtils.java | 17 ++-- .../sdk/extensions/sql/impl/BeamCalciteSchema.java | 16 ++-- .../sql/impl/BeamCalciteSchemaFactory.java | 20 ++--- .../sdk/extensions/sql/impl/BeamCalciteTable.java | 30 +++ .../beam/sdk/extensions/sql/impl/BeamSqlEnv.java | 14 +-- .../sql/impl/BeamSqlPipelineOptionsRegistrar.java | 2 +- .../extensions/sql/impl/BeamTableStatistics.java | 14 +-- .../sql/impl/CalciteConnectionWrapper.java | 16 ++-- .../extensions/sql/impl/CalciteFactoryWrapper.java | 29 --- .../extensions/sql/impl/CalciteQueryPlanner.java | 74 .../sdk/extensions/sql/impl/JavaUdfLoader.java | 2 +- .../sdk/extensions/sql/impl/JdbcConnection.java| 8 +- .../beam/sdk/extensions/sql/impl/JdbcDriver.java | 28 +++--- .../beam/sdk/extensions/sql/impl/JdbcFactory.java | 37 .../beam/sdk/extensions/sql/impl/QueryPlanner.java | 8 +- .../extensions/sql/impl/ScalarFunctionImpl.java| 46 +- .../extensions/sql/impl/TableResolutionUtils.java | 6 +- .../beam/sdk/extensions/sql/impl/UdafImpl.java | 12 +-- .../beam/sdk/extensions/sql/impl/UdfImpl.java | 10 +-- .../sql/impl/UdfImplReflectiveFunctionBase.java| 16 ++-- ...SqlUserDefinedSQLNativeTableValuedFunction.java | 14 +-- .../beam/sdk/extensions/sql/impl/cep/CEPCall.java | 10 +-- .../sdk/extensions/sql/impl/cep/CEPFieldRef.java | 2 +- .../sdk/extensions/sql/impl/cep/CEPLiteral.java| 2 +- .../sdk/extensions/sql/impl/cep/CEPOperation.java | 8 +- .../sdk/extensions/sql/impl/cep/CEPOperator.java | 4 +- .../sdk/extensions/sql/impl/cep/CEPPattern.java| 2 +- .../beam/sdk/extensions/sql/impl/cep/CEPUtils.java | 16 ++-- .../beam/sdk/extensions/sql/impl/cep/OrderKey.java | 2 +- .../extensions/sql/impl/parser/BeamSqlParser.java | 8 +- .../sql/impl/parser/SqlCheckConstraint.java| 18 ++-- .../sql/impl/parser/SqlColumnDeclaration.java | 23 ++--- .../sql/impl/parser/SqlCreateExternalTable.java| 28 +++--- .../sql/impl/parser/SqlCreateFunction.java | 32 +++ .../extensions/sql/impl/parser/SqlDdlNodes.java| 27 +++--- .../extensions/sql/impl/parser/SqlDropObject.java | 25 +++--- .../extensions/sql/impl/parser/SqlDropTable.java | 10 +-- .../sql/impl/parser/SqlSetOptionBeam.java | 18 ++-- .../extensions/sql/impl/planner/BeamCostModel.java | 8 +- .../sql/impl/planner/BeamJavaTypeFactory.java | 12 +-- .../sql/impl/planner/BeamRelDataTypeSystem.java| 6 +- .../sql/impl/planner/BeamRelMetadataQuery.java | 6 +- .../extensions/sql/impl/planner/BeamRuleSets.java | 14 +-- .../sql/impl/planner/NodeStatsMetadata.java| 12 +-- .../sql/impl/planner/RelMdNodeStats.java | 14 +-- .../sql/impl/rel/AbstractBeamCalcRel.java | 18 ++-- .../sql/impl/rel/BeamAggregationRel.java | 18 ++-- .../sdk/extensions/sql/impl/rel/BeamCalcRel.java | 68 +++ .../extensions/sql/impl/rel/BeamCoGBKJoinRel.java | 16 ++-- .../sql/impl/rel/BeamEnumerableConverter.java | 38 - .../sdk/extensions/sql/impl/rel/BeamIOSinkRel.java | 25 +++--- .../extensions/sql/impl/rel/BeamIOSourceRel.java | 20 ++--- .../extensions/sql/impl/rel/BeamIntersectRel.java | 12 +-- .../sdk/extensions/sql/impl/rel/BeamJoinRel.java | 39 + .../sql/impl/rel/BeamLogicalConvention.java| 12 +-- .../sdk/extensions/sql/impl/rel/BeamMatchRel.java | 27 +++--- .../sdk/extensions/sql/impl/rel/BeamMinusRel.java | 12 +-- .../sql/impl/rel/BeamPushDownIOSourceRel.java | 14 +-- .../sdk/extensions/sql/impl/rel/BeamRelNode.java | 8 +- .../sql/impl/rel/BeamSetOperatorRelBase.java | 2 +- .../sql/impl/rel/BeamSideInputJoinRel.java | 16 ++-- .../sql/impl/rel/BeamSideInputLookupJoinRel.java | 14 +-- .../sdk/extensions/sql/impl/rel/BeamSortRel.java | 33 .../extensions/sql/impl/rel/BeamSqlRelUtils.java | 23 +++-- .../sql/impl/rel/BeamTableFunction
[beam] branch master updated (9bca51f -> 48318dd)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 9bca51f [BEAM-12550] Parallelizable skew Implementation (#15809) add 48318dd Merge pull request #15782 from [BEAM-13034] [Playground] add semantics to playground app No new revisions were added by this update. Summary of changes: .../modules/editor/components/editor_textarea.dart | 19 +-- .../output_header/output_placements.dart | 21 +--- .../modules/output/models/output_placement.dart| 17 ++ .../lib/modules/sdk/components/sdk_selector.dart | 60 -- playground/frontend/web/index.html | 2 +- 5 files changed, 79 insertions(+), 40 deletions(-)
[beam] branch master updated (9109354 -> 9bca51f)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 9109354 Merge pull request #15854 from [BEAM-13046][Playground] protobuf supporting examples API add 9bca51f [BEAM-12550] Parallelizable skew Implementation (#15809) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/dataframe/frames.py | 73 +++- sdks/python/apache_beam/dataframe/frames_test.py | 5 +- 2 files changed, 72 insertions(+), 6 deletions(-)
[beam] branch master updated (7db9d2c -> 9109354)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 7db9d2c Merge pull request #15814 from [BEAM-13105] [Playground] - add shortcut hint tooltip add 9109354 Merge pull request #15854 from [BEAM-13046][Playground] protobuf supporting examples API No new revisions were added by this update. Summary of changes: playground/api/v1/api.proto| 54 ++ playground/backend/cmd/server/controller.go| 34 + playground/backend/internal/api/v1/api.pb.go | 711 +++-- playground/backend/internal/api/v1/api_grpc.pb.go | 148 - playground/frontend/lib/api/v1/api.pb.dart | 397 +++- playground/frontend/lib/api/v1/api.pbenum.dart | 33 +- playground/frontend/lib/api/v1/api.pbgrpc.dart | 84 +++ playground/frontend/lib/api/v1/api.pbjson.dart | 100 ++- .../code_client/grpc_code_client.dart | 5 + 9 files changed, 1483 insertions(+), 83 deletions(-)
[beam] branch master updated (92ba229 -> 7db9d2c)
This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 92ba229 [BEAM-12730] Handle corner case for TextIO with custom
delimiter #15864
new 2c3e979 [BEAM-13105] playground - add shorcut hint tooltip
new b97a1c3 Merge branch 'master' into BEAM-13105
new 7db9d2c Merge pull request #15814 from [BEAM-13105] [Playground] -
add shortcut hint tooltip
The 33406 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../actions/components/new_example_action.dart | 19 ++--
.../modules/actions/components/reset_action.dart | 17 ++--
.../lib/modules/editor/components/run_button.dart | 29 +++---
.../modules/shortcuts/components/shortcut_row.dart | 20 +---
.../components/shortcut_tooltip.dart} | 27 ++---
.../shortcuts/constants/global_shortcuts.dart | 112 +++--
.../shortcuts/utils/shortcuts_display_name.dart| 22 +++-
7 files changed, 131 insertions(+), 115 deletions(-)
copy
playground/frontend/lib/modules/{actions/components/header_icon_button.dart =>
shortcuts/components/shortcut_tooltip.dart} (67%)
copy
runners/core-java/src/main/java/org/apache/beam/runners/core/GroupAlsoByWindowsAggregators.java
=> playground/frontend/lib/modules/shortcuts/utils/shortcuts_display_name.dart
(66%)
