(beam) branch asf-site updated: Publishing website 2024/02/01 05:39:42 at commit 5188e37
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new ac1c5d90829 Publishing website 2024/02/01 05:39:42 at commit 5188e37 ac1c5d90829 is described below commit ac1c5d908299a93a35fc5632f92f450b43f1b091 Author: runner AuthorDate: Thu Feb 1 05:39:42 2024 + Publishing website 2024/02/01 05:39:42 at commit 5188e37 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 6285044330b..2de11c538a9 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-31T23:23:50+01:00/categories/blog/2024-01-31T23:23:50+01:00/blog/2024-01-31T23:23:50+01:00/categories/2024-01-31T23:23:50+01:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-31T22:14:09-05:00/categories/blog/2024-01-31T22:14:09-05:00/blog/2024-01-31T22:14:09-05:00/categories/2024-01-31T22:14:09-05:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 64897ac4260 Updating config from bot 64897ac4260 is described below commit 64897ac4260b6c242b8feb4d0fe2ceb0f09c6b3d Author: github-actions AuthorDate: Thu Feb 1 05:05:40 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index ff4c36b077a..fa19efb4f0c 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -2,7 +2,7 @@ "label": "go", "dateOfLastReviewAssignment": { "lostluck": 1706758526034, -"jrmccluskey": 1706679270657, +"jrmccluskey": 1706763937839, "youngoli": 1657688896155, "damccorm": 1680501930289, "riteshghorse": 1706744071021
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4d3f6e07fcb Updating config from bot 4d3f6e07fcb is described below commit 4d3f6e07fcb096342e4479268c00836b10a7e454 Author: github-actions AuthorDate: Thu Feb 1 05:05:39 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30180.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30180.json b/scripts/ci/pr-bot/state/pr-state/pr-30180.json new file mode 100644 index 000..1c2c2e31975 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30180.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"go": "jrmccluskey" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch dependabot/go_modules/sdks/golang.org/x/sys-0.16.0 created (now df7e6f27b58)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/golang.org/x/sys-0.16.0 in repository https://gitbox.apache.org/repos/asf/beam.git at df7e6f27b58 Bump golang.org/x/sys from 0.15.0 to 0.16.0 in /sdks No new revisions were added by this update.
(beam) branch nightly-refs/heads/master updated (1df9d6bba91 -> 4f09e62f520)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) add 4c5ee349f9b [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 36cab4ffe76 Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 41dee464db4 Make ReadFromBigQueryRequest id more randomized (#30156) add 81513f10d60 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#30162) add 89d1c06e1ea Fix Fhir IO Search IT (#30158) add a221f98a5f4 fix: fix error handling in segment reader and add an integration test (#30147) add ddb93272ab6 Fix git trusted directory errors in website build. (#30171) add 30917ff7b3e Don't use '-' as a delimiter in BigQuerySinkMetrics (#30118) add fb0762352c2 Periodically send perWorkerMetrics on the WorkerMessages API (#30135) add 4f09e62f520 Improve varint encoding throughput with unrolled loop (#29689) No new revisions were added by this update. Summary of changes: ...n.json => beam_PostCommit_Java_DataflowV2.json} | 0 .../dataflow/worker/DataflowWorkUnitClient.java| 24 +- .../MetricsToPerStepNamespaceMetricsConverter.java | 27 +- .../dataflow/worker/StreamingDataflowWorker.java | 40 ++- .../worker/StreamingStepMetricsContainer.java | 8 +- .../runners/dataflow/worker/WorkUnitClient.java| 17 +- .../dataflow/worker/streaming/StageInfo.java | 9 + .../worker/DataflowWorkUnitClientTest.java | 37 ++- ...ricsToPerStepNamespaceMetricsConverterTest.java | 16 +- .../worker/StreamingStepMetricsContainerTest.java | 4 +- sdks/go.mod| 18 +- sdks/go.sum| 36 +-- .../apache/beam/sdk/jmh/util/VarIntBenchmark.java | 303 + .../beam/sdk/transforms/join/CoGbkResult.java | 3 +- .../main/java/org/apache/beam/sdk/util/VarInt.java | 62 - .../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 37 +-- .../sdk/io/gcp/bigtable/BigtableServiceImpl.java | 20 +- .../io/gcp/bigquery/BigQuerySinkMetricsTest.java | 59 ++-- .../beam/sdk/io/gcp/bigtable/BigtableReadIT.java | 93 ++- .../beam/sdk/io/gcp/healthcare/FhirIOSearchIT.java | 9 +- sdks/python/apache_beam/io/gcp/bigquery.py | 6 +- website/Dockerfile | 2 + 22 files changed, 699 insertions(+), 131 deletions(-) copy .github/trigger_files/{beam_PostCommit_Python.json => beam_PostCommit_Java_DataflowV2.json} (100%) create mode 100644 sdks/java/core/jmh/src/main/java/org/apache/beam/sdk/jmh/util/VarIntBenchmark.java
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new ecdd919c4e0 Updating config from bot ecdd919c4e0 is described below commit ecdd919c4e078f16e431703ba7ae697d7b4da31b Author: github-actions AuthorDate: Thu Feb 1 03:35:29 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 170f7ab0974..ff4c36b077a 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -1,7 +1,7 @@ { "label": "go", "dateOfLastReviewAssignment": { -"lostluck": 1706648775228, +"lostluck": 1706758526034, "jrmccluskey": 1706679270657, "youngoli": 1657688896155, "damccorm": 1680501930289,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new fdadcf42286 Updating config from bot fdadcf42286 is described below commit fdadcf42286614d24078d1d0c099af340bcaa249 Author: github-actions AuthorDate: Thu Feb 1 03:35:27 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29913.json | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29913.json b/scripts/ci/pr-bot/state/pr-state/pr-29913.json index 242a48d7d3b..8ea94810368 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29913.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29913.json @@ -1,7 +1,9 @@ { - "commentedAboutFailingChecks": true, - "reviewersAssignedForLabels": {}, - "nextAction": "Author", + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"go": "lostluck" + }, + "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], "committerAssigned": false
(beam) branch dependabot/go_modules/sdks/google.golang.org/protobuf-1.32.0 updated (3e8ca5012dd -> 2c8795eacb8)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/google.golang.org/protobuf-1.32.0 in repository https://gitbox.apache.org/repos/asf/beam.git omit 3e8ca5012dd Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /sdks add e6e180b6896 Switch to use self hosted runners for linux unit tests (#29907) add 02e26c1828f raise error if transforms are provided while read mode in MLTransform (#29909) add a5f0cda4f3b Fix: creating templates with the Typescript SDK (#29887) add 308223f447f [release] 2.53.0 Website Updates (#29856) add 24e5f3a6c01 [release] Add extra context for release branch cut action input (#29920) add f88c78b79ca [release] Update Release Guide Instructions (#29919) add 8bd4a1e525b [Typescript] stop service when done (#29929) add c6f0d9c462a Add notebooks for text embeddings (#29893) add a6948ad71e3 Install beam from pypi instead of github in mltransform notebooks (#29934) add b9ae59a9978 Remove TODO in the notebook (#29935) add 0b0d97325c1 Update pmc finalization in release-guide.md (#29936) add f79eaddf574 Preparing the data processing notebooks for import to devsite (#29937) add 3260a7bf25a Final edit on data preprocessing notebooks (#29940) add 4cf1b15024d Updated the doc with Security Fixes for 2.53.0 (#29941) add 66f98a0670c Update finalize_release.yml (#29918) add 75cfbee1591 Update MLTransform docs (#29910) add c9e036e40e4 [Python BQ] Retry get_table for quota errors (#28820) add 992e4fd5019 upgrade prettier add 63503391562 Merge pull request #29942 [Typescript] upgrade prettier add 094eb7dc5d7 Increase granularity of DICOM IO Unit Tests (#29956) add b666c6434dc [Dataflow Streaming] Make SideInputCache bytes and expiry configurable (#29871) add 97ca4430d97 remove threadlocal for DESERIALIZATION_CONTEXT and use copy instead (#29952) add f311d327aac Bump org.checkerframework:checkerframework-gradle-plugin add bb0a2037970 Merge pull request #29811: Bump org.checkerframework:checkerframework-gradle-plugin from 0.6.34 to 0.6.37 add 8fb06da4ce8 [RRIO] Build RequestResponseIO and related dependency changes (#29710) add 5e8fef44939 Bump timeout to 120 minutes (#29964) add 8aa16df9bd1 Bump runner v2 beam-master to 20240109 (#29965) add 6066af3dbd5 [YAML] - Kafka Proto String schema (#29835) add a3e065366e9 Bump GCP Libraries BOM to v26.29.0 (#29922) add 2eb375be82f Downgrade artifact version in build_wheels (#29960) add 0a368056904 Add support for pandas 2.0 (#28636) add ee3d0a3c5ca [YAML] Add AssignTimestamps transform. (#29969) add eb7addc1a5b [yaml] Replace pipeline_spec with yaml_pipeline in main.py add 0c955dfdb7e Merge pull request #29916 [YAML] Replace pipeline_spec with yaml_pipeline in main.py add a80522ef5e7 update dependencies (#29974) add 0bc1556d5ed Allows Java SDK to pass through external coders during proto translation without fully interpretting them (#29859) add 9697ae818d2 Update container tag (#29977) add ba73001d687 Feature/bigtable error handling (#29885) add efd50ede0dd update CHANGES.md with error handling for fileIO and bigtableIO (#29983) add 96d092b31ec Only auth and push on master branch (#29995) add 812684f3cef [#29772][prism] Handle EventTime Timers. (#29900) add 68849914cfc Fixes to environment cloning. add 0fb42e5ee32 [YAML] Better support for inline PyTransforms. add cfec51224e4 Merge pull request #29958 [YAML] Better support for inline PyTransforms. add d70f516ce20 fix: Spanner PostgreSQL should filter on 'public' schema (#29984) add db0a4deea2f [IT] Pub/Sub RM should delete subscriptions that were created by Beam (#29957) add c38dc77283f Upgrade Parquet to 0.13.1 (#29953) add b0d74c3c6f3 Generate per transform schema for validation. add 48650f12894 Conditional validation. add e50b7cc89d2 Conditionally validate transform schemas. add cb825565a9d Better line numbers for messages. add 1c7f8392c3b Fix imports. add 7cb559f2677 Merge pull request #29830 [YAML] More complete schema validation. add 0d58d23f2d9 [#29180][prism] Return total element count to progress loop. Split less aggressively. (#29968) add 057c1d56f88 Change colab links (#30006) add 6cd53fad82a Support DLQ in MLTransform (#29938) add 308c38e243c [Typescript] Worker improvements (#29903) add dd5bbb2d77d Add a read timeout and cache BigQueryIOMetadata (#29662) add c11b7c1f7cb Update scaling-streaming-workload.md (#29921) add cf201198df9 sickbay new reshuffle test on runners that do not support it add 96a9830ac27 Merge pull request #29994: sickbay new reshuffle test on runners that do not support it add 534015c0b8e [Typescript] creating dataflow job template fix (#29928) add a41fd2cba03
(beam) branch dependabot/go_modules/sdks/github.com/opencontainers/runc-1.1.12 deleted (was a7ad749b1ff)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/opencontainers/runc-1.1.12 in repository https://gitbox.apache.org/repos/asf/beam.git was a7ad749b1ff Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks (#30175)
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 5188e37ed44 Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks (#30175) 5188e37ed44 is described below commit 5188e37ed445bbe01ce7e71ee8177c2fde8c91ad Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Wed Jan 31 22:14:09 2024 -0500 Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks (#30175) Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.5 to 1.1.12. - [Release notes](https://github.com/opencontainers/runc/releases) - [Changelog](https://github.com/opencontainers/runc/blob/v1.1.12/CHANGELOG.md) - [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12) --- updated-dependencies: - dependency-name: github.com/opencontainers/runc dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- sdks/go.mod | 4 ++-- sdks/go.sum | 40 ++-- 2 files changed, 8 insertions(+), 36 deletions(-) diff --git a/sdks/go.mod b/sdks/go.mod index de7b3693045..80a017c0917 100644 --- a/sdks/go.mod +++ b/sdks/go.mod @@ -77,6 +77,7 @@ require ( dario.cat/mergo v1.0.0 // indirect github.com/Microsoft/hcsshim v0.11.1 // indirect github.com/containerd/log v0.1.0 // indirect + github.com/frankban/quicktest v1.14.0 // indirect github.com/go-logr/logr v1.3.0 // indirect github.com/go-logr/stdr v1.2.2 // indirect github.com/go-ole/go-ole v1.2.6 // indirect @@ -155,7 +156,6 @@ require ( github.com/klauspost/asmfmt v1.3.2 // indirect github.com/klauspost/compress v1.17.4 // indirect github.com/klauspost/cpuid/v2 v2.2.6 // indirect - github.com/kr/text v0.2.0 // indirect github.com/magiconair/properties v1.8.7 // indirect github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect @@ -166,7 +166,7 @@ require ( github.com/morikuni/aec v1.0.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/image-spec v1.1.0-rc5 // indirect - github.com/opencontainers/runc v1.1.5 // indirect + github.com/opencontainers/runc v1.1.12 // indirect github.com/pierrec/lz4/v4 v4.1.15 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/pkg/xattr v0.4.9 // indirect diff --git a/sdks/go.sum b/sdks/go.sum index 60b711ab059..06b251d29a8 100644 --- a/sdks/go.sum +++ b/sdks/go.sum @@ -138,11 +138,9 @@ github.com/census-instrumentation/opencensus-proto v0.4.1 h1:iKLQ0xPNFxR/2hzXZMr github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw= github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/checkpoint-restore/go-criu/v5 v5.3.0/go.mod h1:E/eQpaFtUKGOOSEBZgmKAcn+zUUwWxqcaKZlF54wK8E= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= -github.com/cilium/ebpf v0.7.0/go.mod h1:/oI2+1shJiTGAMgl6/RgJr36Eo1jzrRcAWbcXO2usCA= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe h1:QQ3GSy+MqSHxm/d8nCtnAiZdYFd45cYZPs8vOOIYKfk= @@ -151,19 +149,15 @@ github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWH github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 h1:/inchEIKaYC1Akx+H+gqO04wryn5h75LSazbRlnya1k= github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/colinmarc/hdfs/v2 v2.1.1/go.mod h1:M3x+k8UKKmxtFu++uAZ0OtDU8jR3jnaZIAc6yK4Ue0c= -github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U= github.com/containerd/containerd v1.7.7 h1:QOC2K4A42RQpcrZyptP6z9EJZnlHfHJUfZrAAHe15q4= github.com/containerd/containerd v1.7.7/go.mod h1:3c4XZv6VeT9qgf9GMTxNTMFxGJrGpI2vz1yk4ye+YY8= github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I= github.com/containerd/log v0.1.0/go.mod h1:
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8e77561ebe5 Updating config from bot 8e77561ebe5 is described below commit 8e77561ebe5ff600d5f6506b1bb2f8efba78b835 Author: github-actions AuthorDate: Thu Feb 1 03:06:13 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index fd4b83c8137..28924af27a9 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -5,7 +5,7 @@ "AnandInguva": 170657132, "chamikaramj": 1706660356104, "damccorm": 1706746787532, -"johnjcasey": 1705059270349, +"johnjcasey": 1706756770922, "jrmccluskey": 1705430101174, "kennknowles": 1705097216618, "lostluck": 1705635383761,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4672b4839d8 Updating config from bot 4672b4839d8 is described below commit 4672b4839d803a92cab9e8c48f10e8505f94671b Author: github-actions AuthorDate: Thu Feb 1 03:06:12 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30168.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30168.json b/scripts/ci/pr-bot/state/pr-state/pr-30168.json new file mode 100644 index 000..b29414341ff --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30168.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"no-matching-label": "johnjcasey" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c34a54fa815 Updating config from bot c34a54fa815 is described below commit c34a54fa8153214744264eb09074c5fa688c26bd Author: github-actions AuthorDate: Thu Feb 1 02:56:40 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30179.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30179.json b/scripts/ci/pr-bot/state/pr-state/pr-30179.json new file mode 100644 index 000..9c2aa5aa212 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30179.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": true, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8f7a710671e Updating config from bot 8f7a710671e is described below commit 8f7a710671e6606d6d1a1bc891232965242cc630 Author: github-actions AuthorDate: Thu Feb 1 00:19:48 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30176.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30176.json b/scripts/ci/pr-bot/state/pr-state/pr-30176.json new file mode 100644 index 000..f6cbd919240 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30176.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"no-matching-label": "damccorm" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7a3959584ab Updating config from bot 7a3959584ab is described below commit 7a3959584ab41c321df9ef8142ad7d65080f7d7a Author: github-actions AuthorDate: Thu Feb 1 00:19:50 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index b4c7447bda5..fd4b83c8137 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -4,7 +4,7 @@ "Abacn": 1706405351155, "AnandInguva": 170657132, "chamikaramj": 1706660356104, -"damccorm": 1704998100377, +"damccorm": 1706746787532, "johnjcasey": 1705059270349, "jrmccluskey": 1705430101174, "kennknowles": 1705097216618,
(beam) branch asf-site updated: Publishing website 2024/01/31 23:40:06 at commit 4f09e62
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new ec494a9c70a Publishing website 2024/01/31 23:40:06 at commit 4f09e62 ec494a9c70a is described below commit ec494a9c70a465fb70b3399db3f51d58f90940a8 Author: runner AuthorDate: Wed Jan 31 23:40:06 2024 + Publishing website 2024/01/31 23:40:06 at commit 4f09e62 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 5695b6b7940..6285044330b 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-31T10:03:38+01:00/categories/blog/2024-01-31T10:03:38+01:00/blog/2024-01-31T10:03:38+01:00/categories/2024-01-31T10:03:38+01:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-31T23:23:50+01:00/categories/blog/2024-01-31T23:23:50+01:00/blog/2024-01-31T23:23:50+01:00/categories/2024-01-31T23:23:50+01:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8aad7bf7cc2 Updating config from bot 8aad7bf7cc2 is described below commit 8aad7bf7cc253cea3a3e82b4b0b329a9e2b2c579 Author: github-actions AuthorDate: Wed Jan 31 23:34:33 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 5227a37f24f..170f7ab0974 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1706679270657, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1706297685702 +"riteshghorse": 1706744071021 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 53efe04b776 Updating config from bot 53efe04b776 is described below commit 53efe04b7762388c9eba16db618e9a4784a5d1e0 Author: github-actions AuthorDate: Wed Jan 31 23:34:32 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30175.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30175.json b/scripts/ci/pr-bot/state/pr-state/pr-30175.json new file mode 100644 index 000..aa3638a74ec --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30175.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"go": "riteshghorse" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) 01/01: Fix script typos for validating docker licenses.
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a commit to branch lostluck-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git commit ec708cda8a4a622e591ab0fc8c9477fec10f5798 Author: Robert Burke AuthorDate: Wed Jan 31 15:16:00 2024 -0800 Fix script typos for validating docker licenses. --- contributor-docs/release-guide.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/contributor-docs/release-guide.md b/contributor-docs/release-guide.md index b3b3213d9c9..559d3c0f9f0 100644 --- a/contributor-docs/release-guide.md +++ b/contributor-docs/release-guide.md @@ -564,11 +564,11 @@ The following should be confirmed: At [https://hub.docker.com/u/apache](https://hub.docker.com/search?q=apache%2Fbeam&type=image), visit each repository and navigate to "tags" tab. Verify images are pushed -with tags: `${RELEASE_VERSION}rc{RC_NUM}` +with tags: `${RELEASE_VERSION}rc${RC_NUM}` Verify that third party licenses are included in Docker. You can do this with a simple script: -RC_TAG=${RELEASE_VERSION}rc{RC_NUM} +RC_TAG=${RELEASE_VERSION}rc${RC_NUM} for pyver in 3.8 3.9 3.10 3.11; do docker run --rm --entrypoint sh \ apache/beam_python${pyver}_sdk:${RC_TAG} \ @@ -577,7 +577,7 @@ Verify that third party licenses are included in Docker. You can do this with a for javaver in 8 11 17; do docker run --rm --entrypoint sh \ - apache/beam_java${pyver}_sdk:${RC_TAG} \ + apache/beam_java${javaver}_sdk:${RC_TAG} \ -c 'ls -al /opt/apache/beam/third_party_licenses/ | wc -l' done
(beam) branch lostluck-patch-2 created (now ec708cda8a4)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch lostluck-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git at ec708cda8a4 Fix script typos for validating docker licenses. This branch includes the following new commits: new ec708cda8a4 Fix script typos for validating docker licenses. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) branch dependabot/go_modules/sdks/github.com/opencontainers/runc-1.1.12 created (now a7ad749b1ff)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/opencontainers/runc-1.1.12 in repository https://gitbox.apache.org/repos/asf/beam.git at a7ad749b1ff Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks No new revisions were added by this update.
(beam) branch fixdeletecontainer updated (4269a1e39c1 -> d9fdb7f255a)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch fixdeletecontainer in repository https://gitbox.apache.org/repos/asf/beam.git omit 4269a1e39c1 Switch public and private add d9fdb7f255a Switch public and private This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (4269a1e39c1) \ N -- N -- N refs/heads/fixdeletecontainer (d9fdb7f255a) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh | 2 -- 1 file changed, 2 deletions(-)
(beam) branch fixdeletecontainer updated (4ce9f0c645d -> 4269a1e39c1)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch fixdeletecontainer in repository https://gitbox.apache.org/repos/asf/beam.git from 4ce9f0c645d Fix stale dataflow prebuilt image cleaner add 4269a1e39c1 Switch public and private No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch master updated: Improve varint encoding throughput with unrolled loop (#29689)
This is an automated email from the ASF dual-hosted git repository. damondouglas pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 4f09e62f520 Improve varint encoding throughput with unrolled loop (#29689) 4f09e62f520 is described below commit 4f09e62f520d8a3bd40f9907e7343814cbd239ae Author: Steven van Rossum AuthorDate: Wed Jan 31 23:23:50 2024 +0100 Improve varint encoding throughput with unrolled loop (#29689) * Improve varint encoding throughput with unrolled loop * Change BlackHole to Blackhole * Add single byte encode tests * Add missing L * Remove public modifier * Remove unused fields --- .../apache/beam/sdk/jmh/util/VarIntBenchmark.java | 303 + .../main/java/org/apache/beam/sdk/util/VarInt.java | 62 - 2 files changed, 358 insertions(+), 7 deletions(-) diff --git a/sdks/java/core/jmh/src/main/java/org/apache/beam/sdk/jmh/util/VarIntBenchmark.java b/sdks/java/core/jmh/src/main/java/org/apache/beam/sdk/jmh/util/VarIntBenchmark.java new file mode 100644 index 000..964928d5055 --- /dev/null +++ b/sdks/java/core/jmh/src/main/java/org/apache/beam/sdk/jmh/util/VarIntBenchmark.java @@ -0,0 +1,303 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.beam.sdk.jmh.util; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.Random; +import org.apache.beam.sdk.util.ByteStringOutputStream; +import org.openjdk.jmh.annotations.Benchmark; +import org.openjdk.jmh.annotations.Level; +import org.openjdk.jmh.annotations.OperationsPerInvocation; +import org.openjdk.jmh.annotations.Scope; +import org.openjdk.jmh.annotations.Setup; +import org.openjdk.jmh.annotations.State; +import org.openjdk.jmh.annotations.TearDown; +import org.openjdk.jmh.infra.Blackhole; + +/** Benchmarks for {@link org.apache.beam.sdk.util.VarInt} and variants. */ +@OperationsPerInvocation(VarIntBenchmark.VALUES_PER_INVOCATION) +public class VarIntBenchmark { + static final int VALUES_PER_INVOCATION = 2048; + private static final Random RNG = new Random(314159); + + /** Output to {@link Blackhole}. Do nothing, assume nothing. */ + @State(Scope.Benchmark) + public static class BlackholeOutput { +OutputStream stream; + +@Setup +public void setup(Blackhole bh) { + stream = + new OutputStream() { +@Override +public void write(int b) { + bh.consume(b); +} + +@Override +public void write(byte[] b) throws IOException { + bh.consume(b); +} + +@Override +public void write(byte[] b, int off, int len) throws IOException { + bh.consume(b); +} + }; +} + } + + /** Output to {@link ByteStringOutputStream}. */ + @State(Scope.Thread) + public static class ByteStringOutput { +final ByteStringOutputStream stream = new ByteStringOutputStream(); + +// Unfortunately, this needs to be cleaned up after use to avoid OOMs. +// It's not generally recommended to use Level.Invocation, but there's no way around it. +@TearDown(Level.Invocation) +public void tearDown(Blackhole bh) { + bh.consume(stream.toByteStringAndReset()); +} + } + + /** Input from randomly generated bytes. */ + @State(Scope.Benchmark) + public static class Bytes { +long[] values = new long[VALUES_PER_INVOCATION]; + +@Setup +public void setup() { + values = new long[VALUES_PER_INVOCATION]; + byte[] bytes = new byte[VALUES_PER_INVOCATION]; + RNG.nextBytes(bytes); + + for (int i = 0; i < VALUES_PER_INVOCATION; i++) { +values[i] = (long) (bytes[i] & 0x7F); + } +} + } + + /** Input from randomly generated longs. */ + @State(Scope.Benchmark) + public static class Longs { +long[] values = new long[VALUES_PER_INVOCATION]; + +@Setup +public void setup() { + values = new long[VALUES_PER_INVOCATION]; + + for (int i = 0; i < VALUES_PER_INVOCATION; i++) { +// This gaussian random is used to
(beam) branch master updated: Periodically send perWorkerMetrics on the WorkerMessages API (#30135)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new fb0762352c2 Periodically send perWorkerMetrics on the WorkerMessages API (#30135) fb0762352c2 is described below commit fb0762352c20ac50f182facdd82d80f4ecfaa052 Author: JayajP AuthorDate: Wed Jan 31 14:21:12 2024 -0800 Periodically send perWorkerMetrics on the WorkerMessages API (#30135) --- .../dataflow/worker/DataflowWorkUnitClient.java| 24 +++-- .../dataflow/worker/StreamingDataflowWorker.java | 40 +++--- .../worker/StreamingStepMetricsContainer.java | 8 - .../runners/dataflow/worker/WorkUnitClient.java| 17 +++-- .../dataflow/worker/streaming/StageInfo.java | 9 + .../worker/DataflowWorkUnitClientTest.java | 37 ++-- 6 files changed, 122 insertions(+), 13 deletions(-) diff --git a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/DataflowWorkUnitClient.java b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/DataflowWorkUnitClient.java index bb39e3bd9af..f3caa8d0f3a 100644 --- a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/DataflowWorkUnitClient.java +++ b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/DataflowWorkUnitClient.java @@ -29,6 +29,7 @@ import static org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Mo import com.google.api.services.dataflow.Dataflow; import com.google.api.services.dataflow.model.LeaseWorkItemRequest; import com.google.api.services.dataflow.model.LeaseWorkItemResponse; +import com.google.api.services.dataflow.model.PerWorkerMetrics; import com.google.api.services.dataflow.model.ReportWorkItemStatusRequest; import com.google.api.services.dataflow.model.ReportWorkItemStatusResponse; import com.google.api.services.dataflow.model.SendWorkerMessagesRequest; @@ -292,13 +293,30 @@ class DataflowWorkUnitClient implements WorkUnitClient { return msg; } - /** Reports the autoscaling signals to dataflow */ @Override - public void reportWorkerMessage(WorkerMessage msg) throws IOException { + public WorkerMessage createWorkerMessageFromPerWorkerMetrics(PerWorkerMetrics report) { +DateTime endTime = DateTime.now(); +logger.debug("Reporting WorkMessageResponse"); +Map labels = +ImmutableMap.of("JOB_ID", options.getJobId(), "WORKER_ID", options.getWorkerId()); +WorkerMessage msg = +new WorkerMessage() +.setTime(toCloudTime(endTime)) +.setPerWorkerMetrics(report) +.setLabels(labels); +return msg; + } + + /** + * Reports the worker messages to dataflow. We currently report autoscaling signals and + * perworkermetrics with this path. + */ + @Override + public void reportWorkerMessage(List messages) throws IOException { SendWorkerMessagesRequest request = new SendWorkerMessagesRequest() .setLocation(options.getRegion()) -.setWorkerMessages(Collections.singletonList(msg)); +.setWorkerMessages(messages); SendWorkerMessagesResponse result = dataflow .projects() diff --git a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java index d915b77995d..f37504692d3 100644 --- a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java +++ b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java @@ -23,12 +23,15 @@ import static org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Pr import com.google.api.services.dataflow.model.CounterUpdate; import com.google.api.services.dataflow.model.MapTask; +import com.google.api.services.dataflow.model.PerStepNamespaceMetrics; +import com.google.api.services.dataflow.model.PerWorkerMetrics; import com.google.api.services.dataflow.model.Status; import com.google.api.services.dataflow.model.StreamingComputationConfig; import com.google.api.services.dataflow.model.StreamingConfigTask; import com.google.api.services.dataflow.model.StreamingScalingReport; import com.google.api.services.dataflow.model.WorkItem; import com.google.api.services.dataflow.model.WorkItemStatus; +import com.google.api.services.dataflow.model.WorkerMessage; import edu.umd.cs.findbugs.annotations.SuppressFBWarnings; import java.io.File; import java.io.IOException; @@ -117,6 +120,7 @@ import org.apache.beam.sdk.fn
(beam) branch master updated: Don't use '-' as a delimiter in BigQuerySinkMetrics (#30118)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 30917ff7b3e Don't use '-' as a delimiter in BigQuerySinkMetrics (#30118) 30917ff7b3e is described below commit 30917ff7b3e1009578fddd1b4d0b6b1db4776d13 Author: JayajP AuthorDate: Wed Jan 31 14:21:00 2024 -0800 Don't use '-' as a delimiter in BigQuerySinkMetrics (#30118) * Don't use '-' as a delimiter in BigQuerySinkMetrics * Use Optional::{filter|map} for returning simple optional --- .../MetricsToPerStepNamespaceMetricsConverter.java | 27 +- ...ricsToPerStepNamespaceMetricsConverterTest.java | 16 +++--- .../worker/StreamingStepMetricsContainerTest.java | 4 +- .../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 37 -- .../io/gcp/bigquery/BigQuerySinkMetricsTest.java | 59 -- 5 files changed, 75 insertions(+), 68 deletions(-) diff --git a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java index f3f31ccfd36..7b72d650778 100644 --- a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java +++ b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java @@ -52,17 +52,14 @@ public class MetricsToPerStepNamespaceMetricsConverter { return Optional.empty(); } -BigQuerySinkMetrics.ParsedMetricName labeledName = -BigQuerySinkMetrics.parseMetricName(metricName.getName()); -if (labeledName == null || labeledName.getBaseName().isEmpty()) { - return Optional.empty(); -} - -return Optional.of( -new MetricValue() -.setMetric(labeledName.getBaseName()) -.setMetricLabels(labeledName.getMetricLabels()) -.setValueInt64(value)); +return BigQuerySinkMetrics.parseMetricName(metricName.getName()) +.filter(labeledName -> !labeledName.getBaseName().isEmpty()) +.map( +labeledName -> +new MetricValue() +.setMetric(labeledName.getBaseName()) +.setMetricLabels(labeledName.getMetricLabels()) +.setValueInt64(value)); } /** @@ -78,9 +75,9 @@ public class MetricsToPerStepNamespaceMetricsConverter { return Optional.empty(); } -BigQuerySinkMetrics.ParsedMetricName labeledName = +Optional labeledName = BigQuerySinkMetrics.parseMetricName(metricName.getName()); -if (labeledName == null || labeledName.getBaseName().isEmpty()) { +if (!labeledName.isPresent() || labeledName.get().getBaseName().isEmpty()) { return Optional.empty(); } @@ -133,8 +130,8 @@ public class MetricsToPerStepNamespaceMetricsConverter { return Optional.of( new MetricValue() -.setMetric(labeledName.getBaseName()) -.setMetricLabels(labeledName.getMetricLabels()) +.setMetric(labeledName.get().getBaseName()) +.setMetricLabels(labeledName.get().getMetricLabels()) .setValueHistogram(histogramValue)); } diff --git a/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverterTest.java b/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverterTest.java index a37bf643b7e..0a4cd06cf8c 100644 --- a/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverterTest.java +++ b/runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverterTest.java @@ -81,7 +81,7 @@ public class MetricsToPerStepNamespaceMetricsConverterTest { Map counters = new HashMap(); MetricName bigQueryMetric1 = MetricName.named("BigQuerySink", "metric1"); MetricName bigQueryMetric2 = -MetricName.named("BigQuerySink", "metric2-label1:val1;label2:val2;"); +MetricName.named("BigQuerySink", "metric2*label1:val1;label2:val2;"); MetricName bigQueryMetric3 = MetricName.named("BigQuerySink", "zeroValue"); counters.put(bigQueryMetric1, 5L); @@ -112,14 +112,14 @@ public class MetricsToPerStepNamespaceMetricsConverterTest { @Test public void testConvert_skipInvalidMetricNames() { Map counters = new HashMap<>(); -MetricName bigQueryMetric1 = MetricName.named("BigQuerySink", "invalid-metric-name1"); -count
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 90b6a540d1b Updating config from bot 90b6a540d1b is described below commit 90b6a540d1bea235cac75bb2181a567ef494e999 Author: github-actions AuthorDate: Wed Jan 31 22:05:38 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json b/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json index b70ee1ddee3..6bee694e16a 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json @@ -1,7 +1,7 @@ { "label": "bigtable", "dateOfLastReviewAssignment": { -"igorbernstein2": 1706573922958, +"igorbernstein2": 1706738733803, "mutianf": 1704742532925 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 0d8667bb50f Updating config from bot 0d8667bb50f is described below commit 0d8667bb50fbd9b10b0e01e1b9ae4905ae3badae Author: github-actions AuthorDate: Wed Jan 31 22:05:36 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index a4eb5580da8..2c54e6e059f 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -7,7 +7,7 @@ "kileys": 1674428678843, "apilloud": 1678822446183, "Abacn": 1706703224505, -"bvolpato": 1706034929353, +"bvolpato": 1706738733802, "m-trieu": 1706573922958, "damondouglas": 1706703218116 }
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new cae9cd43eaa Updating config from bot cae9cd43eaa is described below commit cae9cd43eaa2515150ba645aee780647f49caf2e Author: github-actions AuthorDate: Wed Jan 31 22:05:37 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index 0b2a0219fad..17bad785ddf 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -1,7 +1,7 @@ { "label": "io", "dateOfLastReviewAssignment": { -"chamikaramj": 1706149095380, +"chamikaramj": 1706738733803, "johnjcasey": 1706573922958, "pabloem": 1691787951165, "Abacn": 1706288724292,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 81c06339a9c Updating config from bot 81c06339a9c is described below commit 81c06339a9cf87b192258c7a949f53ab75a1ebe1 Author: github-actions AuthorDate: Wed Jan 31 22:05:35 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30172.json | 12 1 file changed, 12 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30172.json b/scripts/ci/pr-bot/state/pr-state/pr-30172.json new file mode 100644 index 000..049ee37801c --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30172.json @@ -0,0 +1,12 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"java": "bvolpato", +"io": "chamikaramj", +"bigtable": "igorbernstein2" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch master updated: Fix git trusted directory errors in website build. (#30171)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new ddb93272ab6 Fix git trusted directory errors in website build. (#30171) ddb93272ab6 is described below commit ddb93272ab6227b468f1731d8482e14666a684fe Author: Robert Bradshaw AuthorDate: Wed Jan 31 14:00:53 2024 -0800 Fix git trusted directory errors in website build. (#30171) --- website/Dockerfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/Dockerfile b/website/Dockerfile index 0cb992422ea..90d4202728d 100644 --- a/website/Dockerfile +++ b/website/Dockerfile @@ -74,3 +74,5 @@ RUN HUGOHOME="$(mktemp -d)" \ && chmod +x /usr/local/bin/hugo \ && rm -r "${HUGOHOME}" +# Prevent git errors of the form "fatal: detected dubious ownership in repository" +RUN git config --file=/.gitconfig --add safe.directory /opt && chmod a+r /.gitconfig
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7c63bdce3bd Updating config from bot 7c63bdce3bd is described below commit 7c63bdce3bd1348d74ca6e0ddd8c07b970c6f064 Author: github-actions AuthorDate: Wed Jan 31 21:05:50 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30171.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30171.json b/scripts/ci/pr-bot/state/pr-state/pr-30171.json new file mode 100644 index 000..b42821bc7d4 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30171.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"website": "liferoad" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new a087b9bc6ad Updating config from bot a087b9bc6ad is described below commit a087b9bc6ad1c783a51217c4513cf3785870791a Author: github-actions AuthorDate: Wed Jan 31 21:05:52 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-website.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-website.json b/scripts/ci/pr-bot/state/reviewers-for-label-website.json index 7dfae132ee9..a3fab4cecc7 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-website.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-website.json @@ -4,7 +4,7 @@ "rszper": 1706314685072, "melap": 1706549680321, "damccorm": 1706573918703, -"liferoad": 1705415686820, +"liferoad": 1706735149793, "kennknowles": 1705430107048 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new e7e5e2a9125 Updating config from bot e7e5e2a9125 is described below commit e7e5e2a91253991582d35cc1bca73927fb402b23 Author: github-actions AuthorDate: Wed Jan 31 19:53:35 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30170.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30170.json b/scripts/ci/pr-bot/state/pr-state/pr-30170.json new file mode 100644 index 000..9c2aa5aa212 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30170.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": true, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch master updated: fix: fix error handling in segment reader and add an integration test (#30147)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new a221f98a5f4 fix: fix error handling in segment reader and add an integration test (#30147) a221f98a5f4 is described below commit a221f98a5f46be985afcb98a65fcec3b46b81f92 Author: Mattie Fu AuthorDate: Wed Jan 31 14:43:34 2024 -0500 fix: fix error handling in segment reader and add an integration test (#30147) * fix: fix error handling in segment reader and add an integration test * also check for root cause * revert getRootCause change * fix format * add a log line * remove logging * change the log level to debug, update comment --- .../sdk/io/gcp/bigtable/BigtableServiceImpl.java | 20 - .../beam/sdk/io/gcp/bigtable/BigtableReadIT.java | 93 -- 2 files changed, 106 insertions(+), 7 deletions(-) diff --git a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceImpl.java b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceImpl.java index 7537a16a188..d6208be1bf9 100644 --- a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceImpl.java +++ b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableServiceImpl.java @@ -387,6 +387,11 @@ class BigtableServiceImpl implements BigtableService { currentByteSize += response.getSerializedSize(); rows.add(response); if (currentByteSize > maxSegmentByteSize) { +LOG.debug( +"Reached maxSegmentByteSize, cancelling the stream. currentByteSize is {}, maxSegmentByteSize is {}, read rows {}", +currentByteSize, +maxSegmentByteSize, +rows.size()); byteLimitReached = true; controller.cancel(); return; @@ -395,14 +400,25 @@ class BigtableServiceImpl implements BigtableService { @Override public void onError(Throwable t) { - future.setException(t); + if (byteLimitReached) { +// When the byte limit is reached we cancel the stream in onResponse. +// In this case we don't want to fail the request with cancellation +// exception. Instead, we construct the next request. +onComplete(); + } else { +future.setException(t); + } } @Override public void onComplete() { ReadRowsRequest nextNextRequest = null; - // When requested rows < limit, the current request will be the last + // Only schedule the next segment fetch when there's a possibility of more + // data to read. We know there might be more data when the current segment + // ended with the artificial byte limit or the row limit. + // If the RPC ended without hitting the byte limit or row limit, we know + // there's no more data to read and nextNextRequest would be null. if (byteLimitReached || rows.size() == nextRequest.getRowsLimit()) { nextNextRequest = truncateRequest(nextRequest, rows.get(rows.size() - 1).getKey()); diff --git a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java index 704a895992c..bc88858ebc3 100644 --- a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java +++ b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableReadIT.java @@ -17,7 +17,16 @@ */ package org.apache.beam.sdk.io.gcp.bigtable; +import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient; +import com.google.cloud.bigtable.admin.v2.BigtableTableAdminSettings; +import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest; import com.google.cloud.bigtable.config.BigtableOptions; +import com.google.cloud.bigtable.data.v2.BigtableDataClient; +import com.google.cloud.bigtable.data.v2.BigtableDataSettings; +import com.google.cloud.bigtable.data.v2.models.RowMutation; +import java.io.IOException; +import java.util.Date; +import org.apache.beam.repackaged.core.org.apache.commons.lang3.StringUtils; import org.apache.beam.sdk.Pipeline; import org.apache.beam.sdk.extensions.gcp
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8d628632954 Updating config from bot 8d628632954 is described below commit 8d62863295438edaaad130c0400553c0cb37da5a Author: github-actions AuthorDate: Wed Jan 31 19:34:14 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index a886eeb9490..164b6c86eda 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -10,7 +10,7 @@ "y1chi": 1667002607045, "damccorm": 1706565938559, "jrmccluskey": 1706648779876, -"riteshghorse": 1706200695261, +"riteshghorse": 1706729651902, "liferoad": 1706288724292 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 58f6b1d0279 Updating config from bot 58f6b1d0279 is described below commit 58f6b1d0279f47307c0342f3e67040c97dc42cfb Author: github-actions AuthorDate: Wed Jan 31 19:34:13 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30164.json | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30164.json b/scripts/ci/pr-bot/state/pr-state/pr-30164.json index 242a48d7d3b..c43704aa814 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-30164.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-30164.json @@ -1,7 +1,9 @@ { - "commentedAboutFailingChecks": true, - "reviewersAssignedForLabels": {}, - "nextAction": "Author", + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"python": "riteshghorse" + }, + "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], "committerAssigned": false
(beam) branch master updated (81513f10d60 -> 89d1c06e1ea)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 81513f10d60 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#30162) add 89d1c06e1ea Fix Fhir IO Search IT (#30158) No new revisions were added by this update. Summary of changes: ...stCommit_Python.json => beam_PostCommit_Java_DataflowV2.json} | 0 .../org/apache/beam/sdk/io/gcp/healthcare/FhirIOSearchIT.java| 9 +++-- 2 files changed, 3 insertions(+), 6 deletions(-) copy .github/trigger_files/{beam_PostCommit_Python.json => beam_PostCommit_Java_DataflowV2.json} (100%)
(beam) branch hf_inference_api_ updated (fb3e67f5211 -> 13f018e890a)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch hf_inference_api_ in repository https://gitbox.apache.org/repos/asf/beam.git from fb3e67f5211 get hf_token during runtime from the container env add 13f018e890a Pass env variable to the right environment No new revisions were added by this update. Summary of changes: sdks/python/tox.ini | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
(beam) branch damccorm-patch-1 updated (edae3345d0d -> b496238079d)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch damccorm-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git from edae3345d0d Update stale_bq_datasets_cleaner.sh add b496238079d Typo No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_bq_datasets_cleaner.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch damccorm-patch-1 updated (f2e76be0df4 -> edae3345d0d)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch damccorm-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git from f2e76be0df4 Debug add edae3345d0d Update stale_bq_datasets_cleaner.sh No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_bq_datasets_cleaner.sh | 33 -- 1 file changed, 20 insertions(+), 13 deletions(-)
(beam) branch fixdeletecontainer updated (ea8a3b6085d -> 4ce9f0c645d)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch fixdeletecontainer in repository https://gitbox.apache.org/repos/asf/beam.git omit ea8a3b6085d Fix stale dataflow prebuilt image cleaner add 4ce9f0c645d Fix stale dataflow prebuilt image cleaner This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (ea8a3b6085d) \ N -- N -- N refs/heads/fixdeletecontainer (4ce9f0c645d) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(beam) 01/01: Fix stale dataflow prebuilt image cleaner
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch fixdeletecontainer in repository https://gitbox.apache.org/repos/asf/beam.git commit ea8a3b6085d7c3fdb3ef88977c31493fc6aead21 Author: Yi Hu AuthorDate: Wed Jan 31 13:40:33 2024 -0500 Fix stale dataflow prebuilt image cleaner --- .../tools/stale_dataflow_prebuilt_image_cleaner.sh | 41 ++ 1 file changed, 26 insertions(+), 15 deletions(-) diff --git a/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh b/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh index 804f6e2f39d..fdd812328d2 100755 --- a/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh +++ b/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh @@ -27,9 +27,11 @@ PRIVATE_REPOSITORIES=(java-postcommit-it python-postcommit-it jenkins github-act # set as the same as 6-week release period if [[ $OSTYPE == "linux-gnu"* ]]; then # date command usage depending on OS - DELETE_BEFORE_DAY=$(date --iso-8601=s -d '6 weeks ago') + DELETE_BEFORE_PUBLIC=$(date --iso-8601=s -d '6 weeks ago') + DELETE_BEFORE_PRIVATE=$(date --iso-8601=s -d '1 week ago') elif [[ $OSTYPE == "darwin"* ]]; then - DELETE_BEFORE_DAY=$(date -j -v-6w '+%Y-%m-%dT%H:%M:%S') + DELETE_BEFORE_PUBLIC=$(date -j -v-6w '+%Y-%m-%dT%H:%M:%S') + DELETE_BEFORE_PRIVATE=$(date -j -v-1w '+%Y-%m-%dT%H:%M:%S') fi REPOSITORIES=("${PUBLIC_REPOSITORIES[@]/#/gcr.io/apache-beam-testing/}" "${PRIVATE_REPOSITORIES[@]/#/us.gcr.io/apache-beam-testing/}") @@ -54,7 +56,7 @@ while [ -n "$REPOSITORIES" ]; do REPOSITORIES=("${PENDING_REPOSITORIES[@]}") done -STALE_IMAGES="" +HAS_STALE_IMAGES="" FAILED_INSPECT="" for image_name in ${IMAGE_NAMES[@]}; do @@ -64,22 +66,31 @@ for image_name in ${IMAGE_NAMES[@]}; do LATEST_IN_TIME=$(gcloud container images list-tags \ ${image_name} --sort-by="~TIMESTAMP" --filter="NOT tags:latest " --format="get(digest)" --limit=1) if [ -n "$LATEST_IN_TIME" ]; then +# decide timestamp cutoff +if [[ $image_name =~ 'us.gcr.io' ]]; then + DELETE_BEFORE_DAY=$DELETE_BEFORE_PRIVATE +else + DELETE_BEFORE_DAY=$DELETE_BEFORE_PUBLIC +fi # list containers of the image name echo "Command" gcloud container images list-tags \ ${image_name} \ --sort-by=TIMESTAMP --filter="NOT tags:latest AND timestamp.datetime < $DELETE_BEFORE_DAY" \ ---format="get(digest)" -STALE_IMAGES_CURRENT=$(gcloud container images list-tags \ +--format="get(digest,timestamp.year)" +STALE_IMAGES=$(gcloud container images list-tags \ ${image_name} \ --sort-by=TIMESTAMP --filter="NOT tags:latest AND timestamp.datetime < $DELETE_BEFORE_DAY" \ - --format="get(digest)") -STALE_IMAGES+=$STALE_IMAGES_CURRENT -for current in ${STALE_IMAGES_CURRENT[@]}; do + --format="get(digest,timestamp.year)") + +STALE_IMAGES_CURRENT=($STALE_IMAGES) +for (( i_stale_images_current=0; i_stale_images_current<${#STALE_IMAGES_CURRENT[@]} ; i_stale_images_current+=2 )) ; do + current=${STALE_IMAGES_CURRENT[i_stale_images_current]} + currentyear=${STALE_IMAGES_CURRENT[i_stale_images_current+1]} # do not delete the one with latest label and the newest image without latest label # this make sure we leave at least one container under each image name, either labelled "latest" or not if [ "$LATEST_IN_TIME" != "$current" ]; then -if [[ $image_name =~ 'beamgrafana' || $image_name =~ 'beammetricssyncjenkins' || $image_name =~ 'beammetricssyncgithub' ]]; then - # Skip docker manifest inspect for known single arch images, workaround permission issue & saving API call +if [[ $currentyear > 1970 ]]; then + # Skip docker manifest inspect for those not in epoch to save API call SHOULD_DELETE=0 else # Check to see if this image is built on top of earlier images. This is the case for multiarch images, @@ -96,8 +107,8 @@ for image_name in ${IMAGE_NAMES[@]}; do DIGEST=$(echo $MANIFEST | jq -r '.manifests[0].digest') if [ "$DIGEST" != "null" ]; then SHOULD_DELETE=1 -for i in ${STALE_IMAGES_CURRENT[@]}; do - if [ "$i" = "$DIGEST" ]; then +for (( j_stale_images_current=0; j_stale_images_current<${#STALE_IMAGES_CURRENT[@]} ; j_stale_images_current+=2 )) ; do + if [ "${STALE_IMAGES_CURRENT[j_stale_images_current]}" = "$DIGEST" ]; then SHOULD_DELETE=0 break fi @@ -105,8 +116,8 @@ for image_name in ${IMAGE_NAMES[@]}; do fi fi -if [ $SHOULD_DELETE = 0 ] -then +if [ $SHOULD_DELETE = 0 ]; then + HAS_STALE_IMAGES="true" echo "Deleting image. Command: gcloud container images delete ${image_name}@"${current}" --force-delete-tags -q" gcloud container images delete ${image_name}@"${cu
(beam) branch fixdeletecontainer created (now ea8a3b6085d)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch fixdeletecontainer in repository https://gitbox.apache.org/repos/asf/beam.git at ea8a3b6085d Fix stale dataflow prebuilt image cleaner This branch includes the following new commits: new ea8a3b6085d Fix stale dataflow prebuilt image cleaner The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15 updated (6d868facb2c -> d8f0088e9d7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15 in repository https://gitbox.apache.org/repos/asf/beam.git discard 6d868facb2c Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks add 93073e74e11 Decommission all remaining jenkins jobs (#30113) add 772cf0a5c01 Don't run DicomIO Integration Tests on GCP DF Runner (#30103) add 3ae851876e2 Fix failing python BQ test (#30099) add 6b6fc98ec93 Don't depend on the go 1.21 standard lib yet (#30120) add 445151694fe Check the issue number type (#30127) add 3fd7bf29dcd Bump Dataflow dev container version (#30124) add 1ab504c34a6 Update update_python_dependencies.yml with all python versions (#30126) add 4981c0c3abc Upgrade numpy dependency (#29645) add 1d4413d8e95 Update project root documentations reflecting Jenkins shutdown (#30130) add e0e20a10e99 [#28142][Go SDK] Evict closed readers from the cache. (#30119) add 8485ab40e12 Allow JSON type in TableSchema for BigQuery FILE_LOAD (#29923) add 3172736aaa3 Update Python Dependencies (#30140) add dde81384ef2 Fix clean up stale container workflow (#30132) add c9135e00041 Add info on getting started contributing (#30143) add b72eacfba37 make getProcessingDistributionsForWorkId threadsafe (#29979) add c1c81b36187 Respect gcloud active configuration, if any, rather than always using "default." add ee4f8cb33f9 Merge pull request #30134 Respect gcloud active configuration. add 847a8f4486e [YAML] Improve docs on create. add 8521d2768e0 [YAML] Better document naming parameter. add ba29583d46c [YAML] Move pipeline running instructions to the top. add e70a28acbd4 [YAML] Link to additonal resources. add b85e1ebaea3 Some clarification on the mapping docs. add 09aacf1003c Another note useful for trying things out. add 799c1041473 Avoid SQL keywords for SQL table names. add afd8b79e8e0 Clarification on aggregating transform based on user feedback. add 88bf2fc94fe Update sdks/python/apache_beam/yaml/yaml_provider.py add b835111d66f Make note about requiring Graphviz for rendering pipelines. add 3970b618371 Merge branch 'master' into yaml-create-doc add b5dc54da26f Also mention JavaScript. add e199e42a341 Fix whitespace in create suggestion. add ce28c33940c Merge pull request #30007 [YAML] Several improvements to the documentation. add f9029616a91 Update environments.md (#30149) add 097e6e27bfe Bump actions/cache from 3 to 4 (#30033) add f9c9788815e Update 02_io_pubsub.md (#29986) add 5a9a27cbafa Volatile heartbeat (#30148) add b9fd39cb16b Fix Dataproc cleanup race condition (#30154) add 22fefebacab BigQuery: Decouple clustering from time partitioning when writing (#30094) add 415b1658f50 Merge pull request #30137: Fix a misspelling in zetasql overview page add 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) add 4c5ee349f9b [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 36cab4ffe76 Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 41dee464db4 Make ReadFromBigQueryRequest id more randomized (#30156) add 81513f10d60 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#30162) add d8f0088e9d7 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (6d868facb2c) \ N -- N -- N refs/heads/dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15 (d8f0088e9d7) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: ...p_Dataflow.json => beam_PostCommit_Python.json} | 0 .github/workflows/README.md| 1 + .github/workflows/beam_CancelStaleDataflowJobs.yml | 2 + .../workflows/beam_CleanUpDataprocResources.yml| 2 +- .github/workflows/beam_CleanUpGCPResources.yml | 2 + .../workflows/beam_CleanUpPrebuiltSDKImages.yml| 8 +- .../workflows/beam_PostCommit_Java_DataflowV2.yml | 4 +- .github/workflows/playground_frontend_test.yml | 2 +- .github/workflows/tour_of_beam_frontend_test.yml | 2 +- .github/workflows/update_python_dependencies
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1 updated (609258d3188 -> 19d70b4bd03)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1 in repository https://gitbox.apache.org/repos/asf/beam.git omit 609258d3188 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks add 93073e74e11 Decommission all remaining jenkins jobs (#30113) add 772cf0a5c01 Don't run DicomIO Integration Tests on GCP DF Runner (#30103) add 3ae851876e2 Fix failing python BQ test (#30099) add 6b6fc98ec93 Don't depend on the go 1.21 standard lib yet (#30120) add 445151694fe Check the issue number type (#30127) add 3fd7bf29dcd Bump Dataflow dev container version (#30124) add 1ab504c34a6 Update update_python_dependencies.yml with all python versions (#30126) add 4981c0c3abc Upgrade numpy dependency (#29645) add 1d4413d8e95 Update project root documentations reflecting Jenkins shutdown (#30130) add e0e20a10e99 [#28142][Go SDK] Evict closed readers from the cache. (#30119) add 8485ab40e12 Allow JSON type in TableSchema for BigQuery FILE_LOAD (#29923) add 3172736aaa3 Update Python Dependencies (#30140) add dde81384ef2 Fix clean up stale container workflow (#30132) add c9135e00041 Add info on getting started contributing (#30143) add b72eacfba37 make getProcessingDistributionsForWorkId threadsafe (#29979) add c1c81b36187 Respect gcloud active configuration, if any, rather than always using "default." add ee4f8cb33f9 Merge pull request #30134 Respect gcloud active configuration. add 847a8f4486e [YAML] Improve docs on create. add 8521d2768e0 [YAML] Better document naming parameter. add ba29583d46c [YAML] Move pipeline running instructions to the top. add e70a28acbd4 [YAML] Link to additonal resources. add b85e1ebaea3 Some clarification on the mapping docs. add 09aacf1003c Another note useful for trying things out. add 799c1041473 Avoid SQL keywords for SQL table names. add afd8b79e8e0 Clarification on aggregating transform based on user feedback. add 88bf2fc94fe Update sdks/python/apache_beam/yaml/yaml_provider.py add b835111d66f Make note about requiring Graphviz for rendering pipelines. add 3970b618371 Merge branch 'master' into yaml-create-doc add b5dc54da26f Also mention JavaScript. add e199e42a341 Fix whitespace in create suggestion. add ce28c33940c Merge pull request #30007 [YAML] Several improvements to the documentation. add f9029616a91 Update environments.md (#30149) add 097e6e27bfe Bump actions/cache from 3 to 4 (#30033) add f9c9788815e Update 02_io_pubsub.md (#29986) add 5a9a27cbafa Volatile heartbeat (#30148) add b9fd39cb16b Fix Dataproc cleanup race condition (#30154) add 22fefebacab BigQuery: Decouple clustering from time partitioning when writing (#30094) add 415b1658f50 Merge pull request #30137: Fix a misspelling in zetasql overview page add 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) add 4c5ee349f9b [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 36cab4ffe76 Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult add 41dee464db4 Make ReadFromBigQueryRequest id more randomized (#30156) add 81513f10d60 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#30162) add 19d70b4bd03 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (609258d3188) \ N -- N -- N refs/heads/dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1 (19d70b4bd03) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: ...p_Dataflow.json => beam_PostCommit_Python.json} | 0 .github/workflows/README.md| 1 + .github/workflows/beam_CancelStaleDataflowJobs.yml | 2 + .../workflows/beam_CleanUpDataprocResources.yml| 2 +- .github/workflows/beam_CleanUpGCPResources.yml | 2 + .../workflows/beam_CleanUpPrebuiltSDKImages.yml| 8 +- .../workflows/beam_PostCommit_Java_DataflowV2.yml | 4 +- .github/workflows/playground_frontend_test.yml | 2 +- .github/workflows/tour_of_beam_frontend_test.yml | 2 +- .github/workflows/update_python_dependencies.yml | 15 ++- .test-infra/datapr
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.16 deleted (was 2b4d33c9ee7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.16 in repository https://gitbox.apache.org/repos/asf/beam.git was 2b4d33c9ee7 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated (41dee464db4 -> 81513f10d60)
This is an automated email from the ASF dual-hosted git repository. jrmccluskey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 41dee464db4 Make ReadFromBigQueryRequest id more randomized (#30156) add 81513f10d60 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#30162) No new revisions were added by this update. Summary of changes: sdks/go.mod | 18 +- sdks/go.sum | 36 ++-- 2 files changed, 27 insertions(+), 27 deletions(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c823023ecff Updating config from bot c823023ecff is described below commit c823023ecffb62d659a61548ad1d8751f88a3b6b Author: github-actions AuthorDate: Wed Jan 31 18:34:33 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30147.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30147.json b/scripts/ci/pr-bot/state/pr-state/pr-30147.json index c094352d894..8cf06085651 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-30147.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-30147.json @@ -8,5 +8,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch damccorm-patch-2 deleted (was 955ccecaf96)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch damccorm-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git was 955ccecaf96 Update action.yml This change permanently discards the following revisions: discard 955ccecaf96 Update action.yml
(beam) branch users/damccorm/hfToken deleted (was 46de8b55a5e)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/hfToken in repository https://gitbox.apache.org/repos/asf/beam.git was 46de8b55a5e Check if secret is set This change permanently discards the following revisions: discard 46de8b55a5e Check if secret is set
(beam) 01/01: Update action.yml
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch damccorm-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git commit 955ccecaf96f10776736a5c6a2dd30e1689d4f84 Author: Danny McCormick AuthorDate: Wed Jan 31 13:11:29 2024 -0500 Update action.yml --- .github/actions/gradle-command-self-hosted-action/action.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/actions/gradle-command-self-hosted-action/action.yml b/.github/actions/gradle-command-self-hosted-action/action.yml index e2fd768220a..07813afe614 100644 --- a/.github/actions/gradle-command-self-hosted-action/action.yml +++ b/.github/actions/gradle-command-self-hosted-action/action.yml @@ -35,6 +35,7 @@ runs: - name: Run Gradle Command shell: bash run: | +printenv # This step is a workaround to avoid a decryption issue of Beam's # net.linguica.gradle.maven.settings plugin and github's provided maven # settings.xml file
(beam) branch damccorm-patch-2 created (now 955ccecaf96)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch damccorm-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git at 955ccecaf96 Update action.yml This branch includes the following new commits: new 955ccecaf96 Update action.yml The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) branch master updated: Make ReadFromBigQueryRequest id more randomized (#30156)
This is an automated email from the ASF dual-hosted git repository. ahmedabualsaud pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 41dee464db4 Make ReadFromBigQueryRequest id more randomized (#30156) 41dee464db4 is described below commit 41dee464db458fa72eeab7ddc902b242ebc894eb Author: Ahmed Abualsaud <65791736+ahmedab...@users.noreply.github.com> AuthorDate: Wed Jan 31 13:07:02 2024 -0500 Make ReadFromBigQueryRequest id more randomized (#30156) * make ReadFromBigQueryRequest id more randomized --- sdks/python/apache_beam/io/gcp/bigquery.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sdks/python/apache_beam/io/gcp/bigquery.py b/sdks/python/apache_beam/io/gcp/bigquery.py index bba8b8a4af7..7648ab4064d 100644 --- a/sdks/python/apache_beam/io/gcp/bigquery.py +++ b/sdks/python/apache_beam/io/gcp/bigquery.py @@ -361,6 +361,7 @@ import itertools import json import logging import random +import secrets import time import uuid import warnings @@ -2925,8 +2926,9 @@ class ReadFromBigQueryRequest: self.table = table self.validate() -# We use this internal object ID to generate BigQuery export directories. -self.obj_id = random.randint(0, 10) +# We use this internal object ID to generate BigQuery export directories +# and to create BigQuery job names +self.obj_id = '%d_%s' % (int(time.time()), secrets.token_hex(3)) def validate(self): if self.table is not None and self.query is not None:
(beam) branch hf_inference_api_ updated (90708261cb4 -> fb3e67f5211)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch hf_inference_api_ in repository https://gitbox.apache.org/repos/asf/beam.git from 90708261cb4 Address comments add fb3e67f5211 get hf_token during runtime from the container env No new revisions were added by this update. Summary of changes: .../ml/transforms/embeddings/huggingface.py | 19 +-- 1 file changed, 13 insertions(+), 6 deletions(-)
(beam) branch users/damccorm/hfToken created (now 46de8b55a5e)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/hfToken in repository https://gitbox.apache.org/repos/asf/beam.git at 46de8b55a5e Check if secret is set This branch includes the following new commits: new 46de8b55a5e Check if secret is set The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Check if secret is set
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch users/damccorm/hfToken in repository https://gitbox.apache.org/repos/asf/beam.git commit 46de8b55a5e5f4621f0ae30ad938f6e55ecc7771 Author: Danny McCormick AuthorDate: Wed Jan 31 13:01:19 2024 -0500 Check if secret is set --- .github/workflows/beam_PreCommit_Python_Coverage.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/beam_PreCommit_Python_Coverage.yml b/.github/workflows/beam_PreCommit_Python_Coverage.yml index 244f37cc208..c48511f21fe 100644 --- a/.github/workflows/beam_PreCommit_Python_Coverage.yml +++ b/.github/workflows/beam_PreCommit_Python_Coverage.yml @@ -71,6 +71,7 @@ jobs: github.event.comment.body == 'Run Python_Coverage PreCommit' steps: - uses: actions/checkout@v4 + - run: echo "${{ secrets.HF_INFERENCE_TOKEN }}" - name: Setup repository uses: ./.github/actions/setup-action with: @@ -100,4 +101,4 @@ jobs: with: commit: '${{ env.prsha || env.GITHUB_SHA }}' comment_mode: ${{ github.event_name == 'issue_comment' && 'always' || 'off' }} - files: '**/pytest*.xml' \ No newline at end of file + files: '**/pytest*.xml'
(beam) branch hf_inference_api_ updated (c618acd61ea -> 90708261cb4)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch hf_inference_api_ in repository https://gitbox.apache.org/repos/asf/beam.git from c618acd61ea Add spaces add 90708261cb4 Address comments No new revisions were added by this update. Summary of changes: .../ml/transforms/embeddings/huggingface.py| 23 -- 1 file changed, 17 insertions(+), 6 deletions(-)
svn commit: r67079 - /dev/beam/2.54.0/python/
Author: lostluck Date: Wed Jan 31 17:49:42 2024 New Revision: 67079 Log: Staging Python artifacts for Apache Beam 2.54.0 RC Added: dev/beam/2.54.0/python/ dev/beam/2.54.0/python/apache-beam-2.54.0.tar.gz (with props) dev/beam/2.54.0/python/apache-beam-2.54.0.tar.gz.asc dev/beam/2.54.0/python/apache-beam-2.54.0.tar.gz.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-macosx_10_9_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-macosx_10_9_x86_64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-macosx_10_9_x86_64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win32.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win32.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win32.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win_amd64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win_amd64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp310-cp310-win_amd64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-macosx_10_9_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-macosx_10_9_x86_64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-macosx_10_9_x86_64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win32.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win32.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win32.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win_amd64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win_amd64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp311-cp311-win_amd64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-macosx_10_9_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-macosx_10_9_x86_64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-macosx_10_9_x86_64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (with props) dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl.asc dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl.sha512 dev/beam/2.54.0/python/apache_beam-2.54.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (with props) dev/beam/2.54.0/python/apache_beam-2.5
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 9b8bdf938aa Updating config from bot 9b8bdf938aa is described below commit 9b8bdf938aad94527ffeef35edec8a89e06d8f8b Author: github-actions AuthorDate: Wed Jan 31 17:34:36 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30164.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30164.json b/scripts/ci/pr-bot/state/pr-state/pr-30164.json new file mode 100644 index 000..242a48d7d3b --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30164.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": true, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch hf_inference_api_ updated (fdb92dfe750 -> c618acd61ea)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch hf_inference_api_ in repository https://gitbox.apache.org/repos/asf/beam.git from fdb92dfe750 Remove space add c618acd61ea Add spaces No new revisions were added by this update. Summary of changes: .github/workflows/beam_PreCommit_Python_Coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c7f4fa7b4dd Updating config from bot c7f4fa7b4dd is described below commit c7f4fa7b4dda01e43c27d68c742fd231783c23b5 Author: github-actions AuthorDate: Wed Jan 31 12:13:47 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index e2626acea48..a4eb5580da8 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -6,7 +6,7 @@ "robertwb": 1706283832852, "kileys": 1674428678843, "apilloud": 1678822446183, -"Abacn": 1706261696319, +"Abacn": 1706703224505, "bvolpato": 1706034929353, "m-trieu": 1706573922958, "damondouglas": 1706703218116
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f300ceb1622 Updating config from bot f300ceb1622 is described below commit f300ceb1622e0e6fe0d2ba9d4e0bc10e697b73ab Author: github-actions AuthorDate: Wed Jan 31 12:13:42 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index 5a504ba2205..bc7ddc2fde1 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -8,6 +8,6 @@ "ahmedabu98": 1706261696319, "bvolpato": 1706648779876, "manavgarg": 1690826779210, -"damondouglas": 1706098433426 +"damondouglas": 1706703218621 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7a99973194d Updating config from bot 7a99973194d is described below commit 7a99973194d9036fb6137a0ef66b9010c98a00b9 Author: github-actions AuthorDate: Wed Jan 31 12:13:48 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index bc7ddc2fde1..0b2a0219fad 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -5,7 +5,7 @@ "johnjcasey": 1706573922958, "pabloem": 1691787951165, "Abacn": 1706288724292, -"ahmedabu98": 1706261696319, +"ahmedabu98": 1706703225114, "bvolpato": 1706648779876, "manavgarg": 1690826779210, "damondouglas": 1706703218621
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 698262d66fb Updating config from bot 698262d66fb is described below commit 698262d66fbb078f30d78c45fafd13ddd9f3d7b5 Author: github-actions AuthorDate: Wed Jan 31 12:13:46 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28763.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28763.json b/scripts/ci/pr-bot/state/pr-state/pr-28763.json index 063ce8593a5..770d8747020 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28763.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28763.json @@ -1,8 +1,8 @@ { "commentedAboutFailingChecks": false, "reviewersAssignedForLabels": { -"java": "robertwb", -"io": "chamikaramj" +"java": "Abacn", +"io": "ahmedabu98" }, "nextAction": "Reviewers", "stopReviewerNotifications": false,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4397aba96c9 Updating config from bot 4397aba96c9 is described below commit 4397aba96c90caf16a7d7fd398e87d1150313b45 Author: github-actions AuthorDate: Wed Jan 31 12:13:41 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 95425e2db38..e2626acea48 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -9,6 +9,6 @@ "Abacn": 1706261696319, "bvolpato": 1706034929353, "m-trieu": 1706573922958, -"damondouglas": 1706098433426 +"damondouglas": 1706703218116 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new a2ad6d6f656 Updating config from bot a2ad6d6f656 is described below commit a2ad6d6f656385bf448258ca8caa5ad51d5a0ba0 Author: github-actions AuthorDate: Wed Jan 31 12:13:40 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29566.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29566.json b/scripts/ci/pr-bot/state/pr-state/pr-29566.json index 1086e5514f7..e41b428b191 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29566.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29566.json @@ -1,8 +1,8 @@ { "commentedAboutFailingChecks": true, "reviewersAssignedForLabels": { -"java": "Abacn", -"io": "ahmedabu98" +"java": "damondouglas", +"io": "damondouglas" }, "nextAction": "Reviewers", "stopReviewerNotifications": false,
(beam) branch asf-site updated: Publishing website 2024/01/31 05:40:06 at commit 1df9d6b
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 6c979535a6f Publishing website 2024/01/31 05:40:06 at commit 1df9d6b 6c979535a6f is described below commit 6c979535a6f425d9a887b0e693bb51b59474c898 Author: runner AuthorDate: Wed Jan 31 05:40:07 2024 + Publishing website 2024/01/31 05:40:06 at commit 1df9d6b --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index d90be465820..2416c66e5f5 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-30T15:10:28-08:00/categories/blog/2024-01-30T15:10:28-08:00/blog/2024-01-30T15:10:28-08:00/categories/2024-01-30T15:10:28-08:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.53.0/2024-01-30T18:45:22-05:00/categories/blog/2024-01-30T18:45:22-05:00/blog/2024-01-30T18:45:22-05:00/categories/2024-01-30T18:45:22-05:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 9ce8fc33b29 Updating config from bot 9ce8fc33b29 is described below commit 9ce8fc33b29673b58daf76f80607c83a1df5e6c9 Author: github-actions AuthorDate: Wed Jan 31 05:34:33 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 8c80a924bcc..5227a37f24f 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -2,7 +2,7 @@ "label": "go", "dateOfLastReviewAssignment": { "lostluck": 1706648775228, -"jrmccluskey": 1706228354853, +"jrmccluskey": 1706679270657, "youngoli": 1657688896155, "damccorm": 1680501930289, "riteshghorse": 1706297685702
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new bfadc1849b6 Updating config from bot bfadc1849b6 is described below commit bfadc1849b62bfce2b887238c92b11308bfad4e2 Author: github-actions AuthorDate: Wed Jan 31 05:34:32 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30162.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30162.json b/scripts/ci/pr-bot/state/pr-state/pr-30162.json new file mode 100644 index 000..1c2c2e31975 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30162.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"go": "jrmccluskey" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.16 created (now 2b4d33c9ee7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.16 in repository https://gitbox.apache.org/repos/asf/beam.git at 2b4d33c9ee7 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks No new revisions were added by this update.
(beam) branch nightly-refs/heads/master updated (ce28c33940c -> 1df9d6bba91)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from ce28c33940c Merge pull request #30007 [YAML] Several improvements to the documentation. add f9029616a91 Update environments.md (#30149) add 097e6e27bfe Bump actions/cache from 3 to 4 (#30033) add f9c9788815e Update 02_io_pubsub.md (#29986) add 5a9a27cbafa Volatile heartbeat (#30148) add b9fd39cb16b Fix Dataproc cleanup race condition (#30154) add 22fefebacab BigQuery: Decouple clustering from time partitioning when writing (#30094) add 415b1658f50 Merge pull request #30137: Fix a misspelling in zetasql overview page add 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) No new revisions were added by this update. Summary of changes: .github/workflows/README.md| 1 + .../workflows/beam_CleanUpDataprocResources.yml| 2 +- .../workflows/beam_PostCommit_Java_DataflowV2.yml | 4 +-- .github/workflows/playground_frontend_test.yml | 2 +- .github/workflows/tour_of_beam_frontend_test.yml | 2 +- .test-infra/dataproc/cleanup.sh| 4 +-- CHANGES.md | 3 +- learning/prompts/code-explaination/02_io_pubsub.md | 8 ++--- .../runners/dataflow/worker/streaming/Work.java| 2 +- .../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 10 +++--- .../sdk/io/gcp/bigquery/CreateTableHelpers.java| 10 +++--- .../gcp/bigquery/DynamicDestinationsHelpers.java | 42 -- .../io/gcp/bigquery/UpdateSchemaDestination.java | 9 ++--- .../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 38 +++- .../en/documentation/dsls/sql/zetasql/overview.md | 4 +-- .../en/documentation/runtime/environments.md | 2 +- 16 files changed, 78 insertions(+), 65 deletions(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2f2a433c713 Updating config from bot 2f2a433c713 is described below commit 2f2a433c713935273a00e519d9f4341c85d86df8 Author: github-actions AuthorDate: Wed Jan 31 00:24:01 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30161.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30161.json b/scripts/ci/pr-bot/state/pr-state/pr-30161.json new file mode 100644 index 000..9c2aa5aa212 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30161.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": true, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new ae9fa2f7e3d Updating config from bot ae9fa2f7e3d is described below commit ae9fa2f7e3d8cb71343bc51e13c3e008f1233ffc Author: github-actions AuthorDate: Wed Jan 31 00:19:18 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 20ae8d73ce9..b4c7447bda5 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -3,7 +3,7 @@ "dateOfLastReviewAssignment": { "Abacn": 1706405351155, "AnandInguva": 170657132, -"chamikaramj": 1704994473508, +"chamikaramj": 1706660356104, "damccorm": 1704998100377, "johnjcasey": 1705059270349, "jrmccluskey": 1705430101174,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new e5ba274e478 Updating config from bot e5ba274e478 is described below commit e5ba274e478e7cc7d974b17b3b425b72688ac94f Author: github-actions AuthorDate: Wed Jan 31 00:19:17 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30141.json | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30141.json b/scripts/ci/pr-bot/state/pr-state/pr-30141.json index 242a48d7d3b..26e4be5dc8a 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-30141.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-30141.json @@ -1,7 +1,9 @@ { - "commentedAboutFailingChecks": true, - "reviewersAssignedForLabels": {}, - "nextAction": "Author", + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"no-matching-label": "chamikaramj" + }, + "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], "committerAssigned": false
(beam) branch master updated (415b1658f50 -> 1df9d6bba91)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 415b1658f50 Merge pull request #30137: Fix a misspelling in zetasql overview page add 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) No new revisions were added by this update. Summary of changes: .github/workflows/beam_PostCommit_Java_DataflowV2.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(beam) branch asf-site updated: Publishing website 2024/01/30 23:39:32 at commit 415b165
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 8c6dae1cb92 Publishing website 2024/01/30 23:39:32 at commit 415b165 8c6dae1cb92 is described below commit 8c6dae1cb923e8b9728cda6f4a7b78d113c92f9d Author: runner AuthorDate: Tue Jan 30 23:39:32 2024 + Publishing website 2024/01/30 23:39:32 at commit 415b165 --- .../documentation/dsls/sql/zetasql/overview/index.html | 2 +- website/generated-content/sitemap.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/generated-content/documentation/dsls/sql/zetasql/overview/index.html b/website/generated-content/documentation/dsls/sql/zetasql/overview/index.html index bc96d6bf2c5..c1947a4581f 100644 --- a/website/generated-content/documentation/dsls/sql/zetasql/overview/index.html +++ b/website/generated-content/documentation/dsls/sql/zetasql/overview/index.html @@ -36,7 +36,7 @@ https://beam.apache.org/documentation/ml/overview/> -
(beam) branch master updated (22fefebacab -> 415b1658f50)
This is an automated email from the ASF dual-hosted git repository. melap pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 22fefebacab BigQuery: Decouple clustering from time partitioning when writing (#30094) add 415b1658f50 Merge pull request #30137: Fix a misspelling in zetasql overview page No new revisions were added by this update. Summary of changes: .../www/site/content/en/documentation/dsls/sql/zetasql/overview.md| 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new ece9e3ef53d Updating config from bot ece9e3ef53d is described below commit ece9e3ef53d5bf44c60b9edeee239828699d3148 Author: github-actions AuthorDate: Tue Jan 30 22:34:00 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30160.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30160.json b/scripts/ci/pr-bot/state/pr-state/pr-30160.json new file mode 100644 index 000..9c2aa5aa212 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30160.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": true, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch master updated (b9fd39cb16b -> 22fefebacab)
This is an automated email from the ASF dual-hosted git repository. damondouglas pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from b9fd39cb16b Fix Dataproc cleanup race condition (#30154) add 22fefebacab BigQuery: Decouple clustering from time partitioning when writing (#30094) No new revisions were added by this update. Summary of changes: CHANGES.md | 3 +- .../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 10 +++--- .../sdk/io/gcp/bigquery/CreateTableHelpers.java| 10 +++--- .../gcp/bigquery/DynamicDestinationsHelpers.java | 42 -- .../io/gcp/bigquery/UpdateSchemaDestination.java | 9 ++--- .../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 38 +++- 6 files changed, 62 insertions(+), 50 deletions(-)
(beam) branch hf_inference_api_ updated (935cd5a34d2 -> fdb92dfe750)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch hf_inference_api_ in repository https://gitbox.apache.org/repos/asf/beam.git from 935cd5a34d2 Pass env variable in tox add fdb92dfe750 Remove space No new revisions were added by this update. Summary of changes: .github/workflows/beam_PreCommit_Python_Coverage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f2a237a9c5f Updating config from bot f2a237a9c5f is described below commit f2a237a9c5f8f7f0c889e0003728f6c7a3b67945 Author: github-actions AuthorDate: Tue Jan 30 21:06:24 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index e3aba2b8bfe..5a504ba2205 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -6,7 +6,7 @@ "pabloem": 1691787951165, "Abacn": 1706288724292, "ahmedabu98": 1706261696319, -"bvolpato": 1706034929353, +"bvolpato": 1706648779876, "manavgarg": 1690826779210, "damondouglas": 1706098433426 }
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new edf21b2fcef Updating config from bot edf21b2fcef is described below commit edf21b2fcef83d344bc91d7c3f371c381f816129 Author: github-actions AuthorDate: Tue Jan 30 20:53:55 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30158.json | 8 1 file changed, 8 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30158.json b/scripts/ci/pr-bot/state/pr-state/pr-30158.json new file mode 100644 index 000..9c2aa5aa212 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30158.json @@ -0,0 +1,8 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": {}, + "nextAction": "Author", + "stopReviewerNotifications": true, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam-site) branch updates_release_2.54.0 created (now 33a231d205)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch updates_release_2.54.0 in repository https://gitbox.apache.org/repos/asf/beam-site.git at 33a231d205 Update beam-site for release 2.54.0. No new revisions were added by this update.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 05c76e641be Updating config from bot 05c76e641be is described below commit 05c76e641bec7baebfa97c7cddb7bbdb0880ad56 Author: github-actions AuthorDate: Tue Jan 30 21:06:21 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30156.json | 11 +++ 1 file changed, 11 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30156.json b/scripts/ci/pr-bot/state/pr-state/pr-30156.json new file mode 100644 index 000..55cd2385442 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30156.json @@ -0,0 +1,11 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"python": "jrmccluskey", +"io": "bvolpato" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 3d3436b6ad4 Updating config from bot 3d3436b6ad4 is described below commit 3d3436b6ad4b3532b472f4680e13e1b05b2d421e Author: github-actions AuthorDate: Tue Jan 30 21:06:22 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index a78567af7a8..a886eeb9490 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -9,7 +9,7 @@ "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1706565938559, -"jrmccluskey": 1706149095380, +"jrmccluskey": 1706648779876, "riteshghorse": 1706200695261, "liferoad": 1706288724292 }
(beam) annotated tag sdks/v2.54.0-RC1 updated (36b2a54baa2 -> 958a0c8da25)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to annotated tag sdks/v2.54.0-RC1 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag sdks/v2.54.0-RC1 was modified! *** from 36b2a54baa2 (commit) to 958a0c8da25 (tag) tagging 36b2a54baa2cafd272734383e45aca07f227def5 (commit) replaces jupyterlab-sidepanel-v3.0.0 by lostluck on Tue Jan 30 12:14:18 2024 -0800 - Log - Go SDK v2.54.0-RC1 --- No new revisions were added by this update. Summary of changes:
(beam) annotated tag v2.54.0-RC1 updated (36b2a54baa2 -> 9c7ba93b58d)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to annotated tag v2.54.0-RC1 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag v2.54.0-RC1 was modified! *** from 36b2a54baa2 (commit) to 9c7ba93b58d (tag) tagging 36b2a54baa2cafd272734383e45aca07f227def5 (commit) replaces jupyterlab-sidepanel-v3.0.0 by lostluck on Tue Jan 30 12:14:18 2024 -0800 - Log - v2.54.0-RC1 --- No new revisions were added by this update. Summary of changes:
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d060583c1c2 Updating config from bot d060583c1c2 is described below commit d060583c1c2751226bbed28f3b39e63155b9727b Author: github-actions AuthorDate: Tue Jan 30 21:06:18 2024 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 8e0aed32e78..8c80a924bcc 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -1,7 +1,7 @@ { "label": "go", "dateOfLastReviewAssignment": { -"lostluck": 1703136882364, +"lostluck": 1706648775228, "jrmccluskey": 1706228354853, "youngoli": 1657688896155, "damccorm": 1680501930289,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6656acd7300 Updating config from bot 6656acd7300 is described below commit 6656acd73009567d82f412d759d62a0e0c6eeb51 Author: github-actions AuthorDate: Tue Jan 30 21:06:16 2024 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-30157.json | 10 ++ 1 file changed, 10 insertions(+) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30157.json b/scripts/ci/pr-bot/state/pr-state/pr-30157.json new file mode 100644 index 000..8ea94810368 --- /dev/null +++ b/scripts/ci/pr-bot/state/pr-state/pr-30157.json @@ -0,0 +1,10 @@ +{ + "commentedAboutFailingChecks": false, + "reviewersAssignedForLabels": { +"go": "lostluck" + }, + "nextAction": "Reviewers", + "stopReviewerNotifications": false, + "remindAfterTestsPass": [], + "committerAssigned": false +} \ No newline at end of file
(beam) 01/01: Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult
This is an automated email from the ASF dual-hosted git repository. janl pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 36cab4ffe76530ca5993a61a40e2616621567e71 Merge: 1df9d6bba91 4c5ee349f9b Author: Jan Lukavský AuthorDate: Wed Jan 31 10:03:38 2024 +0100 Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult .../src/main/java/org/apache/beam/sdk/transforms/join/CoGbkResult.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
(beam) branch master updated (1df9d6bba91 -> 36cab4ffe76)
This is an automated email from the ASF dual-hosted git repository. janl pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 1df9d6bba91 Bumping timeout for beam_PostCommit_Java_DataflowV2 (#30160) add 4c5ee349f9b [sdks-java-core] Test for Reiterator outside loop in CoGbkResult new 36cab4ffe76 Merge pull request #30163: [sdks-java-core] Test for Reiterator outside loop in CoGbkResult The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../src/main/java/org/apache/beam/sdk/transforms/join/CoGbkResult.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)