(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 3cec9f51035 Updating config from bot
3cec9f51035 is described below
commit 3cec9f51035a1accae027d24efee150bcb5de33c
Author: github-actions
AuthorDate: Fri Dec 8 06:35:17 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29679.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29679.json
b/scripts/ci/pr-bot/state/pr-state/pr-29679.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29679.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch asf-site updated: Publishing website 2023/12/08 05:40:43 at commit 85bf388
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 44bbd528b8a Publishing website 2023/12/08 05:40:43 at commit 85bf388 44bbd528b8a is described below commit 44bbd528b8ad9b96c97a49b1fdbb24579824dbbd Author: runner AuthorDate: Fri Dec 8 05:40:44 2023 + Publishing website 2023/12/08 05:40:43 at commit 85bf388 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 1ec97c3df7c..eda98b98f70 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.52.0/2023-12-07T15:28:57-08:00/categories/blog/2023-12-07T15:28:57-08:00/blog/2023-12-07T15:28:57-08:00/categories/2023-12-07T15:28:57-08:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.52.0/2023-12-07T17:31:04-08:00/categories/blog/2023-12-07T17:31:04-08:00/blog/2023-12-07T17:31:04-08:00/categories/2023-12-07T17:31:04-08:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 564a396d317 Updating config from bot
564a396d317 is described below
commit 564a396d3174f21132ef48398bf57f360243df67
Author: github-actions
AuthorDate: Fri Dec 8 05:34:27 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 5e8399e7de8..f015d5c4aa2 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1701923731135,
-"Abacn": 1701923726209
+"Abacn": 1702013664696
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b9cd022556d Updating config from bot
b9cd022556d is described below
commit b9cd022556dfed30e09b09a4d83c8bf3b3818305
Author: github-actions
AuthorDate: Fri Dec 8 05:34:26 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29682.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29682.json
b/scripts/ci/pr-bot/state/pr-state/pr-29682.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29682.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9b3cc7ea132 Updating config from bot
9b3cc7ea132 is described below
commit 9b3cc7ea132162064c7963bce09064a2accf43db
Author: github-actions
AuthorDate: Fri Dec 8 05:05:51 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29678.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29678.json
b/scripts/ci/pr-bot/state/pr-state/pr-29678.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29678.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 3bfd204f555 Updating config from bot
3bfd204f555 is described below
commit 3bfd204f555288cb00ac3ea2d324fcf798b478ed
Author: github-actions
AuthorDate: Fri Dec 8 05:05:48 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29680.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29680.json
b/scripts/ci/pr-bot/state/pr-state/pr-29680.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29680.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d5a1ec2cda2 Updating config from bot
d5a1ec2cda2 is described below
commit d5a1ec2cda24c360c5baca56b8193a95650b4a7b
Author: github-actions
AuthorDate: Fri Dec 8 05:05:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29677.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29677.json
b/scripts/ci/pr-bot/state/pr-state/pr-29677.json
new file mode 100644
index 000..8ea94810368
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29677.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 7b11f0c0b0c Updating config from bot
7b11f0c0b0c is described below
commit 7b11f0c0b0c91669cdcc8024c8674020355bd617
Author: github-actions
AuthorDate: Fri Dec 8 05:05:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 945014cdafa..83f564f7148 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1701927269486,
+"lostluck": 1702011954363,
"jrmccluskey": 1701961576077,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1a9c840c052 Updating config from bot 1a9c840c052 is described below commit 1a9c840c052f36d1e8868e5500baee2eddfa35f2 Author: github-actions AuthorDate: Fri Dec 8 05:05:53 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 69202bdb292..945014cdafa 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1701961576077, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1701927265858 +"riteshghorse": 1702011950233 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 437cc5f71ee Updating config from bot
437cc5f71ee is described below
commit 437cc5f71ee43d434ec5da2f7339c70eac90936a
Author: github-actions
AuthorDate: Fri Dec 8 05:05:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29681.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29681.json
b/scripts/ci/pr-bot/state/pr-state/pr-29681.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29681.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch dependabot/github_actions/actions/stale-9 created (now e84700ecc71)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/stale-9 in repository https://gitbox.apache.org/repos/asf/beam.git at e84700ecc71 Bump actions/stale from 8 to 9 No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.47.3 deleted (was 1b9a379bd03)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.47.3 in repository https://gitbox.apache.org/repos/asf/beam.git was 1b9a379bd03 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.47.4 created (now d28e5ac85e3)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.47.4 in repository https://gitbox.apache.org/repos/asf/beam.git at d28e5ac85e3 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.6 created (now 9e11483972e)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.6 in repository https://gitbox.apache.org/repos/asf/beam.git at 9e11483972e Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.5 deleted (was c496d93530b)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.5 in repository https://gitbox.apache.org/repos/asf/beam.git was c496d93530b Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.10 deleted (was 58bf0c47c8b)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.10 in repository https://gitbox.apache.org/repos/asf/beam.git was 58bf0c47c8b Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.11 created (now 193fb4c8075)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.16.11 in repository https://gitbox.apache.org/repos/asf/beam.git at 193fb4c8075 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.26.0 created (now 7d3260c5bec)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.26.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 7d3260c5bec Bump github.com/aws/aws-sdk-go-v2/config from 1.25.8 to 1.26.0 in /sdks No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.21.0 created (now 9e6bc1ac4a5)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.21.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 9e6bc1ac4a5 Bump cloud.google.com/go/bigtable from 1.20.0 to 1.21.0 in /sdks No new revisions were added by this update.
(beam) branch nightly-refs/heads/master updated (54c18943bfb -> 85bf3888959)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from 54c18943bfb Bump google.golang.org/api from 0.152.0 to 0.153.0 in /sdks (#29628) add 70926f5da78 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657) add ee5e782c525 Bump actions/setup-go from 4 to 5 (#29653) add 5d11c20cd4e Add Error Handling to Kafka IO (#29546) add e620b5aac97 Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 in /sdks (#29630) add d2d19ef5dfd Bugfix/extend java precommit timeout (#29664) add 8d9fd02af5e Bump Java VR interval and shutdown remaining Dataflow VR Jenkins PostCommit (#29647) add 84d070bae9b Add per-worker keys to stateful BatchElements, plumb through batching options to all model handlers (#29642) add 33a122cd2ac Add additional log messages and javadoc around reads (#29640) add 51d71c0c43e Remove experimental note about streaming. (#29652) add 174e49dc6aa Add cloud profiler dependency to 3.11 containers. (#29651) add 90dd93f5241 Disable failing mypy check in yaml_io.py (#29666) add dab5e86b912 Fix for signature exceptions in ptransform_fn. add 07ebdc6a6f5 Guard entire signature setting. add e15522f007f Merge pull request #29648 Fix for signature exceptions in ptransform_fn. add b426dfaf5ad Add the ability to generate markdown documentation from a set of providers. (#29639) add 74308c39b8b Update to Libraries-Bom 26.28.0 add 6e50c1bbc88 Update libraries-bom java url version to 26.28.0 add be9155205dd Merge pull request #29641 from lqiu96/update-libraries-bom-26.28.0 add 5becfb8ed43 Archive JUnit test result on timeout (#29667) add 31822734526 Add a pipeline update compatibility version option. (#29140) add f1a7f71266c Bump the image used by Dataflow runner in dev SDKs. (#29672) add ed3a58217f4 Fix some lint errors in Python. (#29675) add 85bf3888959 More efficient map side inputs for small maps. (#29587) No new revisions were added by this update. Summary of changes: ...eam_LoadTests_Java_CoGBK_Dataflow_Streaming.yml | 2 +- ...PerformanceTests_BigQueryIO_Batch_Java_Avro.yml | 2 +- ...PerformanceTests_BigQueryIO_Batch_Java_Json.yml | 2 +- ..._PerformanceTests_BigQueryIO_Streaming_Java.yml | 2 +- ...m_PerformanceTests_SQLBigQueryIO_Batch_Java.yml | 2 +- .github/workflows/beam_PostCommit_Java.yml | 2 +- .../beam_PostCommit_Java_Avro_Versions.yml | 2 +- .../beam_PostCommit_Java_BigQueryEarlyRollout.yml | 2 +- .../workflows/beam_PostCommit_Java_DataflowV1.yml | 2 +- .../workflows/beam_PostCommit_Java_DataflowV2.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 2 +- ...beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_V2.yml | 2 +- ...m_PostCommit_Java_Examples_Dataflow_V2_Java.yml | 2 +- .../beam_PostCommit_Java_Examples_Direct.yml | 2 +- .../beam_PostCommit_Java_Examples_Flink.yml| 2 +- .../beam_PostCommit_Java_Examples_Spark.yml| 2 +- .../beam_PostCommit_Java_Hadoop_Versions.yml | 2 +- .../beam_PostCommit_Java_IO_Performance_Tests.yml | 2 +- .../beam_PostCommit_Java_Jpms_Dataflow_Java11.yml | 2 +- .../beam_PostCommit_Java_Jpms_Dataflow_Java17.yml | 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java11.yml| 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java17.yml| 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java21.yml| 2 +- .../beam_PostCommit_Java_Jpms_Flink_Java11.yml | 2 +- .../beam_PostCommit_Java_Jpms_Spark_Java11.yml | 2 +- .../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 +- .../workflows/beam_PostCommit_Java_PVR_Samza.yml | 2 +- .../beam_PostCommit_Java_PVR_Spark3_Streaming.yml | 2 +- .../beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 +- .github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 +- ...am_PostCommit_Java_ValidatesRunner_Dataflow.yml | 4 +- ..._Java_ValidatesRunner_Dataflow_JavaVersions.yml | 4 +- ...mit_Java_ValidatesRunner_Dataflow_Streaming.yml | 4 +- ...PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 +- ..._Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 +- ...beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 +- ...it_Java_ValidatesRunner_Direct_JavaVersions.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 2 +- ...ostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Samza.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Spark.yml | 2 +- ...va_ValidatesRunner_SparkStructuredStreaming.yml | 2 +- ...ostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 +- ...am_PostCommit_Java_ValidatesRunner_Twister2.yml | 2 +- .../beam_PostCom
(beam) branch master updated: More efficient map side inputs for small maps. (#29587)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 85bf3888959 More efficient map side inputs for small maps. (#29587)
85bf3888959 is described below
commit 85bf38889599ffabdb6a42dd0ab345b6105fca8f
Author: Robert Bradshaw
AuthorDate: Thu Dec 7 17:31:04 2023 -0800
More efficient map side inputs for small maps. (#29587)
The ability to perform point lookups for multi-map side inputs is great
for maps that are too large to fit into memory, but can be very inefficient
in requiring an entire state request per key for small maps.
This change adds an optional protocol to request an entire map as a stream
of key-values in one (possibly paginated) API call, and uses this to bulk
pre-fetch an initial set of values from the map before falling back to
point lookups.
---
.../beam/model/fn_execution/v1/beam_fn_api.proto | 34 +++-
.../fnexecution/control/RemoteExecutionTest.java | 13 ++-
.../beam/fn/harness/state/MultimapSideInput.java | 94 +++---
.../fn/harness/state/FakeBeamFnStateClient.java| 4 +
.../fn/harness/state/MultimapSideInputTest.java| 50 ++--
.../runners/portability/fn_api_runner/execution.py | 21 -
.../portability/fn_api_runner/worker_handlers.py | 53 +++-
.../apache_beam/runners/worker/bundle_processor.py | 49 ++-
8 files changed, 274 insertions(+), 44 deletions(-)
diff --git
a/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
b/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
index 777ce8636b7..4b40c7fa4e4 100644
---
a/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
+++
b/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
@@ -52,7 +52,7 @@ import "google/protobuf/duration.proto";
message FnApiTransforms {
enum Runner {
// DataSource is a Root Transform, and a source of data for downstream
-// transforms in the same ProcessBundleDescriptor.
+// transforms in the same ProcessBundleDescriptor.
// It represents a stream of values coming in from an external source/over
// a data channel, typically from the runner. It's not the PCollection
itself
// but a description of how to get the portion of the PCollection for a
given
@@ -82,7 +82,7 @@ message FnApiTransforms {
// request will be sent with the matching instruction ID and transform ID.
// Each PCollection that exits the ProcessBundleDescriptor subgraph will
have
// it's own DataSink, keyed by a transform ID determined by the runner.
-//
+//
// The DataSink will take in a stream of elements for a given instruction
ID
// and encode them for transmission to the remote sink. The coder ID must
be
// for a windowed value coder.
@@ -924,6 +924,35 @@ message StateKey {
bytes window = 3;
}
+ // Represents a request for the keys and values associated with a specified
window in a PCollection. See
+ // https://s.apache.org/beam-fn-state-api-and-bundle-processing for further
+ // details.
+ //
+ // This is expected to be more efficient than iterating over they keys and
+ // looking up the values one at a time. If a runner chooses not to implement
+ // this protocol, or a key has too many values to fit into a single response,
+ // the runner is free to fail the request and a fallback to point lookups
+ // will be performed by the SDK.
+ //
+ // Can only be used to perform StateGetRequests on side inputs of the URN
+ // beam:side_input:multimap:v1.
+ //
+ // For a PCollection>, the response data stream will be a
+ // concatenation of all KVs associated with the specified window,
+ // encoded with the the KV> coder.
+ // See
+ // https://s.apache.org/beam-fn-api-send-and-receive-data for further
+ // details.
+ message MultimapKeysValuesSideInput {
+// (Required) The id of the PTransform containing a side input.
+string transform_id = 1;
+// (Required) The id of the side input.
+string side_input_id = 2;
+// (Required) The window (after mapping the currently executing elements
+// window into the side input windows domain) encoded in a nested context.
+bytes window = 3;
+ }
+
// Represents a request for an unordered set of values associated with a
// specified user key and window for a PTransform. See
// https://s.apache.org/beam-fn-state-api-and-bundle-processing for further
@@ -999,6 +1028,7 @@ message StateKey {
BagUserState bag_user_state = 3;
IterableSideInput iterable_side_input = 4;
MultimapKeysSideInput multimap_keys_side_input = 5;
+MultimapKeysValuesSideInput multimap_keys_values_side_input = 8;
MultimapKeysUserState
(beam) branch master updated: Fix some lint errors in Python. (#29675)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new ed3a58217f4 Fix some lint errors in Python. (#29675)
ed3a58217f4 is described below
commit ed3a58217f42f59cfe90592bf08e545ca9b5c3cf
Author: Robert Bradshaw
AuthorDate: Thu Dec 7 16:39:42 2023 -0800
Fix some lint errors in Python. (#29675)
---
sdks/python/apache_beam/yaml/generate_yaml_docs.py | 8
sdks/python/apache_beam/yaml/yaml_provider.py | 5 ++---
2 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/sdks/python/apache_beam/yaml/generate_yaml_docs.py
b/sdks/python/apache_beam/yaml/generate_yaml_docs.py
index d3d6c0a3e31..1ed629310c1 100644
--- a/sdks/python/apache_beam/yaml/generate_yaml_docs.py
+++ b/sdks/python/apache_beam/yaml/generate_yaml_docs.py
@@ -20,11 +20,9 @@ import re
import yaml
-from apache_beam.yaml import yaml_provider
from apache_beam.portability.api import schema_pb2
-from apache_beam.typehints import native_type_compatibility
-from apache_beam.typehints.schemas import typing_from_runner_api
from apache_beam.utils import subprocess_server
+from apache_beam.yaml import yaml_provider
def _fake_value(name, beam_type):
@@ -96,7 +94,9 @@ def config_docs(schema):
elif type_info == "iterable_type":
return f'Iterable[{pretty_type(beam_type.iterable_type.element_type)}]'
elif type_info == "map_type":
- return f'Map[{pretty_type(beam_type.map_type.key_type)},
{pretty_type(beam_type.map_type.value_type)}]'
+ return (
+ f'Map[{pretty_type(beam_type.map_type.key_type)}, '
+ f'{pretty_type(beam_type.map_type.value_type)}]')
elif type_info == "row_type":
return 'Row'
else:
diff --git a/sdks/python/apache_beam/yaml/yaml_provider.py
b/sdks/python/apache_beam/yaml/yaml_provider.py
index 5ba15ff17e8..6a2d313183e 100755
--- a/sdks/python/apache_beam/yaml/yaml_provider.py
+++ b/sdks/python/apache_beam/yaml/yaml_provider.py
@@ -48,11 +48,9 @@ from apache_beam.portability.api import schema_pb2
from apache_beam.transforms import external
from apache_beam.transforms import window
from apache_beam.transforms.fully_qualified_named_transform import
FullyQualifiedNamedTransform
-from apache_beam.typehints import native_type_compatibility
from apache_beam.typehints import schemas
from apache_beam.typehints import trivial_inference
from apache_beam.typehints.schemas import named_tuple_to_schema
-from apache_beam.typehints.schemas import typing_from_runner_api
from apache_beam.typehints.schemas import typing_to_runner_api
from apache_beam.utils import python_callable
from apache_beam.utils import subprocess_server
@@ -460,7 +458,8 @@ class InlineProvider(Provider):
docstring = self._transform_factories[typ].__doc__ or ''
# These "extra" docstring parameters are not relevant for YAML and mess
# up the parsing.
-docstring = re.sub('Pandas Parameters\s+-.*', '', docstring,
flags=re.S)
+docstring = re.sub(
+r'Pandas Parameters\s+-.*', '', docstring, flags=re.S)
return docstring_parser.parse(
docstring, docstring_parser.DocstringStyle.GOOGLE)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1a582915c80 Updating config from bot 1a582915c80 is described below commit 1a582915c80b25b14a54e074763b219a54ed709a Author: github-actions AuthorDate: Fri Dec 8 00:20:17 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 2f440297462..ad73aa57c97 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1701459274135, +"tvalentyn": 1701994812757, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1701741340223,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bbdcef0ff38 Updating config from bot
bbdcef0ff38 is described below
commit bbdcef0ff3874ca57bbbfc6c39ec73e2724bf517
Author: github-actions
AuthorDate: Fri Dec 8 00:20:14 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29675.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29675.json
b/scripts/ci/pr-bot/state/pr-state/pr-29675.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29675.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch asf-site updated: Publishing website 2023/12/07 23:39:47 at commit f1a7f71
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new be3b69bdb28 Publishing website 2023/12/07 23:39:47 at commit f1a7f71 be3b69bdb28 is described below commit be3b69bdb28e7b8bf8a96d0f68bc92f57e002727 Author: runner AuthorDate: Thu Dec 7 23:39:47 2023 + Publishing website 2023/12/07 23:39:47 at commit f1a7f71 --- website/generated-content/documentation/runners/direct/index.html | 2 +- website/generated-content/documentation/sdks/python/index.html| 2 +- website/generated-content/sitemap.xml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/generated-content/documentation/runners/direct/index.html b/website/generated-content/documentation/runners/direct/index.html index 5c728326b30..bb9b8fed7de 100644 --- a/website/generated-content/documentation/runners/direct/index.html +++ b/website/generated-content/documentation/runners/direct/index.html @@ -44,7 +44,7 @@ This section is not applicable to the Beam SDK for Python.Pipeline options for the Direct RunnerFor general instructions on how to set pipeline options, see the https://beam.apache.org/releases/javadoc/2.52.0/index.html?org/apache/beam/runners/direct/DirectOptions.html>DirectOptions https://beam.apache.org/releases/pydoc/2.52.0/apache_beam.options.pipeline_options.html#apache_beam.options.pipeline_options.DirectOptions>DirectOptions -interface for defaults and additional pipeline configuration options.Additional information and caveatsMemory considerationsLocal execution is limited by the memory available in your local environment. It is highly recommended that you run your pipeline with data sets small enough to fit in local memory. You can create a small in-memory data set using a https://beam.a [...] +interface for defaults and additional pipeline configuration options.Additional information and caveatsMemory considerationsLocal execution is limited by the memory available in your local environment. It is highly recommended that you run your pipeline with data sets small enough to fit in local memory. You can create a small in-memory data set using a https://beam.a [...] By default, targetParallelism is the greater of the number of available processors and 3.Number of threads or subprocesses is defined by setting the direct_num_workers pipeline option. From 2.22.0, direct_num_workers = 0 is supported. When direct_num_workers is set to 0, it will set the number of threads/subprocess to the number of cores of the machine where the pipeline is running.Setting running modeIn Beam 2.19.0 and newer, you can use the direct_running_mode pipeline option to set the running mode. direct_running_mode can be one of ['in_memory', 'multi_threading', 'multi_processing'].in_memory: Runner and workers’ communication happens in memory (not through gRPC). This is a default mode.multi_threading: Runner and workers communicate through gRPC and each worker runs in a thread.multi_processing: Runner and workers communicate th [...] diff --git a/website/generated-content/documentation/sdks/python/index.html b/website/generated-content/documentation/sdks/python/index.html index 51e87098990..1842acf2d3e 100644 --- a/website/generated-content/documentation/sdks/python/index.html +++ b/website/generated-content/documentation/sdks/python/index.html @@ -37,7 +37,7 @@
(beam) branch master updated: Bump the image used by Dataflow runner in dev SDKs. (#29672)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new f1a7f71266c Bump the image used by Dataflow runner in dev SDKs. (#29672) f1a7f71266c is described below commit f1a7f71266cfb53c94eb6ed982c1b1bb137b6ffb Author: tvalentyn AuthorDate: Thu Dec 7 15:28:57 2023 -0800 Bump the image used by Dataflow runner in dev SDKs. (#29672) --- sdks/python/apache_beam/runners/dataflow/internal/names.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/python/apache_beam/runners/dataflow/internal/names.py b/sdks/python/apache_beam/runners/dataflow/internal/names.py index 0dfe0182eeb..82e2306ced3 100644 --- a/sdks/python/apache_beam/runners/dataflow/internal/names.py +++ b/sdks/python/apache_beam/runners/dataflow/internal/names.py @@ -34,6 +34,6 @@ SERIALIZED_SOURCE_KEY = 'serialized_source' # Unreleased sdks use container image tag specified below. # Update this tag whenever there is a change that # requires changes to SDK harness container or SDK harness launcher. -BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231205' +BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231207' DATAFLOW_CONTAINER_IMAGE_REPOSITORY = 'gcr.io/cloud-dataflow/v1beta3'
(beam) branch tvalentyn-patch-3 deleted (was d9906ea0938)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git was d9906ea0938 Update names.py The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4ba7b0c711f Updating config from bot
4ba7b0c711f is described below
commit 4ba7b0c711f72cb1bed186b23b33191c83dbdb95
Author: github-actions
AuthorDate: Thu Dec 7 23:10:54 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29676.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29676.json
b/scripts/ci/pr-bot/state/pr-state/pr-29676.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29676.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch tvalentyn-patch-5 created (now 65c9d77e520)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-5 in repository https://gitbox.apache.org/repos/asf/beam.git at 65c9d77e520 Retry building a wheel up to 3 times. This branch includes the following new commits: new 65c9d77e520 Retry building a wheel up to 3 times. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Retry building a wheel up to 3 times.
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch tvalentyn-patch-5
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 65c9d77e5207517b3f70f0e0134b8ef7fa56272d
Author: tvalentyn
AuthorDate: Thu Dec 7 15:08:05 2023 -0800
Retry building a wheel up to 3 times.
---
sdks/python/build.gradle | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/sdks/python/build.gradle b/sdks/python/build.gradle
index 7795e77e396..ab6f75fc653 100644
--- a/sdks/python/build.gradle
+++ b/sdks/python/build.gradle
@@ -99,11 +99,12 @@ platform_identifiers_map.each { platform, idsuffix ->
environment CIBW_ENVIRONMENT: "SETUPTOOLS_USE_DISTUTILS=stdlib"
// note: sync cibuildwheel version with GitHub Action
// .github/workflow/build_wheel.yml:build_wheels "Install
cibuildwheel" step
+ // note(https://github.com/pypa/cibuildwheel/issues/1692):
cibuildwheel appears to timeout occasionally.
executable 'sh'
args '-c', ". ${envdir}/bin/activate && " +
"pip install cibuildwheel==2.9.0 && " +
"cibuildwheel --print-build-identifiers --platform ${platform}
--archs ${archs} && " +
- "cibuildwheel --output-dir ${buildDir} --platform ${platform}
--archs ${archs}"
+ "for i in {1..3}; do cibuildwheel --output-dir ${buildDir}
--platform ${platform} --archs ${archs} && break; done"
}
}
}
@@ -162,4 +163,4 @@ tasks.register("wordCount") {
args '-c', ". ${envdir}/bin/activate && python -m
apache_beam.examples.wordcount --runner DirectRunner --output /tmp/output.txt"
}
}
-}
\ No newline at end of file
+}
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e3ef4e2b4f5 Updating config from bot
e3ef4e2b4f5 is described below
commit e3ef4e2b4f5e09bbe50ccc4c99bef4c4045967aa
Author: github-actions
AuthorDate: Thu Dec 7 22:10:37 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29672.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29672.json
b/scripts/ci/pr-bot/state/pr-state/pr-29672.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29672.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c7cf4cc3f9d Updating config from bot
c7cf4cc3f9d is described below
commit c7cf4cc3f9de0cb233c293b14bc2695abbddc3f8
Author: github-actions
AuthorDate: Thu Dec 7 22:04:15 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29673.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29673.json
b/scripts/ci/pr-bot/state/pr-state/pr-29673.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29673.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9b587b10ea5 Updating config from bot
9b587b10ea5 is described below
commit 9b587b10ea55fc9a31dcc7eafea9f3d1a205cb68
Author: github-actions
AuthorDate: Thu Dec 7 21:56:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29671.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29671.json
b/scripts/ci/pr-bot/state/pr-state/pr-29671.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29671.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) 01/01: Update names.py
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a commit to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git commit d9906ea0938fbaf6c03a60daba5cffe713325bca Author: tvalentyn AuthorDate: Thu Dec 7 13:56:30 2023 -0800 Update names.py --- sdks/python/apache_beam/runners/dataflow/internal/names.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/python/apache_beam/runners/dataflow/internal/names.py b/sdks/python/apache_beam/runners/dataflow/internal/names.py index 0dfe0182eeb..82e2306ced3 100644 --- a/sdks/python/apache_beam/runners/dataflow/internal/names.py +++ b/sdks/python/apache_beam/runners/dataflow/internal/names.py @@ -34,6 +34,6 @@ SERIALIZED_SOURCE_KEY = 'serialized_source' # Unreleased sdks use container image tag specified below. # Update this tag whenever there is a change that # requires changes to SDK harness container or SDK harness launcher. -BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231205' +BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231207' DATAFLOW_CONTAINER_IMAGE_REPOSITORY = 'gcr.io/cloud-dataflow/v1beta3'
(beam) branch tvalentyn-patch-3 created (now d9906ea0938)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git at d9906ea0938 Update names.py This branch includes the following new commits: new d9906ea0938 Update names.py The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) branch master updated (5becfb8ed43 -> 31822734526)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 5becfb8ed43 Archive JUnit test result on timeout (#29667) add 31822734526 Add a pipeline update compatibility version option. (#29140) No new revisions were added by this update. Summary of changes: .../job_management/v1/beam_expansion_api.proto | 5 +++ .../beam/runners/core/construction/External.java | 1 + .../apache/beam/sdk/options/StreamingOptions.java | 11 ++ .../sdk/expansion/service/ExpansionService.java| 15 +++- .../sdk/expansion/service/ExpansionServerTest.java | 15 ++-- .../python/apache_beam/options/pipeline_options.py | 43 ++ sdks/python/apache_beam/pipeline_test.py | 8 +++- .../runners/portability/expansion_service.py | 15 +++- sdks/python/apache_beam/transforms/external.py | 3 +- 9 files changed, 107 insertions(+), 9 deletions(-)
(beam) branch master updated (be9155205dd -> 5becfb8ed43)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from be9155205dd Merge pull request #29641 from lqiu96/update-libraries-bom-26.28.0 add 5becfb8ed43 Archive JUnit test result on timeout (#29667) No new revisions were added by this update. Summary of changes: .github/workflows/beam_LoadTests_Java_CoGBK_Dataflow_Streaming.yml | 2 +- .github/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml | 2 +- .github/workflows/beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml | 2 +- .github/workflows/beam_PerformanceTests_BigQueryIO_Streaming_Java.yml | 2 +- .github/workflows/beam_PerformanceTests_SQLBigQueryIO_Batch_Java.yml| 2 +- .github/workflows/beam_PostCommit_Java.yml | 2 +- .github/workflows/beam_PostCommit_Java_Avro_Versions.yml| 2 +- .github/workflows/beam_PostCommit_Java_BigQueryEarlyRollout.yml | 2 +- .github/workflows/beam_PostCommit_Java_DataflowV1.yml | 2 +- .github/workflows/beam_PostCommit_Java_DataflowV2.yml | 2 +- .github/workflows/beam_PostCommit_Java_Examples_Dataflow.yml| 2 +- .github/workflows/beam_PostCommit_Java_Examples_Dataflow_ARM.yml| 2 +- .github/workflows/beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 +- .github/workflows/beam_PostCommit_Java_Examples_Dataflow_V2.yml | 2 +- .github/workflows/beam_PostCommit_Java_Examples_Dataflow_V2_Java.yml| 2 +- .github/workflows/beam_PostCommit_Java_Examples_Direct.yml | 2 +- .github/workflows/beam_PostCommit_Java_Examples_Flink.yml | 2 +- .github/workflows/beam_PostCommit_Java_Examples_Spark.yml | 2 +- .github/workflows/beam_PostCommit_Java_Hadoop_Versions.yml | 2 +- .github/workflows/beam_PostCommit_Java_IO_Performance_Tests.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Dataflow_Java11.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Dataflow_Java17.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Direct_Java11.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Direct_Java17.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml | 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Flink_Java11.yml| 2 +- .github/workflows/beam_PostCommit_Java_Jpms_Spark_Java11.yml| 2 +- .github/workflows/beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_PVR_Samza.yml| 2 +- .github/workflows/beam_PostCommit_Java_PVR_Spark3_Streaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 +- .github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Direct_JavaVersions.yml| 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Flink.yml| 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Samza.yml| 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Spark.yml| 2 +- .../beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_Twister2.yml | 2 +- .github/workflows/beam_PostCommit_Java_ValidatesRunner_ULR.yml | 2 +- .github/workflows/beam_PostCommit_SQL.yml | 2 +- .github/workflows/beam_PostCommit_XVR_Direct.yml| 2 +- .github/workflows/beam_PostCommit_XVR_Flink.yml | 2 +- .github/workflows/beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 2 +- .github/workflows/beam_PostCommit_XVR_JavaUsingPython_Dataflow.yml | 2 +- .github/workflows/beam_PostCommit_XVR_Samza.yml | 2 +- .github/workflows/beam_PostCommit_XVR_Spark3.yml| 2 +- .github/workflows/beam_PreCommit_ItFramework.yml| 2 +- .github/workflows/beam_PreCommit_Java.yml | 2 +- .../workflows/beam_PreCommit_Java_Amazon-Web-Services2_IO_Direct.yml| 2 +- .github/workflows/beam_PreCommit_Java_Amazon-Web-
(beam) branch master updated (b426dfaf5ad -> be9155205dd)
This is an automated email from the ASF dual-hosted git repository. bce pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from b426dfaf5ad Add the ability to generate markdown documentation from a set of providers. (#29639) new 74308c39b8b Update to Libraries-Bom 26.28.0 new 6e50c1bbc88 Update libraries-bom java url version to 26.28.0 new be9155205dd Merge pull request #29641 from lqiu96/update-libraries-bom-26.28.0 The 41198 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 10 +- sdks/java/container/license_scripts/dep_urls_java.yaml | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-)
(beam) branch master updated (e15522f007f -> b426dfaf5ad)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from e15522f007f Merge pull request #29648 Fix for signature exceptions in ptransform_fn. add b426dfaf5ad Add the ability to generate markdown documentation from a set of providers. (#29639) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/transforms/external.py | 35 ++-- sdks/python/apache_beam/typehints/schemas.py | 14 ++ sdks/python/apache_beam/yaml/generate_yaml_docs.py | 188 + sdks/python/apache_beam/yaml/standard_io.yaml | 3 +- sdks/python/apache_beam/yaml/yaml_io.py| 2 +- sdks/python/apache_beam/yaml/yaml_mapping.py | 22 +++ sdks/python/apache_beam/yaml/yaml_provider.py | 97 ++- sdks/python/setup.py | 4 + 8 files changed, 305 insertions(+), 60 deletions(-) create mode 100644 sdks/python/apache_beam/yaml/generate_yaml_docs.py mode change 100644 => 100755 sdks/python/apache_beam/yaml/yaml_provider.py
(beam) branch master updated (90dd93f5241 -> e15522f007f)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 90dd93f5241 Disable failing mypy check in yaml_io.py (#29666) add dab5e86b912 Fix for signature exceptions in ptransform_fn. add 07ebdc6a6f5 Guard entire signature setting. add e15522f007f Merge pull request #29648 Fix for signature exceptions in ptransform_fn. No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/transforms/ptransform.py | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-)
(beam) branch master updated: Disable failing mypy check in yaml_io.py (#29666)
This is an automated email from the ASF dual-hosted git repository.
jrmccluskey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 90dd93f5241 Disable failing mypy check in yaml_io.py (#29666)
90dd93f5241 is described below
commit 90dd93f5241284da2e49c818af03e98b5132d30a
Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com>
AuthorDate: Thu Dec 7 14:12:41 2023 -0500
Disable failing mypy check in yaml_io.py (#29666)
* Disable failing pylint check in yaml_io.py
* Correct suppression
* Fine-grained disable
---
sdks/python/apache_beam/yaml/yaml_io.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/sdks/python/apache_beam/yaml/yaml_io.py
b/sdks/python/apache_beam/yaml/yaml_io.py
index bf4009719b8..b8abc47056c 100644
--- a/sdks/python/apache_beam/yaml/yaml_io.py
+++ b/sdks/python/apache_beam/yaml/yaml_io.py
@@ -152,10 +152,11 @@ def _create_parser(
elif format == 'avro':
beam_schema = avroio.avro_schema_to_beam_schema(schema)
covert_to_row = avroio.avro_dict_to_beam_row(schema, beam_schema)
+# pylint: disable=line-too-long
return (
beam_schema,
lambda record: covert_to_row(
-fastavro.schemaless_reader(io.BytesIO(record), schema)))
+fastavro.schemaless_reader(io.BytesIO(record), schema))) # type:
ignore[call-arg]
else:
raise ValueError(f'Unknown format: {format}')
(beam) branch master updated (51d71c0c43e -> 174e49dc6aa)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 51d71c0c43e Remove experimental note about streaming. (#29652) add 174e49dc6aa Add cloud profiler dependency to 3.11 containers. (#29651) No new revisions were added by this update. Summary of changes: .../container/base_image_requirements_manual.txt | 3 +-- .../container/py310/base_image_requirements.txt| 24 ++-- .../container/py311/base_image_requirements.txt| 25 - .../container/py38/base_image_requirements.txt | 26 +++--- .../container/py39/base_image_requirements.txt | 26 +++--- 5 files changed, 53 insertions(+), 51 deletions(-)
(beam) branch master updated (33a122cd2ac -> 51d71c0c43e)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 33a122cd2ac Add additional log messages and javadoc around reads (#29640) add 51d71c0c43e Remove experimental note about streaming. (#29652) No new revisions were added by this update. Summary of changes: website/www/site/content/en/documentation/runners/direct.md | 4 website/www/site/content/en/documentation/sdks/python.md| 2 +- 2 files changed, 5 insertions(+), 1 deletion(-)
(beam) branch master updated: Add additional log messages and javadoc around reads (#29640)
This is an automated email from the ASF dual-hosted git repository.
johncasey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 33a122cd2ac Add additional log messages and javadoc around reads
(#29640)
33a122cd2ac is described below
commit 33a122cd2acdb8b0ce82cff2da274b94594a21d2
Author: Niel Markwick
AuthorDate: Thu Dec 7 19:41:08 2023 +0100
Add additional log messages and javadoc around reads (#29640)
---
.../java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java | 3 ++-
.../java/org/apache/beam/sdk/io/gcp/spanner/NaiveSpannerRead.java | 5 +
.../main/java/org/apache/beam/sdk/io/gcp/spanner/ReadOperation.java | 5 +
.../src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 6 ++
4 files changed, 18 insertions(+), 1 deletion(-)
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java
index 5e5c215fdeb..1e25d1e72ab 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/BatchSpannerRead.java
@@ -246,7 +246,8 @@ abstract class BatchSpannerRead
}
} catch (SpannerException e) {
serviceCallMetric.call(e.getErrorCode().getGrpcStatusCode().toString());
-LOG.error("Error while processing element", e);
+LOG.error(
+"Error while reading partition for operation: " +
op.getReadOperation().toString(), e);
throw (e);
}
serviceCallMetric.call("ok");
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/NaiveSpannerRead.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/NaiveSpannerRead.java
index 45cc331657f..2cc07b88898 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/NaiveSpannerRead.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/NaiveSpannerRead.java
@@ -34,6 +34,8 @@ import org.apache.beam.sdk.values.PCollection;
import org.apache.beam.sdk.values.PCollectionView;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.annotations.VisibleForTesting;
import org.checkerframework.checker.nullness.qual.Nullable;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/** A naive version of Spanner read that doesn't use the Batch API. */
@VisibleForTesting
@@ -44,6 +46,8 @@ import org.checkerframework.checker.nullness.qual.Nullable;
abstract class NaiveSpannerRead
extends PTransform, PCollection> {
+ private static final Logger LOG =
LoggerFactory.getLogger(NaiveSpannerRead.class);
+
public static NaiveSpannerRead create(
SpannerConfig spannerConfig,
PCollectionView txView,
@@ -109,6 +113,7 @@ abstract class NaiveSpannerRead
}
} catch (SpannerException e) {
serviceCallMetric.call(e.getErrorCode().getGrpcStatusCode().toString());
+LOG.error("Error while reading operation: " + op.toString(), e);
throw (e);
}
serviceCallMetric.call("ok");
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/ReadOperation.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/ReadOperation.java
index 959b0cb5502..4cbfe12b231 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/ReadOperation.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/ReadOperation.java
@@ -69,6 +69,11 @@ public abstract class ReadOperation implements Serializable {
abstract Builder setKeySet(KeySet keySet);
+/**
+ * Note: {@link PartitionOptions} are currently ignored. See https://cloud.google.com/spanner/docs/reference/rpc/google.spanner.v1#google.spanner.v1.PartitionOptions";>
+ * PartitionOptions in RPC documents
+ */
abstract Builder setPartitionOptions(PartitionOptions partitionOptions);
abstract ReadOperation build();
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
index b6ec8097a5f..6db79ab69b4 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
@@ -852,6 +852,11 @@ public class SpannerIO {
return withReadOperation(getReadOperation().withIndex(index));
}
+/**
+
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 3989573a378 Updating config from bot
3989573a378 is described below
commit 3989573a37810a4d25799fdd9e752e44aaf5e9ce
Author: github-actions
AuthorDate: Thu Dec 7 18:06:33 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29662.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29662.json
b/scripts/ci/pr-bot/state/pr-state/pr-29662.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29662.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated: Add per-worker keys to stateful BatchElements, plumb through batching options to all model handlers (#29642)
This is an automated email from the ASF dual-hosted git repository. jrmccluskey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 84d070bae9b Add per-worker keys to stateful BatchElements, plumb through batching options to all model handlers (#29642) 84d070bae9b is described below commit 84d070bae9b59160d4d88aa2fab9d98b4ed2589f Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com> AuthorDate: Thu Dec 7 12:40:26 2023 -0500 Add per-worker keys to stateful BatchElements, plumb through batching options to all model handlers (#29642) * Key streaming BatchElements bundles per-worker * Plumb through max_batch_duration_secs support * Linting * Tag shared handle --- .../ml/inference/huggingface_inference.py | 15 +++ .../apache_beam/ml/inference/onnx_inference.py | 18 ++ .../apache_beam/ml/inference/pytorch_inference.py | 10 .../apache_beam/ml/inference/sklearn_inference.py | 10 .../ml/inference/tensorflow_inference.py | 10 .../apache_beam/ml/inference/tensorrt_inference.py | 7 +- .../ml/inference/vertex_ai_inference.py| 22 +++- .../apache_beam/ml/inference/xgboost_inference.py | 21 sdks/python/apache_beam/transforms/util.py | 29 +- 9 files changed, 139 insertions(+), 3 deletions(-) diff --git a/sdks/python/apache_beam/ml/inference/huggingface_inference.py b/sdks/python/apache_beam/ml/inference/huggingface_inference.py index 878d7bfc9cf..1bc92c462c9 100644 --- a/sdks/python/apache_beam/ml/inference/huggingface_inference.py +++ b/sdks/python/apache_beam/ml/inference/huggingface_inference.py @@ -224,6 +224,7 @@ class HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str, inference_args: Optional[Dict[str, Any]] = None, min_batch_size: Optional[int] = None, max_batch_size: Optional[int] = None, + max_batch_duration_secs: Optional[int] = None, large_model: bool = False, **kwargs): """ @@ -255,6 +256,8 @@ class HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str, Defaults to None. min_batch_size: the minimum batch size to use when batching inputs. max_batch_size: the maximum batch size to use when batching inputs. + max_batch_duration_secs: the maximum amount of time to buffer a batch +before emitting; used in streaming contexts. large_model: set to true if your model is large enough to run into memory pressure if you load multiple copies. Given a model that consumes N memory and a machine with W cores and M memory, you should @@ -277,6 +280,8 @@ class HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str, self._batching_kwargs["min_batch_size"] = min_batch_size if max_batch_size is not None: self._batching_kwargs["max_batch_size"] = max_batch_size +if max_batch_duration_secs is not None: + self._batching_kwargs["max_batch_duration_secs"] = max_batch_duration_secs self._large_model = large_model self._framework = framework @@ -405,6 +410,7 @@ class HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor], inference_args: Optional[Dict[str, Any]] = None, min_batch_size: Optional[int] = None, max_batch_size: Optional[int] = None, + max_batch_duration_secs: Optional[int] = None, large_model: bool = False, **kwargs): """ @@ -436,6 +442,8 @@ class HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor], Defaults to None. min_batch_size: the minimum batch size to use when batching inputs. max_batch_size: the maximum batch size to use when batching inputs. + max_batch_duration_secs: the maximum amount of time to buffer a batch +before emitting; used in streaming contexts. large_model: set to true if your model is large enough to run into memory pressure if you load multiple copies. Given a model that consumes N memory and a machine with W cores and M memory, you should @@ -458,6 +466,8 @@ class HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor], self._batching_kwargs["min_batch_size"] = min_batch_size if max_batch_size is not None: self._batching_kwargs["max_batch_size"] = max_batch_size +if max_batch_duration_secs is not None: + self._batching_kwargs["max_batch_duration_secs"] = max_batch_duration_secs self._large_model = large_model self._framework = "" @@ -579,6 +589,7 @@ class HuggingFacePipelineModelHandler(ModelHandler[str, inference_args: Optional[Dict[str, Any]] = None, min_batch_size: Optional[int] = None, max_batch_size: Optional[int] = None, + max_batch_duration_secs: Optional[int] = None,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 00dc642584c Updating config from bot
00dc642584c is described below
commit 00dc642584cc1fcf8e5036589f99a470287cf3a1
Author: github-actions
AuthorDate: Thu Dec 7 16:37:40 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index f6788517130..68aa7dbe142 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -2,7 +2,7 @@
"label": "io",
"dateOfLastReviewAssignment": {
"chamikaramj": 1701459290723,
-"johnjcasey": 1701390112035,
+"johnjcasey": 1701967055102,
"pabloem": 1691787951165,
"Abacn": 1701761214632,
"ahmedabu98": 1701912589475,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b665d6ee8d4 Updating config from bot
b665d6ee8d4 is described below
commit b665d6ee8d48b37a6d85a726296771b7bf4e8e52
Author: github-actions
AuthorDate: Thu Dec 7 16:37:38 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index f01127fe34c..9837f9b3697 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -2,7 +2,7 @@
"label": "java",
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
-"kennknowles": 1701432903511,
+"kennknowles": 1701967055102,
"robertwb": 1701912589475,
"kileys": 1674428678843,
"apilloud": 1678822446183,
(beam) branch master updated (d2d19ef5dfd -> 8d9fd02af5e)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from d2d19ef5dfd Bugfix/extend java precommit timeout (#29664) add 8d9fd02af5e Bump Java VR interval and shutdown remaining Dataflow VR Jenkins PostCommit (#29647) No new revisions were added by this update. Summary of changes: ...am_PostCommit_Java_ValidatesRunner_Dataflow.yml | 2 +- ..._Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 +- ...mit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 +- ...PostCommit_Java_ValidatesRunner_Dataflow.groovy | 51 -- ...mit_Java_ValidatesRunner_Dataflow_Java11.groovy | 60 -- ...mit_Java_ValidatesRunner_Dataflow_Java17.groovy | 57 ..._Java_ValidatesRunner_Dataflow_Streaming.groovy | 50 -- ...tCommit_Java_ValidatesRunner_Dataflow_V2.groovy | 51 -- ...va_ValidatesRunner_Dataflow_V2_Streaming.groovy | 51 -- ...stCommit_Python_ValidatesRunner_Dataflow.groovy | 45 10 files changed, 3 insertions(+), 368 deletions(-) delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_Java11.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_Java17.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_V2.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.groovy delete mode 100644 .test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow.groovy
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7fce2ba7e4c Updating config from bot 7fce2ba7e4c is described below commit 7fce2ba7e4c76b4fc26bccc2bcb0cfcea6dc0362 Author: github-actions AuthorDate: Thu Dec 7 16:06:18 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29585.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29585.json b/scripts/ci/pr-bot/state/pr-state/pr-29585.json index 55cd2385442..2403b9b4513 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29585.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29585.json @@ -7,5 +7,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 803543b9454 Updating config from bot 803543b9454 is described below commit 803543b94548bd9ac8544782e2c8478a27469f88 Author: github-actions AuthorDate: Thu Dec 7 16:06:08 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29651.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29651.json b/scripts/ci/pr-bot/state/pr-state/pr-29651.json index 6272b4e797d..7220f359527 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29651.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29651.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 28b4efc0ecc Updating config from bot 28b4efc0ecc is described below commit 28b4efc0ecc728124bd317485970264645468eca Author: github-actions AuthorDate: Thu Dec 7 15:34:43 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29647.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29647.json b/scripts/ci/pr-bot/state/pr-state/pr-29647.json index 8a92900947e..c82cdd128d1 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29647.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29647.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch master updated: Bugfix/extend java precommit timeout (#29664)
This is an automated email from the ASF dual-hosted git repository.
johncasey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new d2d19ef5dfd Bugfix/extend java precommit timeout (#29664)
d2d19ef5dfd is described below
commit d2d19ef5dfd8d2b0967d178312e11d5c00e12a7b
Author: johnjcasey <95318300+johnjca...@users.noreply.github.com>
AuthorDate: Thu Dec 7 10:18:05 2023 -0500
Bugfix/extend java precommit timeout (#29664)
* Update 2.50 release notes to include new Kafka topicPattern feature
* Create groovy class for io performance tests
Create gradle task and github actions config for GCS using this.
* delete unnecessary class
* fix env call
* fix call to gradle
* run on hosted runner for testing
* add additional checkout
* add destination for triggered tests
* move env variables to correct location
* try uploading against separate dataset
* try without a user
* update branch checkout, try to view the failure log
* run on failure
* update to use correct BigQuery instance
* convert to matrix
* add result reporting
* add failure clause
* remove failure clause, update to run on self-hosted
* address comments, clean up build
* clarify branching
* extend timeout
* Update .github/workflows/beam_PreCommit_Java.yml
Co-authored-by: Danny McCormick
-
Co-authored-by: Danny McCormick
---
.github/workflows/beam_PreCommit_Java.yml | 1 -
1 file changed, 1 deletion(-)
diff --git a/.github/workflows/beam_PreCommit_Java.yml
b/.github/workflows/beam_PreCommit_Java.yml
index b4aa233d522..3254ac2dc97 100644
--- a/.github/workflows/beam_PreCommit_Java.yml
+++ b/.github/workflows/beam_PreCommit_Java.yml
@@ -155,7 +155,6 @@ env:
jobs:
beam_PreCommit_Java:
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
-timeout-minutes: 180
runs-on: [self-hosted, ubuntu-20.04, main]
strategy:
fail-fast: false
(beam) branch dependabot/go_modules/sdks/golang.org/x/oauth2-0.15.0 deleted (was 0fa8fcb7516)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/golang.org/x/oauth2-0.15.0 in repository https://gitbox.apache.org/repos/asf/beam.git was 0fa8fcb7516 Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated (5d11c20cd4e -> e620b5aac97)
This is an automated email from the ASF dual-hosted git repository. jrmccluskey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 5d11c20cd4e Add Error Handling to Kafka IO (#29546) add e620b5aac97 Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 in /sdks (#29630) No new revisions were added by this update. Summary of changes: sdks/go.mod | 2 +- sdks/go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
(beam) branch master updated: Add Error Handling to Kafka IO (#29546)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 5d11c20cd4e Add Error Handling to Kafka IO (#29546) 5d11c20cd4e is described below commit 5d11c20cd4e5a0df132f4cd2cff0d880a0d75323 Author: johnjcasey <95318300+johnjca...@users.noreply.github.com> AuthorDate: Thu Dec 7 10:11:16 2023 -0500 Add Error Handling to Kafka IO (#29546) * Update 2.50 release notes to include new Kafka topicPattern feature * Create groovy class for io performance tests Create gradle task and github actions config for GCS using this. * delete unnecessary class * fix env call * fix call to gradle * run on hosted runner for testing * add additional checkout * add destination for triggered tests * move env variables to correct location * try uploading against separate dataset * try without a user * update branch checkout, try to view the failure log * run on failure * update to use correct BigQuery instance * convert to matrix * add result reporting * add failure clause * remove failure clause, update to run on self-hosted * address comments, clean up build * clarify branching * Add error handling base implementation & test DLQ enabled class * Add test cases * apply spotless * Fix Checkstyles * Fix Checkstyles * make DLH serializable * rename dead letter to bad record * make DLH serializable * Change bad record router name, and use multioutputreceiver instead of process context * Refactor BadRecord to be nested * clean up checkstyle * Update error handler test * Add metrics for counting error records, and for measuring feature usage * apply spotless * fix checkstyle * make metric reporting static * spotless * Rework annotations to be an explicit label on a PTransform, instead of using java annotations * fix checkstyle * Address comments * Address comments * Fix test cases, spotless * remove flatting without error collections * fix nullness * spotless + encoding issues * spotless * throw error when error handler isn't used * add concrete bad record error handler class * spotless, fix test category * fix checkstyle * clean up comments * fix test case * initial wiring of error handler into KafkaIO Read * remove "failing transform" field on bad record, add note to CHANGES.md * fix failing test cases * fix failing test cases * apply spotless * Add tests * Add tests * fix test case * add documentation * wire error handler into kafka write * fix failing test case * Add tests for writing to kafka with exception handling * fix sdf testing * fix sdf testing * spotless * deflake tests * add error handling to kafka streaming example update error handler to be serializable to support using it as a member of an auto-value based PTransform * apply final comments * apply final comments * apply final comments * add line to CHANGES.md * fix spotless * fix checkstyle * make sink transform static for serialization * spotless * fix typo * fix typo * fix spotbugs --- CHANGES.md | 1 + .../beam/gradle/kafka/KafkaTestUtilities.groovy| 3 +- .../org/apache/beam/examples/KafkaStreaming.java | 67 ++-- .../sdk/transforms/errorhandling/ErrorHandler.java | 32 ++-- sdks/java/io/kafka/kafka-01103/build.gradle| 1 + sdks/java/io/kafka/kafka-100/build.gradle | 3 +- sdks/java/io/kafka/kafka-111/build.gradle | 1 + sdks/java/io/kafka/kafka-201/build.gradle | 1 + sdks/java/io/kafka/kafka-211/build.gradle | 1 + sdks/java/io/kafka/kafka-222/build.gradle | 1 + sdks/java/io/kafka/kafka-231/build.gradle | 1 + sdks/java/io/kafka/kafka-241/build.gradle | 1 + sdks/java/io/kafka/kafka-251/build.gradle | 1 + sdks/java/io/kafka/kafka-integration-test.gradle | 2 +- .../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 127 ++-- .../KafkaIOReadImplementationCompatibility.java| 1 + .../org/apache/beam/sdk/io/kafka/KafkaWriter.java | 46 -- .../beam/sdk/io/kafka/ReadFromKafkaDoFn.java | 110 +- .../beam/sdk/io/kafka/KafkaIOExternalTest.java | 8 +- .
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 70417b67a73 Updating config from bot
70417b67a73 is described below
commit 70417b67a735367f0bbcea156936994d532be14f
Author: github-actions
AuthorDate: Thu Dec 7 15:06:17 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29630.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29630.json
b/scripts/ci/pr-bot/state/pr-state/pr-29630.json
index 242a48d7d3b..1c2c2e31975 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29630.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29630.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new af66866ba77 Updating config from bot
af66866ba77 is described below
commit af66866ba778254a2857b3b6a35ad6e5bb619908
Author: github-actions
AuthorDate: Thu Dec 7 15:06:19 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index d9f44bce977..69202bdb292 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1701927269486,
-"jrmccluskey": 1701839150581,
+"jrmccluskey": 1701961576077,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1701927265858
(beam) branch dependabot/github_actions/actions/setup-python-5 updated (c69eebd191d -> b0bcf313614)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-python-5 in repository https://gitbox.apache.org/repos/asf/beam.git discard c69eebd191d Bump actions/setup-python from 4 to 5 add 70926f5da78 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657) add ee5e782c525 Bump actions/setup-go from 4 to 5 (#29653) add b0bcf313614 Bump actions/setup-python from 4 to 5 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (c69eebd191d) \ N -- N -- N refs/heads/dependabot/github_actions/actions/setup-python-5 (b0bcf313614) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .github/workflows/go_tests.yml | 2 +- .github/workflows/local_env_tests.yml | 4 ++-- .github/workflows/python_tests.yml | 2 +- .github/workflows/tour_of_beam_backend.yml | 2 +- sdks/go.mod| 2 +- sdks/go.sum| 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
(beam) branch dependabot/github_actions/actions/setup-go-5 deleted (was 88274a4da92)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-go-5 in repository https://gitbox.apache.org/repos/asf/beam.git was 88274a4da92 Bump actions/setup-go from 4 to 5 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Bump actions/setup-go from 4 to 5 (#29653)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new ee5e782c525 Bump actions/setup-go from 4 to 5 (#29653)
ee5e782c525 is described below
commit ee5e782c5252efa8259bd4315b9edefef109e02a
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 7 09:54:26 2023 -0500
Bump actions/setup-go from 4 to 5 (#29653)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot]
Co-authored-by: dependabot[bot]
<49699333+dependabot[bot]@users.noreply.github.com>
---
.github/workflows/go_tests.yml | 2 +-
.github/workflows/local_env_tests.yml | 4 ++--
.github/workflows/python_tests.yml | 2 +-
.github/workflows/tour_of_beam_backend.yml | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/go_tests.yml b/.github/workflows/go_tests.yml
index 02947eff5ca..2d180afc79e 100644
--- a/.github/workflows/go_tests.yml
+++ b/.github/workflows/go_tests.yml
@@ -43,7 +43,7 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 2
- - uses: actions/setup-go@v4
+ - uses: actions/setup-go@v5
with:
go-version: '1.21'
cache-dependency-path: |
diff --git a/.github/workflows/local_env_tests.yml
b/.github/workflows/local_env_tests.yml
index 7dbff4feeb4..49d37298f1c 100644
--- a/.github/workflows/local_env_tests.yml
+++ b/.github/workflows/local_env_tests.yml
@@ -46,7 +46,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- - uses: actions/setup-go@v4
+ - uses: actions/setup-go@v5
with:
go-version: '1.21'
- uses: actions/setup-python@v4
@@ -64,7 +64,7 @@ jobs:
runs-on: macos-latest
steps:
- uses: actions/checkout@v4
- - uses: actions/setup-go@v4
+ - uses: actions/setup-go@v5
with:
go-version: '1.21'
- uses: actions/setup-python@v4
diff --git a/.github/workflows/python_tests.yml
b/.github/workflows/python_tests.yml
index 0309329e84e..483d4776aec 100644
--- a/.github/workflows/python_tests.yml
+++ b/.github/workflows/python_tests.yml
@@ -168,7 +168,7 @@ jobs:
with:
python-version: ${{ matrix.python }}
- name: Install go
-uses: actions/setup-go@v4
+uses: actions/setup-go@v5
with:
go-version: '1.21'
- name: Download source from artifacts
diff --git a/.github/workflows/tour_of_beam_backend.yml
b/.github/workflows/tour_of_beam_backend.yml
index 3585bd0e670..e3a016a4b5a 100644
--- a/.github/workflows/tour_of_beam_backend.yml
+++ b/.github/workflows/tour_of_beam_backend.yml
@@ -42,7 +42,7 @@ jobs:
working-directory: ./learning/tour-of-beam/backend
steps:
- uses: actions/checkout@v4
- - uses: actions/setup-go@v4
+ - uses: actions/setup-go@v5
with:
# pin to the biggest Go version supported by Cloud Functions runtime
go-version: '1.16'
(beam) branch dependabot/go_modules/sdks/golang.org/x/oauth2-0.15.0 updated (a9f825eea7a -> 0fa8fcb7516)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/golang.org/x/oauth2-0.15.0 in repository https://gitbox.apache.org/repos/asf/beam.git discard a9f825eea7a Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 in /sdks add dff9b107b3b fix whitespace check (#29634) add ee84aaf4430 Update dataflow container tag (#29635) add 9fa7298d1ad Shutdown most of Jenkins PostCommit (#29626) add 50e0751d11c Bump github.com/nats-io/nats-server/v2 from 2.10.5 to 2.10.6 in /sdks (#29629) add 9090b5b3843 Better introspection of PTransforms defined with @ptransform_fn. (#29589) add b566480cec6 fix: have a timeout on the wait when closing BigtableIO#Writer (#29548) add 643a07610d7 Add buildx install to postcommit_XVR_Go (#29623) add 3d8bb4a007f Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#29631) add cdbd77898a8 [yaml] fix yaml label appending add f710795e798 Merge pull request #29643 [yaml] Append rather than replace yaml label. add aef21959bf6 Split BigQuery DatasetService and WriteStreamService (#29604) add 54c18943bfb Bump google.golang.org/api from 0.152.0 to 0.153.0 in /sdks (#29628) add 70926f5da78 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657) add 0fa8fcb7516 Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 in /sdks This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a9f825eea7a) \ N -- N -- N refs/heads/dependabot/go_modules/sdks/golang.org/x/oauth2-0.15.0 (0fa8fcb7516) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .github/workflows/README.md| 42 +++ .../beam_PostCommit_XVR_GoUsingJava_Dataflow.yml | 7 +- .github/workflows/cut_release_branch.yml | 78 + .../jenkins/job_PostCommit_CommunityMetrics.groovy | 45 --- ...ommit_CrossLanguageValidatesRunner_Flink.groovy | 52 - ...ValidatesRunner_JavaUsingPython_Dataflow.groovy | 49 ...idatesRunner_PythonUsingJavaSQL_Dataflow.groovy | 49 ...ommit_CrossLanguageValidatesRunner_Samza.groovy | 52 - ...mmit_CrossLanguageValidatesRunner_Spark3.groovy | 51 .test-infra/jenkins/job_PostCommit_Go.groovy | 43 --- .../job_PostCommit_Go_ValidatesRunner_Flink.groovy | 38 -- .../job_PostCommit_Go_ValidatesRunner_Samza.groovy | 38 -- .../job_PostCommit_Go_ValidatesRunner_Spark.groovy | 38 -- .test-infra/jenkins/job_PostCommit_Java.groovy | 49 .../job_PostCommit_Java_Avro_Versions.groovy | 47 .../jenkins/job_PostCommit_Java_Dataflow.groovy| 48 ...PostCommit_Java_Dataflow_Examples_Java11.groovy | 46 ...PostCommit_Java_Dataflow_Examples_Java17.groovy | 46 ...job_PostCommit_Java_Examples_Dataflow_V2.groovy | 46 ...tCommit_Java_Examples_Dataflow_V2_Java11.groovy | 51 ...tCommit_Java_Examples_Dataflow_V2_Java17.groovy | 51 .../job_PostCommit_Java_Examples_Direct.groovy | 43 --- .../job_PostCommit_Java_Examples_Flink.groovy | 43 --- .../job_PostCommit_Java_Examples_Spark.groovy | 43 --- .../job_PostCommit_Java_Hadoop_Versions.groovy | 48 ...job_PostCommit_Java_Jpms_Dataflow_Java11.groovy | 49 ...job_PostCommit_Java_Jpms_Dataflow_Java17.groovy | 51 .../job_PostCommit_Java_Jpms_Direct_Java11.groovy | 49 .../job_PostCommit_Java_Jpms_Direct_Java17.groovy | 51 .../job_PostCommit_Java_Jpms_Flink_Java11.groovy | 49 .../job_PostCommit_Java_Jpms_Spark_Java11.groovy | 49 .../job_PostCommit_Java_Nexmark_Dataflow.groovy| 60 -- .../job_PostCommit_Java_Nexmark_Dataflow_V2.groovy | 67 --- ...stCommit_Java_Nexmark_Dataflow_V2_Java11.groovy | 67 --- ...stCommit_Java_Nexmark_Dataflow_V2_Java17.groovy | 67 --- .../job_PostCommit_Java_Nexmark_Direct.groovy | 54 - .../job_PostCommit_Java_Nexmark_Flink.groovy | 56 - .../job_PostCommit_Java_Nexmark_Spark.groovy | 63 -- ..._PortableValidatesRunner_Flink_Streaming.groovy | 50 ...ommit_Java_PortableValidatesRunner_Samza.groovy | 43 --- ...PortableValidatesRunner_Spark3_Streaming.groovy | 43 --- ...Java_PortableV
(beam-starter-go) branch main updated (ad212fa -> 291d696)
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git from ad212fa Bump github.com/apache/beam/sdks/v2 from 2.51.0 to 2.52.0 (#26) add c55a548 Bump actions/setup-go from 4 to 5 new 291d696 Merge pull request #27 from apache/dependabot/github_actions/actions/setup-go-5 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/test.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam-starter-go) 01/01: Merge pull request #27 from apache/dependabot/github_actions/actions/setup-go-5
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git commit 291d6961358359eb8bd48182b01dd559dce4e05f Merge: ad212fa c55a548 Author: Ritesh Ghorse AuthorDate: Thu Dec 7 09:40:50 2023 -0500 Merge pull request #27 from apache/dependabot/github_actions/actions/setup-go-5 Bump actions/setup-go from 4 to 5 .github/workflows/test.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam-starter-go) branch dependabot/github_actions/actions/setup-go-5 deleted (was c55a548)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-go-5 in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git was c55a548 Bump actions/setup-go from 4 to 5 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/go_modules/sdks/go.mongodb.org/mongo-driver-1.13.1 deleted (was 66ca3e00d89)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/go.mongodb.org/mongo-driver-1.13.1 in repository https://gitbox.apache.org/repos/asf/beam.git was 66ca3e00d89 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657)
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 70926f5da78 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657) 70926f5da78 is described below commit 70926f5da784e6f0ca1eebc64352e942a6e556c7 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Dec 7 09:40:04 2023 -0500 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657) Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.13.0 to 1.13.1. - [Release notes](https://github.com/mongodb/mongo-go-driver/releases) - [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.13.0...v1.13.1) --- updated-dependencies: - dependency-name: go.mongodb.org/mongo-driver dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- sdks/go.mod | 2 +- sdks/go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/sdks/go.mod b/sdks/go.mod index fb53e17b4a3..2060c3c0a50 100644 --- a/sdks/go.mod +++ b/sdks/go.mod @@ -53,7 +53,7 @@ require ( github.com/tetratelabs/wazero v1.5.0 github.com/xitongsys/parquet-go v1.6.2 github.com/xitongsys/parquet-go-source v0.0.0-20220315005136-aec0fe3e777c - go.mongodb.org/mongo-driver v1.13.0 + go.mongodb.org/mongo-driver v1.13.1 golang.org/x/net v0.19.0 golang.org/x/oauth2 v0.14.0 golang.org/x/sync v0.5.0 diff --git a/sdks/go.sum b/sdks/go.sum index 2cf0742086d..3c99d658727 100644 --- a/sdks/go.sum +++ b/sdks/go.sum @@ -477,8 +477,8 @@ github.com/zeebo/assert v1.3.0 h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ= github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0= github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA= go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= -go.mongodb.org/mongo-driver v1.13.0 h1:67DgFFjYOCMWdtTEmKFpV3ffWlFnh+CYZ8ZS/tXWUfY= -go.mongodb.org/mongo-driver v1.13.0/go.mod h1:/rGBTebI3XYboVmgz+Wv3Bcbl3aD0QF9zl6kDDw18rQ= +go.mongodb.org/mongo-driver v1.13.1 h1:YIc7HTYsKndGK4RFzJ3covLz1byri52x0IoMB0Pt/vk= +go.mongodb.org/mongo-driver v1.13.1/go.mod h1:wcDf1JBCXy2mOW0bWHwO/IOYqdca1MPCwDtFu/Z9+eo= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
(beam-starter-go) branch dependabot/github_actions/actions/setup-go-5 created (now c55a548)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-go-5 in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git at c55a548 Bump actions/setup-go from 4 to 5 No new revisions were added by this update.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b7fd67d8d0c Updating config from bot b7fd67d8d0c is described below commit b7fd67d8d0cfcdf5af0f9827bf0a73b76f0063e7 Author: github-actions AuthorDate: Thu Dec 7 08:20:20 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 5f0f478a726..f01127fe34c 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -7,7 +7,7 @@ "kileys": 1674428678843, "apilloud": 1678822446183, "Abacn": 1701459290723, -"bvolpato": 1701331009251, +"bvolpato": 1701937217789, "m-trieu": 1701761214632, "damondouglas": 1701692033312 }
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 784376ab66a Updating config from bot
784376ab66a is described below
commit 784376ab66a0e05bb3dbc1f74f3a9b2f69cc7b35
Author: github-actions
AuthorDate: Thu Dec 7 08:20:19 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29633.json | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29633.json
b/scripts/ci/pr-bot/state/pr-state/pr-29633.json
index 242a48d7d3b..700fd9146fb 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29633.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29633.json
@@ -1,7 +1,9 @@
{
"commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "reviewersAssignedForLabels": {
+"java": "bvolpato"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
