[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5d658aad50f Updating config from bot
5d658aad50f is described below
commit 5d658aad50fd29233dab082c1adddce642f22712
Author: github-actions
AuthorDate: Wed Aug 16 05:34:49 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28020.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28020.json
b/scripts/ci/pr-bot/state/pr-state/pr-28020.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28020.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new aae60343dc0 Updating config from bot
aae60343dc0 is described below
commit aae60343dc09d870e938f4d9f88e56a6c70b33ac
Author: github-actions
AuthorDate: Wed Aug 16 05:05:50 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28019.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28019.json
b/scripts/ci/pr-bot/state/pr-state/pr-28019.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28019.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch dependabot/gradle/com.github.spotbugs.snom-spotbugs-gradle-plugin-5.1.3 created (now 63c2834092b)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.github.spotbugs.snom-spotbugs-gradle-plugin-5.1.3 in repository https://gitbox.apache.org/repos/asf/beam.git at 63c2834092b Bump com.github.spotbugs.snom:spotbugs-gradle-plugin No new revisions were added by this update.
[beam] branch nightly-refs/heads/master updated (9c548b34c7d -> 1981cb0f6cc)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 9c548b34c7d pull out StreamPool/StreamData from WindmillServerStub
file. Organize streaming appliance files into their own directory. (#27593)
add 61ea132f2e4 Bump github.com/testcontainers/testcontainers-go in /sdks
(#28004)
add 767a2d72510 bump hcls api (#27997)
add e59f001fdb3 Feature/externalize dataflow xlang kafka tests (#27805)
add 9cff2c4d214 Re-enable firestore integration tests (revert #27267)
(#27743)
add 369265db4cd Fix the generated unique tag for Java and Python
multi-arch containers (#28003)
add 9701e5cb7c1 Remove legacy path for Go Dataflow post commits. (#27994)
add a2f8bf9cba0 Transform service test suite
add 618a14d1cd6 Merge pull request #27578: Transform service test suite
add 99710c8ce44 Add required dependency for Github Actions Java ARM Test
Suite. (#27996)
add 1981cb0f6cc [#27839] Write PipelineOptions to a file instead of an
environment variable. (#27842)
No new revisions were added by this update.
Summary of changes:
.../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 13 +-
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 9 +-
.test-infra/jenkins/README.md | 1 +
...Commit_Python_CrossLanguage_IO_Dataflow.groovy} | 11 +-
... job_PostCommit_TransformService_Direct.groovy} | 20 +--
build.gradle.kts | 13 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 135 -
release/src/main/scripts/jenkins_jobs.txt | 1 +
.../google-cloud-dataflow-java/arm/build.gradle| 5 +-
runners/google-cloud-dataflow-java/build.gradle| 4 -
sdks/go.mod| 4 +-
sdks/go.sum| 8 +-
sdks/go/test/build.gradle | 7 --
sdks/java/container/boot.go| 23 +++-
.../launcher/TransformServiceLauncher.java | 15 +--
.../io/external/xlang_kafkaio_it_test.py | 53 +++-
.../apache_beam/io/gcp/bigtableio_it_test.py | 2 +
sdks/python/pytest.ini | 1 +
sdks/python/scripts/run_transform_service.sh | 85 +
sdks/python/test-suites/dataflow/build.gradle | 6 +
sdks/python/test-suites/dataflow/common.gradle | 3 +-
sdks/python/test-suites/direct/xlang/build.gradle | 19 +++
sdks/python/test-suites/xlang/build.gradle | 13 ++
23 files changed, 380 insertions(+), 71 deletions(-)
copy
.test-infra/jenkins/{job_PostCommit_Python_CrossLanguage_Gcp_Dataflow.groovy =>
job_PostCommit_Python_CrossLanguage_IO_Dataflow.groovy} (76%)
copy
.test-infra/jenkins/{job_PostCommit_CrossLanguageValidatesRunner_Direct.groovy
=> job_PostCommit_TransformService_Direct.groovy} (63%)
create mode 100755 sdks/python/scripts/run_transform_service.sh
[beam] branch dependabot/go_modules/sdks/cloud.google.com/go/storage-1.32.0 created (now 7997efeb4cc)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/cloud.google.com/go/storage-1.32.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 7997efeb4cc Bump cloud.google.com/go/storage from 1.31.0 to 1.32.0 in /sdks No new revisions were added by this update.
[beam] branch asf-site updated: Publishing website 2023/08/16 04:17:40 at commit 5fbc699
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 537abd4243b Publishing website 2023/08/16 04:17:40 at commit 5fbc699 537abd4243b is described below commit 537abd4243b8f1f8b647bd5623468fc34113b050 Author: jenkins AuthorDate: Wed Aug 16 04:17:40 2023 + Publishing website 2023/08/16 04:17:40 at commit 5fbc699 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index e70f0df4e78..ab13921e115 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/case-studies/octo/2023-08-15T14:18:16-07:00/blog/beam-2.49.0/2023-08-15T14:18:16-07:00/categories/blog/2023-08-15T14:18:16-07:00/blog/2023-08-15T14:18:16-07:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/case-studies/octo/2023-08-15T19:22:10-07:00/blog/beam-2.49.0/2023-08-15T19:22:10-07:00/categories/blog/2023-08-15T19:22:10-07:00/blog/2023-08-15T19:22:10-07:00 [...] \ No newline at end of file
[beam] branch master updated: Add docker buildx to the release yml file to build multi-arch containers (#28018)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 5fbc699cfb5 Add docker buildx to the release yml file to build multi-arch containers (#28018) 5fbc699cfb5 is described below commit 5fbc699cfb5e0441796b85a5dc93dd1371b391b5 Author: Celeste Zeng <61256376+celeste-z...@users.noreply.github.com> AuthorDate: Tue Aug 15 19:22:10 2023 -0700 Add docker buildx to the release yml file to build multi-arch containers (#28018) * Add docker buildx to build multi-arch containers. * use v2 for buildx --- .github/workflows/build_release_candidate.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/build_release_candidate.yml b/.github/workflows/build_release_candidate.yml index 462b3e2bdad..7b304e9aa65 100644 --- a/.github/workflows/build_release_candidate.yml +++ b/.github/workflows/build_release_candidate.yml @@ -163,6 +163,8 @@ jobs: with: distribution: 'temurin' java-version: '8' + - name: Set up Docker Buildx +uses: docker/setup-buildx-action@v2 - name: Remove default github maven configuration # This step is a workaround to avoid a decryption issue of Beam's # net.linguica.gradle.maven.settings plugin and github's provided maven
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 52ceafcb6d0 Updating config from bot
52ceafcb6d0 is described below
commit 52ceafcb6d091c5f958656001a5df7edfd942b49
Author: github-actions
AuthorDate: Wed Aug 16 02:09:33 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 9e5ec935d2f..556cc661e0c 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1692092081029,
+"damccorm": 1692151771267,
"Abacn": 1692128093816
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c7b349f556e Updating config from bot
c7b349f556e is described below
commit c7b349f556e9d6a147354410d3337515695e40ed
Author: github-actions
AuthorDate: Wed Aug 16 02:09:32 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28018.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28018.json
b/scripts/ci/pr-bot/state/pr-state/pr-28018.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28018.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: [#27839] Write PipelineOptions to a file instead of an environment variable. (#27842)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 1981cb0f6cc [#27839] Write PipelineOptions to a file instead of an
environment variable. (#27842)
1981cb0f6cc is described below
commit 1981cb0f6cc62ee3a679e195d3bdd83da03ba6e7
Author: Robert Burke
AuthorDate: Tue Aug 15 18:08:51 2023 -0700
[#27839] Write PipelineOptions to a file instead of an environment
variable. (#27842)
* remove use of deprecated io/ioutil package.
* [#27839] Write pipeline options to a local file for SDK use.
* finish comment
* Use guave 32.1.2
* fix bad merges
* further merge fixes
* Fix write error handling.
-
Co-authored-by: lostluck <13907733+lostl...@users.noreply.github.com>
Co-authored-by: Yi Hu
---
sdks/java/container/boot.go | 23 ---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/sdks/java/container/boot.go b/sdks/java/container/boot.go
index 61d5a6a3faa..0e39d907f07 100644
--- a/sdks/java/container/boot.go
+++ b/sdks/java/container/boot.go
@@ -22,7 +22,6 @@ import (
"encoding/json"
"flag"
"fmt"
- "io/ioutil"
"log"
"os"
"path/filepath"
@@ -125,7 +124,9 @@ func main() {
// (3) Invoke the Java harness, preserving artifact ordering in
classpath.
os.Setenv("HARNESS_ID", *id)
- os.Setenv("PIPELINE_OPTIONS", options)
+ if err := makePipelineOptionsFile(options); err != nil {
+ logger.Fatalf(ctx, "Failed to load pipeline options to worker:
%v", err)
+ }
os.Setenv("LOGGING_API_SERVICE_DESCRIPTOR",
proto.MarshalTextString({Url: *loggingEndpoint}))
os.Setenv("CONTROL_API_SERVICE_DESCRIPTOR",
proto.MarshalTextString({Url: *controlEndpoint}))
os.Setenv("RUNNER_CAPABILITIES",
strings.Join(info.GetRunnerCapabilities(), " "))
@@ -245,6 +246,22 @@ func main() {
logger.Fatalf(ctx, "Java exited: %v", execx.Execute("java", args...))
}
+// makePipelineOptionsFile writes the pipeline options to a file.
+// Assumes the options string is JSON formatted.
+func makePipelineOptionsFile(options string) error {
+ fn := "pipeline_options.json"
+ f, err := os.Create(fn)
+ if err != nil {
+ return fmt.Errorf("unable to create %v: %w", fn, err)
+ }
+ defer f.Close()
+ if _, err := f.WriteString(options); err != nil {
+ return fmt.Errorf("error writing %v: %w", f.Name(), err)
+ }
+ os.Setenv("PIPELINE_OPTIONS_FILE", f.Name())
+ return nil
+}
+
// heapSizeLimit returns 80% of the runner limit, if provided. If not provided,
// it returns 70% of the physical memory on the machine. If it cannot determine
// that value, it returns 1GB. This is an imperfect heuristic. It aims to
@@ -327,7 +344,7 @@ func LoadMetaOptions(ctx context.Context, logger
*tools.Logger, dir string) ([]*
return nil
}
- content, err := ioutil.ReadFile(path)
+ content, err := os.ReadFile(path)
if err != nil {
return err
}
[beam] branch dependabot/github_actions/actions/upload-artifact-3 updated (385043b9ead -> 2d1879df3b3)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/upload-artifact-3
in repository https://gitbox.apache.org/repos/asf/beam.git
omit 385043b9ead Bump actions/upload-artifact from 2 to 3
add 9701e5cb7c1 Remove legacy path for Go Dataflow post commits. (#27994)
add a2f8bf9cba0 Transform service test suite
add 618a14d1cd6 Merge pull request #27578: Transform service test suite
add 99710c8ce44 Add required dependency for Github Actions Java ARM Test
Suite. (#27996)
add 2d1879df3b3 Bump actions/upload-artifact from 2 to 3
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (385043b9ead)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/upload-artifact-3 (2d1879df3b3)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 5 +-
... job_PostCommit_TransformService_Direct.groovy} | 20 ++--
build.gradle.kts | 13 +--
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 126 +
.../google-cloud-dataflow-java/arm/build.gradle| 5 +-
sdks/go/test/build.gradle | 7 --
.../launcher/TransformServiceLauncher.java | 15 +--
.../apache_beam/io/gcp/bigtableio_it_test.py | 2 +
sdks/python/pytest.ini | 1 +
sdks/python/scripts/run_transform_service.sh | 85 ++
sdks/python/test-suites/direct/xlang/build.gradle | 19
11 files changed, 260 insertions(+), 38 deletions(-)
copy
.test-infra/jenkins/{job_PostCommit_CrossLanguageValidatesRunner_Direct.groovy
=> job_PostCommit_TransformService_Direct.groovy} (63%)
create mode 100755 sdks/python/scripts/run_transform_service.sh
[beam] branch master updated (618a14d1cd6 -> 99710c8ce44)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 618a14d1cd6 Merge pull request #27578: Transform service test suite add 99710c8ce44 Add required dependency for Github Actions Java ARM Test Suite. (#27996) No new revisions were added by this update. Summary of changes: .github/workflows/beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 5 - runners/google-cloud-dataflow-java/arm/build.gradle | 5 +++-- 2 files changed, 7 insertions(+), 3 deletions(-)
[beam] branch master updated (9701e5cb7c1 -> 618a14d1cd6)
This is an automated email from the ASF dual-hosted git repository.
chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 9701e5cb7c1 Remove legacy path for Go Dataflow post commits. (#27994)
add a2f8bf9cba0 Transform service test suite
add 618a14d1cd6 Merge pull request #27578: Transform service test suite
No new revisions were added by this update.
Summary of changes:
... job_PostCommit_TransformService_Direct.groovy} | 20 ++--
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 126 +
.../launcher/TransformServiceLauncher.java | 15 +--
.../apache_beam/io/gcp/bigtableio_it_test.py | 2 +
sdks/python/pytest.ini | 1 +
sdks/python/scripts/run_transform_service.sh | 85 ++
sdks/python/test-suites/direct/xlang/build.gradle | 19
7 files changed, 252 insertions(+), 16 deletions(-)
copy
.test-infra/jenkins/{job_PostCommit_CrossLanguageValidatesRunner_Direct.groovy
=> job_PostCommit_TransformService_Direct.groovy} (63%)
create mode 100755 sdks/python/scripts/run_transform_service.sh
svn commit: r63469 - in /dev/beam/2.50.0: ./ apache-beam-2.50.0-source-release.zip apache-beam-2.50.0-source-release.zip.asc apache-beam-2.50.0-source-release.zip.sha512
Author: lostluck Date: Tue Aug 15 23:54:21 2023 New Revision: 63469 Log: Staging Java artifacts for Apache Beam 2.50.0 RC1 Added: dev/beam/2.50.0/ dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip (with props) dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.asc dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.sha512 Added: dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip == Binary file - no diff available. Propchange: dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.asc == --- dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.asc (added) +++ dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.asc Tue Aug 15 23:54:21 2023 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- + +iQJMBAABCgA2FiEEkTwzkqdwx4HtxN2r0gMW9xIhNCIFAmTcECoYHHByaXZhdGVA +YmVhbS5hcGFjaGUub3JnAAoJENIDFvcSITQiVsMP/ipbFYpstq/Aiuq21P7PMNQr +W2UUBRQNmmXvo387oL8nkD8Bsl0dhbDKmi27W7A/GG3hRiEoxrES8wHtWFfFk/cO +fBAyG7HiAkaHO9rhhOobyljlRKXMnstuR0tqGLlBLOwv1Z+7mufBVRNX/YPnCxdZ +Z3htc8tH/fFidpWYSZeAgfc43cP3c17laH6jBHeWYVYvYHHz8V+p8+l+RTVVrs9b +Ic/uRNo4F/j9mqHx8eu7b4fdJWdbyqs6vLKXP5Djq9sTzbRhI1wzGpwphmEhIm2S +XMU6w8wfzD5pkqm6ErMwb2H6kaYwOjT5GFFJV3srwKtn72BVoJbSizBOk6qEagqO +hGgwnE6dhjrOc1EAVvGaDo7PT61mPouruQvtBNH3HjNKO83cEdStfhub5eiKLxak +D3IAGcObjOjTD3ud1gXJGE5x7l64vwcnetbii8o8HqHOYKcwrhiIQnzvRI5SJXpd +ICpkPi05o4vTcob1JYqhBVo+EQhT68BtE/WA4338whNAb5DjzTOsADs52E7bQYa8 +P57ef5QkIcHfm391ebUllQxFKqo0+YHy+myvVQCXqV5TlS8Y42AKbyTHydgV/MFz +yGPaSyF+8mEl8UbGMxstIf17V7yB0uO+gTN0C3NDi9N+2dTDfH+5i4bERueTdDE/ +Khlj3QEZXLGSUwSTk+ie +=3SCt +-END PGP SIGNATURE- Added: dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.sha512 == --- dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.sha512 (added) +++ dev/beam/2.50.0/apache-beam-2.50.0-source-release.zip.sha512 Tue Aug 15 23:54:21 2023 @@ -0,0 +1 @@ +00d80c49d33e81b829c162a0e0dbb73eaa9a00c1b7e1fb45252ac6db16e730ec0087b6c43c0768b76b31d4a5f044d28dc2c004e91aea18f5579ed56740871efc apache-beam-2.50.0-source-release.zip
[beam] annotated tag v2.50.0-RC1 updated (dd6ec894871 -> 00e2e73d4c1)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to annotated tag v2.50.0-RC1 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag v2.50.0-RC1 was modified! *** from dd6ec894871 (commit) to 00e2e73d4c1 (tag) tagging dd6ec894871d5be77aab37f73b12db9c60bd88bd (commit) replaces jupyterlab-sidepanel-v3.0.0 by lostluck on Tue Aug 15 16:49:53 2023 -0700 - Log - v2.50.0-RC1 --- No new revisions were added by this update. Summary of changes:
[beam] annotated tag sdks/v2.50.0-RC1 updated (dd6ec894871 -> b864bbaec80)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to annotated tag sdks/v2.50.0-RC1 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag sdks/v2.50.0-RC1 was modified! *** from dd6ec894871 (commit) to b864bbaec80 (tag) tagging dd6ec894871d5be77aab37f73b12db9c60bd88bd (commit) replaces jupyterlab-sidepanel-v3.0.0 by lostluck on Tue Aug 15 16:49:53 2023 -0700 - Log - Go SDK v2.50.0-RC1 --- No new revisions were added by this update. Summary of changes:
[beam] branch release-2.50.0 updated: Cherry pick #27842 (#28015)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch release-2.50.0
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/release-2.50.0 by this push:
new c42daeeab23 Cherry pick #27842 (#28015)
c42daeeab23 is described below
commit c42daeeab23b194529e5476b8e2843fbea8bb17c
Author: Robert Burke
AuthorDate: Tue Aug 15 16:30:40 2023 -0700
Cherry pick #27842 (#28015)
Co-authored-by: lostluck <13907733+lostl...@users.noreply.github.com>
---
sdks/java/container/boot.go | 23 ---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/sdks/java/container/boot.go b/sdks/java/container/boot.go
index 61d5a6a3faa..0e39d907f07 100644
--- a/sdks/java/container/boot.go
+++ b/sdks/java/container/boot.go
@@ -22,7 +22,6 @@ import (
"encoding/json"
"flag"
"fmt"
- "io/ioutil"
"log"
"os"
"path/filepath"
@@ -125,7 +124,9 @@ func main() {
// (3) Invoke the Java harness, preserving artifact ordering in
classpath.
os.Setenv("HARNESS_ID", *id)
- os.Setenv("PIPELINE_OPTIONS", options)
+ if err := makePipelineOptionsFile(options); err != nil {
+ logger.Fatalf(ctx, "Failed to load pipeline options to worker:
%v", err)
+ }
os.Setenv("LOGGING_API_SERVICE_DESCRIPTOR",
proto.MarshalTextString({Url: *loggingEndpoint}))
os.Setenv("CONTROL_API_SERVICE_DESCRIPTOR",
proto.MarshalTextString({Url: *controlEndpoint}))
os.Setenv("RUNNER_CAPABILITIES",
strings.Join(info.GetRunnerCapabilities(), " "))
@@ -245,6 +246,22 @@ func main() {
logger.Fatalf(ctx, "Java exited: %v", execx.Execute("java", args...))
}
+// makePipelineOptionsFile writes the pipeline options to a file.
+// Assumes the options string is JSON formatted.
+func makePipelineOptionsFile(options string) error {
+ fn := "pipeline_options.json"
+ f, err := os.Create(fn)
+ if err != nil {
+ return fmt.Errorf("unable to create %v: %w", fn, err)
+ }
+ defer f.Close()
+ if _, err := f.WriteString(options); err != nil {
+ return fmt.Errorf("error writing %v: %w", f.Name(), err)
+ }
+ os.Setenv("PIPELINE_OPTIONS_FILE", f.Name())
+ return nil
+}
+
// heapSizeLimit returns 80% of the runner limit, if provided. If not provided,
// it returns 70% of the physical memory on the machine. If it cannot determine
// that value, it returns 1GB. This is an imperfect heuristic. It aims to
@@ -327,7 +344,7 @@ func LoadMetaOptions(ctx context.Context, logger
*tools.Logger, dir string) ([]*
return nil
}
- content, err := ioutil.ReadFile(path)
+ content, err := os.ReadFile(path)
if err != nil {
return err
}
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f0fd51f7961 Updating config from bot
f0fd51f7961 is described below
commit f0fd51f79613260e7bed0043f4bf083e7b80ec9f
Author: github-actions
AuthorDate: Tue Aug 15 22:34:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index b6b008b0e18..b53c4006992 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -1,7 +1,7 @@
{
"label": "io",
"dateOfLastReviewAssignment": {
-"chamikaramj": 1691519695784,
+"chamikaramj": 1692138891203,
"johnjcasey": 1691979002397,
"pabloem": 1691787951165,
"Abacn": 1691773563896,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d5782f731db Updating config from bot d5782f731db is described below commit d5782f731db209184485a7b71c02c9e59e2c63a9 Author: github-actions AuthorDate: Tue Aug 15 22:34:56 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 3ed7456985b..1ba91f3eb70 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -6,7 +6,7 @@ "robertwb": 1691979002396, "kileys": 1674428678843, "apilloud": 1678822446183, -"Abacn": 1691787951165, +"Abacn": 1692138891203, "bvolpato": 1692092081029 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d509283ae1c Updating config from bot d509283ae1c is described below commit d509283ae1c65a22a8fee1cac7e154497197881f Author: github-actions AuthorDate: Tue Aug 15 22:34:54 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 877c6f941de..63f36639c14 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -11,6 +11,6 @@ "damccorm": 1692101626737, "jrmccluskey": 1691786090445, "riteshghorse": 1691699674756, -"liferoad": 1691636755122 +"liferoad": 1692138891203 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 79d99c6481e Updating config from bot
79d99c6481e is described below
commit 79d99c6481ed818c64b7ac71d928b63baf378a11
Author: github-actions
AuthorDate: Tue Aug 15 22:34:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28015.json | 12
1 file changed, 12 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28015.json
b/scripts/ci/pr-bot/state/pr-state/pr-28015.json
new file mode 100644
index 000..6fd9fc17059
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28015.json
@@ -0,0 +1,12 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "liferoad",
+"java": "Abacn",
+"io": "chamikaramj"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 932a1aa865f Updating config from bot 932a1aa865f is described below commit 932a1aa865fecd147f7c7e2aaef62ba72032c417 Author: github-actions AuthorDate: Tue Aug 15 22:34:49 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 6c7cfa43dc9..fd259c1edd8 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -6,7 +6,7 @@ "chamikaramj": 1691789684434, "damccorm": 1691904879031, "johnjcasey": 1692122770100, -"jrmccluskey": 1691701532543, +"jrmccluskey": 1692138885856, "kennknowles": 1692131683677, "lostluck": 1691735695247, "riteshghorse": 1691706156992,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 2e5d3b8288d Updating config from bot
2e5d3b8288d is described below
commit 2e5d3b8288dc179f9b41538025cd1fcbf02e0f9b
Author: github-actions
AuthorDate: Tue Aug 15 22:34:48 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28016.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28016.json
b/scripts/ci/pr-bot/state/pr-state/pr-28016.json
new file mode 100644
index 000..cd352f44205
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28016.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch asf-site updated: Publishing website 2023/08/15 22:16:07 at commit 9701e5c
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new fe96ad0b4d1 Publishing website 2023/08/15 22:16:07 at commit 9701e5c fe96ad0b4d1 is described below commit fe96ad0b4d12a42fdc52586fcfd9b2d438a1f8e4 Author: jenkins AuthorDate: Tue Aug 15 22:16:07 2023 + Publishing website 2023/08/15 22:16:07 at commit 9701e5c --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 2566a2cfd8f..e70f0df4e78 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/case-studies/octo/2023-08-15T11:30:18-04:00/blog/beam-2.49.0/2023-08-15T11:30:18-04:00/categories/blog/2023-08-15T11:30:18-04:00/blog/2023-08-15T11:30:18-04:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/case-studies/octo/2023-08-15T14:18:16-07:00/blog/beam-2.49.0/2023-08-15T14:18:16-07:00/categories/blog/2023-08-15T14:18:16-07:00/blog/2023-08-15T14:18:16-07:00 [...] \ No newline at end of file
[beam] branch master updated (369265db4cd -> 9701e5cb7c1)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 369265db4cd Fix the generated unique tag for Java and Python multi-arch containers (#28003) add 9701e5cb7c1 Remove legacy path for Go Dataflow post commits. (#27994) No new revisions were added by this update. Summary of changes: build.gradle.kts | 13 + sdks/go/test/build.gradle | 7 --- 2 files changed, 1 insertion(+), 19 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 2a14cb65536 Updating config from bot
2a14cb65536 is described below
commit 2a14cb655368fc07b8459833813f1df8ad9e6e68
Author: github-actions
AuthorDate: Tue Aug 15 21:05:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28014.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28014.json
b/scripts/ci/pr-bot/state/pr-state/pr-28014.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28014.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 381d0f58808 Updating config from bot
381d0f58808 is described below
commit 381d0f58808f60a4f54dcd0c891b8b8c48e49b07
Author: github-actions
AuthorDate: Tue Aug 15 20:34:45 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28013.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28013.json
b/scripts/ci/pr-bot/state/pr-state/pr-28013.json
new file mode 100644
index 000..8ad90f2eea6
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28013.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "kennknowles"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new eca587fed9e Updating config from bot eca587fed9e is described below commit eca587fed9e99153b4ccbf64fccb05e8ba853244 Author: github-actions AuthorDate: Tue Aug 15 20:34:47 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 76fc6f10a9e..6c7cfa43dc9 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -7,7 +7,7 @@ "damccorm": 1691904879031, "johnjcasey": 1692122770100, "jrmccluskey": 1691701532543, -"kennknowles": 1691521550469, +"kennknowles": 1692131683677, "lostluck": 1691735695247, "riteshghorse": 1691706156992, "robertwb": 1691757659011,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new fec4ea3d0db Updating config from bot fec4ea3d0db is described below commit fec4ea3d0db44ba1a23080fb0a425c7c58ba9eac Author: github-actions AuthorDate: Tue Aug 15 20:05:54 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27996.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27996.json b/scripts/ci/pr-bot/state/pr-state/pr-27996.json index 9c51466b9ee..ebb27d68fb5 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27996.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27996.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 54c202d7d24 Updating config from bot
54c202d7d24 is described below
commit 54c202d7d24a0b74f5abf8ac6076ee480c9ec76a
Author: github-actions
AuthorDate: Tue Aug 15 19:34:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27996.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27996.json
b/scripts/ci/pr-bot/state/pr-state/pr-27996.json
index 242a48d7d3b..9c51466b9ee 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27996.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27996.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4848037fadb Updating config from bot
4848037fadb is described below
commit 4848037fadb83351973d2d02367d02ca92c90806
Author: github-actions
AuthorDate: Tue Aug 15 19:34:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 69ed548853c..9e5ec935d2f 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1692092081029,
-"Abacn": 1692077679834
+"Abacn": 1692128093816
}
}
\ No newline at end of file
[beam] branch dependabot/github_actions/actions/upload-artifact-3 updated (3754bd075d4 -> 385043b9ead)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/upload-artifact-3
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 3754bd075d4 Bump actions/upload-artifact from 2 to 3
add 61ea132f2e4 Bump github.com/testcontainers/testcontainers-go in /sdks
(#28004)
add 767a2d72510 bump hcls api (#27997)
add e59f001fdb3 Feature/externalize dataflow xlang kafka tests (#27805)
add 9cff2c4d214 Re-enable firestore integration tests (revert #27267)
(#27743)
add 369265db4cd Fix the generated unique tag for Java and Python
multi-arch containers (#28003)
add 385043b9ead Bump actions/upload-artifact from 2 to 3
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3754bd075d4)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/upload-artifact-3 (385043b9ead)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 8 ++--
...beam_Python_ValidatesContainer_Dataflow_ARM.yml | 9 ++--
.test-infra/jenkins/README.md | 1 +
...Commit_Python_CrossLanguage_IO_Dataflow.groovy} | 11 +++--
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 9 +++-
release/src/main/scripts/jenkins_jobs.txt | 1 +
runners/google-cloud-dataflow-java/build.gradle| 4 --
sdks/go.mod| 4 +-
sdks/go.sum| 8 ++--
.../io/external/xlang_kafkaio_it_test.py | 53 --
sdks/python/test-suites/dataflow/build.gradle | 6 +++
sdks/python/test-suites/dataflow/common.gradle | 3 +-
sdks/python/test-suites/xlang/build.gradle | 13 ++
13 files changed, 100 insertions(+), 30 deletions(-)
copy
.test-infra/jenkins/{job_PostCommit_Python_CrossLanguage_Gcp_Dataflow.groovy =>
job_PostCommit_Python_CrossLanguage_IO_Dataflow.groovy} (76%)
[beam] branch master updated (9cff2c4d214 -> 369265db4cd)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 9cff2c4d214 Re-enable firestore integration tests (revert #27267) (#27743) add 369265db4cd Fix the generated unique tag for Java and Python multi-arch containers (#28003) No new revisions were added by this update. Summary of changes: .github/workflows/beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 8 .../workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml| 9 + 2 files changed, 9 insertions(+), 8 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 937cc1bf2fd Updating config from bot 937cc1bf2fd is described below commit 937cc1bf2fde3b67557a12d334b89dc3a6a6d45b Author: github-actions AuthorDate: Tue Aug 15 18:06:13 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 2e3575fab18..76fc6f10a9e 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -5,7 +5,7 @@ "AnandInguva": 1691784361908, "chamikaramj": 1691789684434, "damccorm": 1691904879031, -"johnjcasey": 1691442350909, +"johnjcasey": 1692122770100, "jrmccluskey": 1691701532543, "kennknowles": 1691521550469, "lostluck": 1691735695247,
[beam] branch master updated (e59f001fdb3 -> 9cff2c4d214)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from e59f001fdb3 Feature/externalize dataflow xlang kafka tests (#27805) add 9cff2c4d214 Re-enable firestore integration tests (revert #27267) (#27743) No new revisions were added by this update. Summary of changes: runners/google-cloud-dataflow-java/build.gradle | 4 1 file changed, 4 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 55080469ed4 Updating config from bot
55080469ed4 is described below
commit 55080469ed4e0090e06fb4a219515acffe4b7894
Author: github-actions
AuthorDate: Tue Aug 15 18:06:12 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28010.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28010.json
b/scripts/ci/pr-bot/state/pr-state/pr-28010.json
new file mode 100644
index 000..b29414341ff
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28010.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "johnjcasey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 99d89976a0c Updating config from bot
99d89976a0c is described below
commit 99d89976a0c624672012cd7d4034cd3aa76de873
Author: github-actions
AuthorDate: Tue Aug 15 16:08:58 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28009.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28009.json
b/scripts/ci/pr-bot/state/pr-state/pr-28009.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28009.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Feature/externalize dataflow xlang kafka tests (#27805)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new e59f001fdb3 Feature/externalize dataflow xlang kafka tests (#27805) e59f001fdb3 is described below commit e59f001fdb3d5104cf7f82cdbe0b099e32c7b9c1 Author: johnjcasey <95318300+johnjca...@users.noreply.github.com> AuthorDate: Tue Aug 15 11:30:18 2023 -0400 Feature/externalize dataflow xlang kafka tests (#27805) * Update 2.50 release notes to include new Kafka topicPattern feature * Begin running Kafka xlang tests as part of dataflow test suites * Load the Kafka jar to enable running kafkaio tests. Fix linting errors * Update build dependency * Update dependency definitions * Configure kafka xlang test to check for running expansion service * fix lint issue & update expansion service to provide kafka by default * Take two on updating expansion service config * Have dataflow xlang kafka tests depend on hardcoded kafka instance * fix formatting * formatting * format * refactor IO Xlang to separate postcommit * update documentation * wire in xlang io postcommit to dataflow postcommits * split append failure exception message * revert README autoformat * Undo performance test autoformat * Move kafka configuration, fix README.md * fix module plugin * fix BeamModulePlugin * fix BeamModulePlugin * fix whitespace --- .test-infra/jenkins/README.md | 1 + ...tCommit_Python_CrossLanguage_IO_Dataflow.groovy | 55 ++ .../org/apache/beam/gradle/BeamModulePlugin.groovy | 7 +++ release/src/main/scripts/jenkins_jobs.txt | 1 + .../io/external/xlang_kafkaio_it_test.py | 53 +++-- sdks/python/test-suites/dataflow/build.gradle | 6 +++ sdks/python/test-suites/dataflow/common.gradle | 3 +- sdks/python/test-suites/xlang/build.gradle | 13 + 8 files changed, 133 insertions(+), 6 deletions(-) diff --git a/.test-infra/jenkins/README.md b/.test-infra/jenkins/README.md index 02cddfdc65c..aae5842d274 100644 --- a/.test-infra/jenkins/README.md +++ b/.test-infra/jenkins/README.md @@ -162,6 +162,7 @@ Beam Jenkins overview page: [link](https://ci-beam.apache.org/) | beam_PostCommit_Python_VR_Spark | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/) | `Run Python Spark ValidatesRunner` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark) | | beam_PostCommit_Python_Xlang_Gcp_Direct | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct_PR/) | `Run Python_Xlang_Gcp_Direct PostCommit` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Direct/) | | beam_PostCommit_Python_Xlang_Gcp_Dataflow | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow_PR/) | `Run Python_Xlang_Gcp_Dataflow PostCommit` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_Gcp_Dataflow/) | +| beam_PostCommit_Python_Xlang_IO_Dataflow | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_IO_Dataflow/), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_IO_Dataflow_PR/) | `Run Python_Xlang_IO_Dataflow PostCommit` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_IO_Dataflow/| | beam_PostCommit_Python38 | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python38), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python38_PR/) | `Run Python 3.8 PostCommit` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python38) | | beam_PostCommit_Python39 | [cron](https://ci-beam.apache.org/job/beam_PostCommit_Python39), [phrase](https://ci-beam.apache.org/job/beam_PostCommit_Python39_PR/) | `Run Python 3.9 PostCommit` | [](https://ci-beam.apache.org/job/beam_PostCommit_Python39) | | beam_PostCommit_Python310 |
[beam] branch master updated: bump hcls api (#27997)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 767a2d72510 bump hcls api (#27997)
767a2d72510 is described below
commit 767a2d72510bf0d6d100e80f45c74ede913ece30
Author: Svetak Sundhar
AuthorDate: Tue Aug 15 10:21:15 2023 -0400
bump hcls api (#27997)
---
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
index a86e83f8818..a79badee668 100644
--- a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
+++ b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
@@ -663,7 +663,7 @@ class BeamModulePlugin implements Plugin {
// Keep version consistent with the version in
google_cloud_resourcemanager, managed by google_cloud_platform_libraries_bom
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20230129-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20220920-$google_clients_version",
-google_api_services_healthcare :
"com.google.apis:google-api-services-healthcare:v1-rev20230510-$google_clients_version",
+google_api_services_healthcare :
"com.google.apis:google-api-services-healthcare:v1-rev20230728-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev20220904-$google_clients_version",
// Keep version consistent with the version in google_cloud_nio,
managed by google_cloud_platform_libraries_bom
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20230617-$google_clients_version",
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1c5c5471bb7 Updating config from bot 1c5c5471bb7 is described below commit 1c5c5471bb751e7cb658e262adee9c1d22c6c94c Author: github-actions AuthorDate: Tue Aug 15 12:13:50 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index b6026a93c43..877c6f941de 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -8,7 +8,7 @@ "tvalentyn": 1692015251336, "pabloem": 1681281324703, "y1chi": 1667002607045, -"damccorm": 1691624090644, +"damccorm": 1692101626737, "jrmccluskey": 1691786090445, "riteshghorse": 1691699674756, "liferoad": 1691636755122
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new fe01a579cec Updating config from bot
fe01a579cec is described below
commit fe01a579cec34d4413f60746aaec238f841e6f41
Author: github-actions
AuthorDate: Tue Aug 15 12:13:49 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27806.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27806.json
b/scripts/ci/pr-bot/state/pr-state/pr-27806.json
index e1b3308f259..88d1920a8bb 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27806.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27806.json
@@ -1,7 +1,7 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"python": "liferoad"
+"python": "damccorm"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
[beam] branch dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.23.0 deleted (was 70e134c48b8)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.23.0 in repository https://gitbox.apache.org/repos/asf/beam.git was 70e134c48b8 Bump github.com/testcontainers/testcontainers-go in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (9c548b34c7d -> 61ea132f2e4)
This is an automated email from the ASF dual-hosted git repository. jrmccluskey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 9c548b34c7d pull out StreamPool/StreamData from WindmillServerStub file. Organize streaming appliance files into their own directory. (#27593) add 61ea132f2e4 Bump github.com/testcontainers/testcontainers-go in /sdks (#28004) No new revisions were added by this update. Summary of changes: sdks/go.mod | 4 ++-- sdks/go.sum | 8 2 files changed, 6 insertions(+), 6 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 8b1598e950e Updating config from bot
8b1598e950e is described below
commit 8b1598e950e73455cf2978bb4d1f0269d65f3387
Author: github-actions
AuthorDate: Tue Aug 15 09:34:45 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index e421d953c19..69ed548853c 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1692056078820,
+"damccorm": 1692092081029,
"Abacn": 1692077679834
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new a9dc077f809 Updating config from bot a9dc077f809 is described below commit a9dc077f80995d8413cc4726078a419e57469ee7 Author: github-actions AuthorDate: Tue Aug 15 09:34:44 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 209a13575fc..3ed7456985b 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -7,6 +7,6 @@ "kileys": 1674428678843, "apilloud": 1678822446183, "Abacn": 1691787951165, -"bvolpato": 1691773563896 +"bvolpato": 1692092081029 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 138a6162401 Updating config from bot
138a6162401 is described below
commit 138a61624013d3544a97ea762e787f9f9c94cd65
Author: github-actions
AuthorDate: Tue Aug 15 09:34:42 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28007.json | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28007.json
b/scripts/ci/pr-bot/state/pr-state/pr-28007.json
index 242a48d7d3b..f03c6816fe8 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-28007.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28007.json
@@ -1,7 +1,10 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "bvolpato",
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f115c49359f Updating config from bot
f115c49359f is described below
commit f115c49359f62a9db510ab0129ecabdee2b12c80
Author: github-actions
AuthorDate: Tue Aug 15 07:34:50 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28007.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28007.json
b/scripts/ci/pr-bot/state/pr-state/pr-28007.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28007.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
