[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 902d6cd011f Updating config from bot
902d6cd011f is described below
commit 902d6cd011f495475a4b59dd7fc9badbea29589f
Author: github-actions
AuthorDate: Tue Jul 11 05:34:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 9c4b80b9bd8..b3d0af35880 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1688965554372,
-"jrmccluskey": 1688931281217,
+"jrmccluskey": 1689053694130,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1688967312701
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 39e4e5d7d8b Updating config from bot
39e4e5d7d8b is described below
commit 39e4e5d7d8b47a39a096baa9dd47c7497d84da83
Author: github-actions
AuthorDate: Tue Jul 11 05:34:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27446.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27446.json
b/scripts/ci/pr-bot/state/pr-state/pr-27446.json
new file mode 100644
index 000..1c2c2e31975
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27446.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1660f428283 Updating config from bot
1660f428283 is described below
commit 1660f428283ae0aadf495f556d074c573d31cde2
Author: github-actions
AuthorDate: Tue Jul 11 04:52:38 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27196.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27196.json
b/scripts/ci/pr-bot/state/pr-state/pr-27196.json
index ee5d1b9f8a7..6272b4e797d 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27196.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27196.json
@@ -3,7 +3,7 @@
"reviewersAssignedForLabels": {
"python": "AnandInguva"
},
- "nextAction": "Author",
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch dependabot/go_modules/sdks/github.com/tetratelabs/wazero-1.3.0 created (now 9299cf68f23)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/tetratelabs/wazero-1.3.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 9299cf68f23 Bump github.com/tetratelabs/wazero from 1.2.1 to 1.3.0 in /sdks No new revisions were added by this update.
[beam] branch asf-site updated: Publishing website 2023/07/11 04:16:00 at commit a0004b3
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new b8895ca7287 Publishing website 2023/07/11 04:16:00 at commit a0004b3 b8895ca7287 is described below commit b8895ca728747785dd5d36313290d2c55c081a94 Author: jenkins AuthorDate: Tue Jul 11 04:16:00 2023 + Publishing website 2023/07/11 04:16:00 at commit a0004b3 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 8ce9de484a5..4f305b60985 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/categories/blog/2023-07-10T14:10:43-07:00/blog/2023-07-10T14:10:43-07:00/categories/2023-07-10T14:10:43-07:00/blog/managing-beam-dependencies-in-java/2023-07-10T14:10:43-07:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/categories/blog/2023-07-10T21:02:17-07:00/blog/2023-07-10T21:02:17-07:00/categories/2023-07-10T21:02:17-07:00/blog/managing-beam-dependencies-in-java/2023-07-10T21:02:17-07:00 [...] \ No newline at end of file
[beam] branch master updated: Provision a redis cluster on Kubernetes (#27409)
This is an automated email from the ASF dual-hosted git repository.
damondouglas pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new a0004b33e56 Provision a redis cluster on Kubernetes (#27409)
a0004b33e56 is described below
commit a0004b33e56308256f8f82304410462c03181d86
Author: Damon
AuthorDate: Mon Jul 10 21:02:17 2023 -0700
Provision a redis cluster on Kubernetes (#27409)
---
.../03.io/api-overuse-study/02.redis/README.md | 46 ++
.../03.io/api-overuse-study/02.redis/common.tfvars | 19 +
.../03.io/api-overuse-study/02.redis/data.tf | 24 +++
.../03.io/api-overuse-study/02.redis/provider.tf | 27 +
.../03.io/api-overuse-study/02.redis/redis.tf | 35
.../03.io/api-overuse-study/02.redis/variables.tf | 22 +++
6 files changed, 173 insertions(+)
diff --git
a/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/README.md
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/README.md
new file mode 100644
index 000..9ea3a98ee77
--- /dev/null
+++
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/README.md
@@ -0,0 +1,46 @@
+
+
+# Overview
+
+This directory provisions a redis cluster in Kubernetes.
+
+# Usage
+
+Follow terraform workflow convention to apply this module. It assumes the
+working directory is at
+[.test-infra/pipelines/infrastructure/03.io/api-overuse-study](..).
+
+## Terraform Init
+
+Initialize the terraform workspace.
+
+```
+DIR=02.redis
+terraform -chdir=$DIR init
+```
+
+## Terraform Apply
+
+Apply the terraform module.
+
+```
+DIR=02.redis
+terraform -chdir=$DIR apply -var-file=common.tfvars
+```
\ No newline at end of file
diff --git
a/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/common.tfvars
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/common.tfvars
new file mode 100644
index 000..f71b496b5a2
--- /dev/null
+++
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/common.tfvars
@@ -0,0 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+namespace = "api-overuse-study"
\ No newline at end of file
diff --git
a/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/data.tf
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/data.tf
new file mode 100644
index 000..32cb7434d30
--- /dev/null
+++
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/data.tf
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Query the Kubernetes namespace to verify existence.
+data "kubernetes_namespace" "default" {
+ metadata {
+name = var.namespace
+ }
+}
\ No newline at end of file
diff --git
a/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/provider.tf
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/provider.tf
new file mode 100644
index 000..20bdbac74ea
--- /dev/null
+++
b/.test-infra/pipelines/infrastructure/03.io/api-overuse-study/02.redis/provider.tf
@@ -0,0 +1,27 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ *
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b801f183a3e Updating config from bot
b801f183a3e is described below
commit b801f183a3ec5bb6371c3cd88f194986249681c1
Author: github-actions
AuthorDate: Tue Jul 11 02:54:13 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index cbc6b194182..b000c2efb22 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1689001558041,
+"damccorm": 1689044049988,
"Abacn": 1689010497261
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f5a5eff6f70 Updating config from bot
f5a5eff6f70 is described below
commit f5a5eff6f70ebff633a412283cb254785def45ba
Author: github-actions
AuthorDate: Tue Jul 11 02:54:11 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27261.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27261.json
b/scripts/ci/pr-bot/state/pr-state/pr-27261.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27261.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bff5437ed9d Updating config from bot
bff5437ed9d is described below
commit bff5437ed9dd88e21863ce5e45089ed2f5aa9e1c
Author: github-actions
AuthorDate: Tue Jul 11 02:30:53 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index e259ad3223a..235e514a418 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -1,7 +1,7 @@
{
"label": "io",
"dateOfLastReviewAssignment": {
-"chamikaramj": 1688051147551,
+"chamikaramj": 1689042646484,
"johnjcasey": 1689017692295,
"pabloem": 1688472842104,
"Abacn": 1688386491174,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b0a33ed681b Updating config from bot b0a33ed681b is described below commit b0a33ed681b326c9eecd50be29b93cf6a18f9edf Author: github-actions AuthorDate: Tue Jul 11 02:30:50 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index b4e68db4170..85c1ee1a08a 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -11,6 +11,6 @@ "damccorm": 1689012506785, "jrmccluskey": 1688749553476, "riteshghorse": 1689024882446, -"liferoad": 1688585680741 +"liferoad": 1689042646483 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e77f3aa4b0c Updating config from bot
e77f3aa4b0c is described below
commit e77f3aa4b0cf550e57c33a60e30aec8940f07265
Author: github-actions
AuthorDate: Tue Jul 11 02:30:48 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27444.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27444.json
b/scripts/ci/pr-bot/state/pr-state/pr-27444.json
new file mode 100644
index 000..99b4c661337
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27444.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "liferoad",
+"io": "chamikaramj"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Remove unneeded Dataflow Runner v1 code. (#27196)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new ca674aad086 Remove unneeded Dataflow Runner v1 code. (#27196)
ca674aad086 is described below
commit ca674aad086e4d1a40e8c01598073030a22484fa
Author: Robert Bradshaw
AuthorDate: Mon Jul 10 16:31:55 2023 -0700
Remove unneeded Dataflow Runner v1 code. (#27196)
---
sdks/python/apache_beam/coders/coders.py | 133
.../apache_beam/coders/coders_test_common.py | 41 -
sdks/python/apache_beam/coders/row_coder.py| 9 -
sdks/python/apache_beam/coders/row_coder_test.py | 11 -
.../runners/dataflow/dataflow_runner.py| 828 +
.../runners/dataflow/dataflow_runner_test.py | 369 +
.../runners/dataflow/internal/apiclient.py | 95 +--
.../runners/dataflow/internal/apiclient_test.py| 53 --
.../apache_beam/runners/dataflow/internal/names.py | 76 --
.../runners/dataflow/ptransform_overrides.py | 126
10 files changed, 49 insertions(+), 1692 deletions(-)
diff --git a/sdks/python/apache_beam/coders/coders.py
b/sdks/python/apache_beam/coders/coders.py
index d4ca99b80fb..7c5c8e09303 100644
--- a/sdks/python/apache_beam/coders/coders.py
+++ b/sdks/python/apache_beam/coders/coders.py
@@ -280,29 +280,6 @@ class Coder(object):
# refined in user-defined Coders.
return []
- def as_cloud_object(self, coders_context=None):
-"""For internal use only; no backwards-compatibility guarantees.
-
-Returns Google Cloud Dataflow API description of this coder."""
-# This is an internal detail of the Coder API and does not need to be
-# refined in user-defined Coders.
-
-value = {
-# We pass coders in the form "$" to make the
-# job description JSON more readable. Data before the $ is ignored by
-# the worker.
-'@type': serialize_coder(self),
-'component_encodings': [
-component.as_cloud_object(coders_context)
-for component in self._get_component_coders()
-],
-}
-
-if coders_context:
- value['pipeline_proto_coder_id'] = coders_context.get_id(self)
-
-return value
-
def __repr__(self):
return self.__class__.__name__
@@ -493,11 +470,6 @@ class BytesCoder(FastCoder):
def to_type_hint(self):
return bytes
- def as_cloud_object(self, coders_context=None):
-return {
-'@type': 'kind:bytes',
-}
-
def __eq__(self, other):
return type(self) == type(other)
@@ -667,11 +639,6 @@ class VarIntCoder(FastCoder):
def to_type_hint(self):
return int
- def as_cloud_object(self, coders_context=None):
-return {
-'@type': 'kind:varint',
-}
-
def __eq__(self, other):
return type(self) == type(other)
@@ -846,21 +813,6 @@ class _PickleCoderBase(FastCoder):
# GroupByKey operations.
return False
- def as_cloud_object(self, coders_context=None, is_pair_like=True):
-value = super().as_cloud_object(coders_context)
-# We currently use this coder in places where we cannot infer the coder to
-# use for the value type in a more granular way. In places where the
-# service expects a pair, it checks for the "is_pair_like" key, in which
-# case we would fail without the hack below.
-if is_pair_like:
- value['is_pair_like'] = True
- value['component_encodings'] = [
- self.as_cloud_object(coders_context, is_pair_like=False),
- self.as_cloud_object(coders_context, is_pair_like=False)
- ]
-
-return value
-
# We allow .key_coder() and .value_coder() to be called on PickleCoder since
# we can't always infer the return values of lambdas in ParDo operations, the
# result of which may be used in a GroupBykey.
@@ -983,21 +935,6 @@ class FastPrimitivesCoder(FastCoder):
def to_type_hint(self):
return Any
- def as_cloud_object(self, coders_context=None, is_pair_like=True):
-value = super().as_cloud_object(coders_context)
-# We currently use this coder in places where we cannot infer the coder to
-# use for the value type in a more granular way. In places where the
-# service expects a pair, it checks for the "is_pair_like" key, in which
-# case we would fail without the hack below.
-if is_pair_like:
- value['is_pair_like'] = True
- value['component_encodings'] = [
- self.as_cloud_object(coders_context, is_pair_like=False),
- self.as_cloud_object(coders_context, is_pair_like=False)
- ]
-
-return value
-
# We allow .key_coder() and .value_coder() to be called on
FastPrimitivesCoder
# since we can't always infer the return values of lambdas in ParDo
# operations, the result of which may be used in a GroupBykey.
@@ -1231,19 +1168,6 @@ class TupleCoder(FastCoder):
#
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7b2b8a38836 Updating config from bot 7b2b8a38836 is described below commit 7b2b8a38836c86919d7668177f4ce612ee89ea23 Author: github-actions AuthorDate: Mon Jul 10 23:05:53 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27409.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27409.json b/scripts/ci/pr-bot/state/pr-state/pr-27409.json index c9972b0d659..7e5492c676d 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27409.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27409.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6b6f7143b65 Updating config from bot 6b6f7143b65 is described below commit 6b6f7143b653b17dac9b101dac588d08898a6b44 Author: github-actions AuthorDate: Mon Jul 10 23:05:51 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27429.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27429.json b/scripts/ci/pr-bot/state/pr-state/pr-27429.json index 321b31c0f05..7d1dfe55f2f 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27429.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27429.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5c3a873c873 Updating config from bot
5c3a873c873 is described below
commit 5c3a873c873f4635061a9bbb826e838dfed5cfde
Author: github-actions
AuthorDate: Mon Jul 10 22:39:02 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27196.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27196.json
b/scripts/ci/pr-bot/state/pr-state/pr-27196.json
index 6272b4e797d..ee5d1b9f8a7 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27196.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27196.json
@@ -3,7 +3,7 @@
"reviewersAssignedForLabels": {
"python": "AnandInguva"
},
- "nextAction": "Reviewers",
+ "nextAction": "Author",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 27967de4cc8 Updating config from bot
27967de4cc8 is described below
commit 27967de4cc89451c137da53f13a108259ee5d165
Author: github-actions
AuthorDate: Mon Jul 10 22:21:43 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27440.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27440.json
b/scripts/ci/pr-bot/state/pr-state/pr-27440.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27440.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch tvalentyn-patch-3 created (now d60cb4ac612)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git at d60cb4ac612 Update runinference.md This branch includes the following new commits: new d60cb4ac612 Update runinference.md The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Update runinference.md
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a commit to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git commit d60cb4ac612545927727b7d4fe0ad99b7b034f09 Author: tvalentyn AuthorDate: Mon Jul 10 15:19:50 2023 -0700 Update runinference.md Add a link to ML docs page. --- .../en/documentation/transforms/python/elementwise/runinference.md | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md index 39db1aa68d3..369b7471442 100644 --- a/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md +++ b/website/www/site/content/en/documentation/transforms/python/elementwise/runinference.md @@ -29,11 +29,9 @@ limitations under the License. -Uses models to do local and remote inference. A `RunInference` transform performs inference on a `PCollection` of examples using a machine learning (ML) model. The transform outputs a `PCollection` that contains the input examples and output predictions. +Uses models to do local and remote inference. A `RunInference` transform performs inference on a `PCollection` of examples using a machine learning (ML) model. The transform outputs a `PCollection` that contains the input examples and output predictions. Avaliable in Apache Beam 2.40.0 and later versions. -You must have Apache Beam 2.40.0 or later installed to run these pipelines. - -See more [RunInference API pipeline examples](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference). +For more information, read about Beam RunInference APIs on [Machine Learning with Python](https://beam.apache.org/documentation/sdks/python-machine-learning) page and explore [RunInference API pipeline examples](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference). ## Examples
[beam] branch asf-site updated: Publishing website 2023/07/10 22:17:10 at commit c5d6183
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 2d439061ac8 Publishing website 2023/07/10 22:17:10 at commit c5d6183 2d439061ac8 is described below commit 2d439061ac876d3b8d6d1d37cff805cd104191f0 Author: jenkins AuthorDate: Mon Jul 10 22:17:10 2023 + Publishing website 2023/07/10 22:17:10 at commit c5d6183 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 9bde8ac809b..8ce9de484a5 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/categories/blog/2023-07-10T18:01:47+02:00/blog/2023-07-10T18:01:47+02:00/categories/2023-07-10T18:01:47+02:00/blog/managing-beam-dependencies-in-java/2023-07-10T18:01:47+02:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/categories/blog/2023-07-10T14:10:43-07:00/blog/2023-07-10T14:10:43-07:00/categories/2023-07-10T14:10:43-07:00/blog/managing-beam-dependencies-in-java/2023-07-10T14:10:43-07:00 [...] \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f7ae5282f85 Updating config from bot f7ae5282f85 is described below commit f7ae5282f850d85ca5dff8ff40c2d9e5d897b722 Author: github-actions AuthorDate: Mon Jul 10 21:34:46 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index fab315c9177..b4e68db4170 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -10,7 +10,7 @@ "y1chi": 1667002607045, "damccorm": 1689012506785, "jrmccluskey": 1688749553476, -"riteshghorse": 1688137558599, +"riteshghorse": 1689024882446, "liferoad": 1688585680741 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 050bce98b45 Updating config from bot
050bce98b45 is described below
commit 050bce98b4537024171c9d4d53d82620290902a8
Author: github-actions
AuthorDate: Mon Jul 10 21:34:44 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27436.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27436.json
b/scripts/ci/pr-bot/state/pr-state/pr-27436.json
new file mode 100644
index 000..c43704aa814
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27436.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 2af35e8b971 Updating config from bot
2af35e8b971 is described below
commit 2af35e8b971c22b713571ad07b36fe546ee7432b
Author: github-actions
AuthorDate: Mon Jul 10 21:16:09 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27439.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27439.json
b/scripts/ci/pr-bot/state/pr-state/pr-27439.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27439.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Implement environment variable type (#27407)
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new c5d6183cee1 Implement environment variable type (#27407)
c5d6183cee1 is described below
commit c5d6183cee1c22c572cbf51ec4a2c1490dafafe8
Author: Damon
AuthorDate: Mon Jul 10 14:10:43 2023 -0700
Implement environment variable type (#27407)
---
.test-infra/pipelines/go.mod | 21 ++
.test-infra/pipelines/go.sum | 2 +
.../src/main/go/internal/environment/variable.go | 81 +
.../main/go/internal/environment/variable_test.go | 358 +
4 files changed, 462 insertions(+)
diff --git a/.test-infra/pipelines/go.mod b/.test-infra/pipelines/go.mod
new file mode 100644
index 000..0204d037a4c
--- /dev/null
+++ b/.test-infra/pipelines/go.mod
@@ -0,0 +1,21 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// This module contains all Go code for internal use by .test-infra pipelines.
+module github.com/apache/beam/test-infra/pipelines
+
+go 1.20
+
+require github.com/google/go-cmp v0.5.9 // indirect
diff --git a/.test-infra/pipelines/go.sum b/.test-infra/pipelines/go.sum
new file mode 100644
index 000..62841cdb151
--- /dev/null
+++ b/.test-infra/pipelines/go.sum
@@ -0,0 +1,2 @@
+github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
+github.com/google/go-cmp v0.5.9/go.mod
h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
diff --git a/.test-infra/pipelines/src/main/go/internal/environment/variable.go
b/.test-infra/pipelines/src/main/go/internal/environment/variable.go
new file mode 100644
index 000..1e81a2e2ce8
--- /dev/null
+++ b/.test-infra/pipelines/src/main/go/internal/environment/variable.go
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package environment provides helpers for interacting with environment
variables.
+package environment
+
+import (
+ "fmt"
+ "os"
+ "strings"
+)
+
+// Variable defines an environment variable via a string type alias.
+// Variable's string defaultValue assigns the system environment variable key.
+type Variable string
+
+// Default a defaultValue to the system environment.
+func (v Variable) Default(value string) error {
+ if v.Missing() {
+ return os.Setenv((string)(v), value)
+ }
+ return nil
+}
+
+// Missing reports whether the system environment variable is an empty string.
+func (v Variable) Missing() bool {
+ return v.Value() == ""
+}
+
+// Key returns the system environment variable key.
+func (v Variable) Key() string {
+ return (string)(v)
+}
+
+// Value returns the system environment variable defaultValue.
+func (v Variable) Value() string {
+ return os.Getenv((string)(v))
+}
+
+// KeyValue returns a concatenated string of the system environment variable's
+// =.
+func (v Variable) KeyValue() string {
+ return fmt.Sprintf("%s=%s", (string)(v), v.Value())
+}
+
+// Missing reports as an error listing all Variable among vars that are
+// not assigned in the system environment.
+func Missing(vars ...Variable) error {
+ var missing []string
+ for _, v := range vars {
+ if v.Missing() {
+ missing = append(missing, v.KeyValue())
+ }
+ }
+ if
[beam] branch master updated: Refactor MLTrasform BaseOperation class (#27389)
This is an automated email from the ASF dual-hosted git repository.
anandinguva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 5a5779fe43f Refactor MLTrasform BaseOperation class (#27389)
5a5779fe43f is described below
commit 5a5779fe43f5547f6227f5ce0c4dbca304eee3e1
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Mon Jul 10 20:58:23 2023 +
Refactor MLTrasform BaseOperation class (#27389)
* Refactor code. Make get_artifacts abstract method of BaseOperation
* rename apply to apply_transform
* Add Docstring to MLTransform
* Add _validate_transform method
* Fix mypy
* Provide default value while fetching vocab size from utils
* Fix base test
---
sdks/python/apache_beam/ml/transforms/base.py | 59 +++---
sdks/python/apache_beam/ml/transforms/base_test.py | 2 +-
sdks/python/apache_beam/ml/transforms/handlers.py | 8 +--
.../apache_beam/ml/transforms/handlers_test.py | 13 ++---
sdks/python/apache_beam/ml/transforms/tft.py | 40 +++
sdks/python/apache_beam/ml/transforms/utils.py | 3 +-
6 files changed, 79 insertions(+), 46 deletions(-)
diff --git a/sdks/python/apache_beam/ml/transforms/base.py
b/sdks/python/apache_beam/ml/transforms/base.py
index f2906409484..04aa387580a 100644
--- a/sdks/python/apache_beam/ml/transforms/base.py
+++ b/sdks/python/apache_beam/ml/transforms/base.py
@@ -17,6 +17,7 @@
# pytype: skip-file
import abc
+from typing import Dict
from typing import Generic
from typing import List
from typing import Optional
@@ -55,16 +56,38 @@ class BaseOperation(Generic[OperationInputT,
OperationOutputT], abc.ABC):
self.columns = columns
@abc.abstractmethod
- def apply(
- self, data: OperationInputT, output_column_name: str) ->
OperationOutputT:
+ def apply_transform(self, data: OperationInputT,
+ output_column_name: str) -> Dict[str, OperationOutputT]:
"""
-Define any processing logic in the apply() method.
+Define any processing logic in the apply_transform() method.
processing logics are applied on inputs and returns a transformed
output.
Args:
inputs: input data.
"""
+ @abc.abstractmethod
+ def get_artifacts(
+ self, data: OperationInputT,
+ output_column_prefix: str) -> Optional[Dict[str, OperationOutputT]]:
+"""
+If the operation generates any artifacts, they can be returned from this
+method.
+"""
+pass
+
+ def __call__(self, data: OperationInputT,
+ output_column_name: str) -> Dict[str, OperationOutputT]:
+"""
+This method is called when the instance of the class is called.
+This method will invoke the apply() method of the class.
+"""
+transformed_data = self.apply_transform(data, output_column_name)
+artifacts = self.get_artifacts(data, output_column_name)
+if artifacts:
+ transformed_data = {**transformed_data, **artifacts}
+return transformed_data
+
class ProcessHandler(Generic[ExampleT, MLTransformOutputT], abc.ABC):
"""
@@ -96,6 +119,20 @@ class
MLTransform(beam.PTransform[beam.PCollection[ExampleT],
artifact_mode: str = ArtifactMode.PRODUCE,
transforms: Optional[Sequence[BaseOperation]] = None):
"""
+MLTransform is a Beam PTransform that can be used to apply
+transformations to the data. MLTransform is used to wrap the
+data processing transforms provided by Apache Beam. MLTransform
+works in two modes: produce and consume. In the produce mode,
+MLTransform will apply the transforms to the data and store the
+artifacts in the artifact_location. In the consume mode, MLTransform
+will read the artifacts from the artifact_location and apply the
+transforms to the data. The artifact_location should be a valid
+storage path where the artifacts can be written to or read from.
+
+Note that when consuming artifacts, it is not necessary to pass the
+transforms since they are inherently stored within the artifacts
+themselves.
+
Args:
artifact_location: A storage location for artifacts resulting from
MLTransform. These artifacts include transformations applied to
@@ -113,15 +150,16 @@ class
MLTransform(beam.PTransform[beam.PCollection[ExampleT],
i-th transform is the output of the (i-1)-th transform. Multi-input
transforms are not supported yet.
artifact_mode: Whether to produce or consume artifacts. If set to
-'consume', the handler will assume that the artifacts are already
+'consume', MLTransform will assume that the artifacts are already
computed and stored in the artifact_location. Pass the same artifact
location that was passed during produce phase to ensure that the
-right
[beam] branch dependabot/gradle/com.gradle.enterprise-3.13.4 deleted (was 9db4a84ed8a)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.gradle.enterprise-3.13.4 in repository https://gitbox.apache.org/repos/asf/beam.git was 9db4a84ed8a Bump com.gradle.enterprise from 3.13.2 to 3.13.4 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.19.0 deleted (was 7dff8a2c824)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.19.0 in repository https://gitbox.apache.org/repos/asf/beam.git was 7dff8a2c824 Bump cloud.google.com/go/bigtable from 1.18.1 to 1.19.0 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (738c02cac4d -> 9f8b0e3f9f9)
This is an automated email from the ASF dual-hosted git repository. riteshghorse pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 738c02cac4d Runner image buildx and Website Precommit update (#27427) add 9f8b0e3f9f9 Bump cloud.google.com/go/bigtable from 1.18.1 to 1.19.0 in /sdks (#27421) No new revisions were added by this update. Summary of changes: sdks/go.mod | 2 +- sdks/go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
[beam] branch master updated: Runner image buildx and Website Precommit update (#27427)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 738c02cac4d Runner image buildx and Website Precommit update (#27427)
738c02cac4d is described below
commit 738c02cac4d6a88faa3dd02606d50f1d2efed5e7
Author: Vlado Djerek <20559691+volatilemolo...@users.noreply.github.com>
AuthorDate: Mon Jul 10 22:40:10 2023 +0200
Runner image buildx and Website Precommit update (#27427)
* add buildx and fix gcloud
* precommit website update and enable
---
.../arc/images/Dockerfile | 8 ++-
.github/workflows/beam_PreCommit_Website.yml | 79 --
2 files changed, 65 insertions(+), 22 deletions(-)
diff --git a/.github/gh-actions-self-hosted-runners/arc/images/Dockerfile
b/.github/gh-actions-self-hosted-runners/arc/images/Dockerfile
index 5a646c6940c..61f05ef649a 100644
--- a/.github/gh-actions-self-hosted-runners/arc/images/Dockerfile
+++ b/.github/gh-actions-self-hosted-runners/arc/images/Dockerfile
@@ -17,8 +17,11 @@
# under the License.
#
-
FROM
ghcr.io/actions-runner-controller/actions-runner-controller/actions-runner:ubuntu-20.04
+#install BuildX
+COPY --from=docker/buildx-bin:0.11.1 /buildx
/usr/libexec/docker/cli-plugins/docker-buildx
+RUN docker buildx install && docker buildx version
+
USER root
#Install Node
@@ -56,7 +59,8 @@ RUN curl -OL
https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-c
rm google-cloud-sdk-367.0.0-linux-x86_64.tar.gz && \
mv google-cloud-sdk /usr/local/google-cloud-sdk && \
/usr/local/google-cloud-sdk/install.sh --quiet && \
-/usr/local/google-cloud-sdk/bin/gcloud components install kubectl
+/usr/local/google-cloud-sdk/bin/gcloud components install kubectl && \
+chown -R runner:runner /home/runner/.config/gcloud
ENV PATH="${PATH}:/usr/local/google-cloud-sdk/bin"
# Needed to transfer path addtitions to runner environment
diff --git a/.github/workflows/beam_PreCommit_Website.yml
b/.github/workflows/beam_PreCommit_Website.yml
index 3152f2eda8c..4254a998eba 100644
--- a/.github/workflows/beam_PreCommit_Website.yml
+++ b/.github/workflows/beam_PreCommit_Website.yml
@@ -13,27 +13,66 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-name: beam_PreCommit_Website
+name: Website PreCommit
on:
- workflow_dispatch:
+ push:
+tags: ['v*']
+branches: ['master', 'release-*']
+paths: ['website/**']
+ pull_request_target:
+branches: ['master', 'release-*']
+paths: ['website/**','.github/workflows/beam_PreCommit_Website.yml']
+ issue_comment:
+types: [created]
+ schedule:
+- cron: '* */6 * * *'
+#Setting explicit permissions for the action to avoid the default permissions
which are `write-all` in case of pull_request_target event
+permissions:
+ actions: write
+ pull-requests: read
+ checks: read
+ contents: read
+ deployments: read
+ id-token: none
+ issues: read
+ discussions: read
+ packages: read
+ pages: read
+ repository-projects: read
+ security-events: read
+ statuses: read
jobs:
-beam_PreCommit_Website:
-if: ${{github.event.issue.pull_request}} ||
${{github.event.comment.body == 'Run Website PreCommit'}} ||
${{github.event.schedule}}
-runs-on: [self-hosted, ubuntu-20.04]
-name: beam_PreCommit_Website
-steps:
- - name: Git checkout
-uses: actions/checkout@v3
- - name: Install Java
-uses: actions/setup-java@v3.8.0
-with:
- distribution: 'zulu'
- java-version: '8'
- - name: Setup Gradle
-uses: gradle/gradle-build-action@v2
-with:
- cache-read-only: false
- - name: run websitePreCommit script
-run: ./gradlew :websitePreCommit
\ No newline at end of file
+ beam_PreCommit_Website:
+if: |
+ github.event_name == 'push' ||
+ github.event_name == 'pull_request_target' ||
+ github.event.comment.body == 'Run Website PreCommit' ||
+ github.event_name == 'schedule'
+runs-on: [self-hosted, ubuntu-20.04, small]
+name: beam_PreCommit_Website
+steps:
+ - uses: actions/checkout@v3
+with:
+ ref: ${{ github.event.pull_request.head.sha }}
+ - name: Rerun on comment
+if: github.event.comment.body == 'Run Python_Coverage PreCommit'
+uses: ./.github/actions/rerun-job-action
+with:
+ pull_request_url: ${{ github.event.issue.pull_request.url }}
+ github_repository: ${{ github.repository }}
+ github_token: ${{ secrets.GITHUB_TOKEN }}
+ github_job: ${{ github.job }}
+ github_current_run_id: ${{ github.run_id }}
+ - name: Install Java
+uses:
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new aa3fedf2c74 Updating config from bot aa3fedf2c74 is described below commit aa3fedf2c7440f153d35331a14af8c8be5172780 Author: github-actions AuthorDate: Mon Jul 10 20:34:57 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index a55389dd15f..fab315c9177 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1688081692886, +"tvalentyn": 1689021293028, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1689012506785,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 8137eb2d445 Updating config from bot
8137eb2d445 is described below
commit 8137eb2d445042abad5683b3273a7177cca3a53a
Author: github-actions
AuthorDate: Mon Jul 10 20:34:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27429.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27429.json
b/scripts/ci/pr-bot/state/pr-state/pr-27429.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27429.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5df7fc8da6f Updating config from bot
5df7fc8da6f is described below
commit 5df7fc8da6fead79897d43a17fa6b091c9f4d444
Author: github-actions
AuthorDate: Mon Jul 10 19:41:07 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27437.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27437.json
b/scripts/ci/pr-bot/state/pr-state/pr-27437.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27437.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 58ab83fac0b Updating config from bot
58ab83fac0b is described below
commit 58ab83fac0bb1a769821745c88b8ac4c2f7a2a0f
Author: github-actions
AuthorDate: Mon Jul 10 19:34:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 2ffbde788fa..ea52f7313d1 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
"kennknowles": 1688896598481,
-"robertwb": 1688386489750,
+"robertwb": 1689017692295,
"kileys": 1674428678843,
"apilloud": 1678822446183,
"Abacn": 1688472841176,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 403f001abe2 Updating config from bot
403f001abe2 is described below
commit 403f001abe24229eaa968639828c44cf1ea09853
Author: github-actions
AuthorDate: Mon Jul 10 19:34:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index aefdb0b1860..e259ad3223a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -2,7 +2,7 @@
"label": "io",
"dateOfLastReviewAssignment": {
"chamikaramj": 1688051147551,
-"johnjcasey": 1687956392754,
+"johnjcasey": 1689017692295,
"pabloem": 1688472842104,
"Abacn": 1688386491174,
"ahmedabu98": 1688585680741,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 537b8118bfa Updating config from bot
537b8118bfa is described below
commit 537b8118bfa27f2c63cb6512f87840707f937948
Author: github-actions
AuthorDate: Mon Jul 10 19:34:54 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27434.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27434.json
b/scripts/ci/pr-bot/state/pr-state/pr-27434.json
new file mode 100644
index 000..9ad2820ae6d
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27434.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "robertwb",
+"io": "johnjcasey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated (8140ccd41cd -> e8e604d1c51)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 8140ccd41cd Beam pre commit java examples dataflow (#27432) add e8e604d1c51 Add build.gradle.kts to whitespace check (#27431) No new revisions were added by this update. Summary of changes: .test-infra/jenkins/job_PreCommit_Whitespace.groovy | 1 + build.gradle.kts| 6 +++--- learning/tour-of-beam/terraform/build.gradle.kts| 10 +- sdks/python/test-suites/tox/py38/build.gradle | 2 ++ 4 files changed, 11 insertions(+), 8 deletions(-)
[beam] branch master updated: Beam pre commit java examples dataflow (#27432)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 8140ccd41cd Beam pre commit java examples dataflow (#27432) 8140ccd41cd is described below commit 8140ccd41cdd6d2489076589a218b85dfa0029e1 Author: Andrey Devyatkin AuthorDate: Mon Jul 10 21:09:12 2023 +0200 Beam pre commit java examples dataflow (#27432) * added beam_PreCommit_Java_Examples_Dataflow job * revert pull_request_target back --- .../beam_PreCommit_Java_Examples_Dataflow.yml | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/.github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml b/.github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml index 0d5fd31e8d2..8d3e90a52e9 100644 --- a/.github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml +++ b/.github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml @@ -29,15 +29,15 @@ on: - 'examples/kotlin/**' - 'release/**' - '.github/workflows/beam_PreCommit_Java_Examples_Dataflow.yml' - #pull_request_target: - # branches: ['master', 'release-*'] - # paths: - #- 'model/**' - #- 'sdks/java/**' - #- 'runners/google-cloud-dataflow-java/**' - #- 'examples/java/**' - #- 'examples/kotlin/**' - #- 'release/**' + pull_request_target: + branches: ['master', 'release-*'] + paths: + - 'model/**' + - 'sdks/java/**' + - 'runners/google-cloud-dataflow-java/**' + - 'examples/java/**' + - 'examples/kotlin/**' + - 'release/**' issue_comment: types: [created] schedule:
[beam] branch dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.19.0 updated (0c8a508878b -> 7dff8a2c824)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.19.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 0c8a508878b Bump cloud.google.com/go/bigtable from 1.18.1 to 1.19.0 in
/sdks
add 88e635a2b0a [Python] Make BigQuery load job names deterministic for
retry resilience (#27384)
add dcd8765da76 Support for read from Cosmos DB Core SQL API (#25672)
add cc4b8f0782c fix SparkRunner url (#27405)
add fcde1d15b29 Bump cloud.google.com/go/datastore from 1.12.0 to 1.12.1
in /sdks (#27422)
add 7dff8a2c824 Bump cloud.google.com/go/bigtable from 1.18.1 to 1.19.0 in
/sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0c8a508878b)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/cloud.google.com/go/bigtable-1.19.0
(7dff8a2c824)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 1 +
README.md | 2 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 6 +-
sdks/go.mod| 2 +-
sdks/go.sum| 4 +-
sdks/java/io/{azure => azure-cosmos}/build.gradle | 37 +--
.../apache/beam/sdk/io/azure/cosmos/CosmosIO.java | 307 +
.../beam/sdk/io/azure/cosmos/CosmosOptions.java| 81 ++
.../beam/sdk/io/azure/cosmos/NormalizedRange.java | 66 +
.../beam/sdk/io/azure/cosmos/package-info.java | 20 ++
.../com/azure/cosmos/examples/common/Address.java | 50
.../com/azure/cosmos/examples/common/Child.java| 68 +
.../com/azure/cosmos/examples/common/Families.java | 133 +
.../com/azure/cosmos/examples/common/Family.java | 88 ++
.../com/azure/cosmos/examples/common/Parent.java | 48
.../java/com/azure/cosmos/examples/common/Pet.java | 32 +++
.../beam/sdk/io/azure/cosmos/CosmosIOTest.java | 176
sdks/java/io/azure/README.md | 19 --
sdks/java/io/azure/build.gradle| 23 +-
.../{BlobstoreOptions.java => AzureOptions.java} | 72 +
.../sdk/io/azure/options/BlobstoreOptions.java | 39 +--
.../apache_beam/io/gcp/bigquery_file_loads.py | 21 +-
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 56 ++--
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 2 +-
settings.gradle.kts| 1 +
25 files changed, 1160 insertions(+), 194 deletions(-)
copy sdks/java/io/{azure => azure-cosmos}/build.gradle (60%)
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/CosmosIO.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/CosmosOptions.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/NormalizedRange.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/package-info.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Address.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Child.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Families.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Family.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Parent.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Pet.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/org/apache/beam/sdk/io/azure/cosmos/CosmosIOTest.java
delete mode 100644 sdks/java/io/azure/README.md
copy
sdks/java/io/azure/src/main/java/org/apache/beam/sdk/io/azure/options/{BlobstoreOptions.java
=> AzureOptions.java} (51%)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 9b93b781d42 Updating config from bot 9b93b781d42 is described below commit 9b93b781d42d4fed17d6918e129eb013fa39972e Author: github-actions AuthorDate: Mon Jul 10 18:35:35 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27213.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27213.json b/scripts/ci/pr-bot/state/pr-state/pr-27213.json index ec82f373f0a..9c31b6ec87e 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27213.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27213.json @@ -4,7 +4,7 @@ "no-matching-label": "lostluck" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch dependabot/go_modules/sdks/cloud.google.com/go/datastore-1.12.1 deleted (was 79d08a9e231)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/cloud.google.com/go/datastore-1.12.1 in repository https://gitbox.apache.org/repos/asf/beam.git was 79d08a9e231 Bump cloud.google.com/go/datastore from 1.12.0 to 1.12.1 in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (cc4b8f0782c -> fcde1d15b29)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from cc4b8f0782c fix SparkRunner url (#27405) add fcde1d15b29 Bump cloud.google.com/go/datastore from 1.12.0 to 1.12.1 in /sdks (#27422) No new revisions were added by this update. Summary of changes: sdks/go.mod | 2 +- sdks/go.sum | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
[beam] branch master updated: fix SparkRunner url (#27405)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new cc4b8f0782c fix SparkRunner url (#27405) cc4b8f0782c is described below commit cc4b8f0782c8dd9c51f882d2b67acbae64d64c22 Author: case-k <40357957+case-k-...@users.noreply.github.com> AuthorDate: Tue Jul 11 03:30:05 2023 +0900 fix SparkRunner url (#27405) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b793a01f593..b579fb58b3a 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ Beam supports executing programs on multiple distributed processing backends thr - The `DirectRunner` runs the pipeline on your local machine. - The `DataflowRunner` submits the pipeline to the [Google Cloud Dataflow](http://cloud.google.com/dataflow/). - The `FlinkRunner` runs the pipeline on an Apache Flink cluster. The code has been donated from [dataArtisans/flink-dataflow](https://github.com/dataArtisans/flink-dataflow) and is now part of Beam. -- The `SparkRunner` runs the pipeline on an Apache Spark cluster. The code has been donated from [cloudera/spark-dataflow](https://github.com/cloudera/spark-dataflow) and is now part of Beam. +- The `SparkRunner` runs the pipeline on an Apache Spark cluster. - The `JetRunner` runs the pipeline on a Hazelcast Jet cluster. The code has been donated from [hazelcast/hazelcast-jet](https://github.com/hazelcast/hazelcast-jet) and is now part of Beam. - The `Twister2Runner` runs the pipeline on a Twister2 cluster. The code has been donated from [DSC-SPIDAL/twister2](https://github.com/DSC-SPIDAL/twister2) and is now part of Beam.
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 24132bdf949 Updating config from bot 24132bdf949 is described below commit 24132bdf949dd364b35396518fddbf5b3e90ba50 Author: github-actions AuthorDate: Mon Jul 10 18:08:30 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 4deccad8c7b..a55389dd15f 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -8,7 +8,7 @@ "tvalentyn": 1688081692886, "pabloem": 1681281324703, "y1chi": 1667002607045, -"damccorm": 1688079973756, +"damccorm": 1689012506785, "jrmccluskey": 1688749553476, "riteshghorse": 1688137558599, "liferoad": 1688585680741
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b28e85f4bc3 Updating config from bot
b28e85f4bc3 is described below
commit b28e85f4bc33b7b13581d199c7913a53cbf1bc7e
Author: github-actions
AuthorDate: Mon Jul 10 18:08:28 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27430.json | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27430.json
b/scripts/ci/pr-bot/state/pr-state/pr-27430.json
index 242a48d7d3b..badbe485421 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27430.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27430.json
@@ -1,7 +1,9 @@
{
"commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "reviewersAssignedForLabels": {
+"python": "damccorm"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 725668e118e Updating config from bot
725668e118e is described below
commit 725668e118e4b6f196a28047d056c287beca63d7
Author: github-actions
AuthorDate: Mon Jul 10 17:35:00 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index a101c025665..cbc6b194182 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1689001558041,
-"Abacn": 1688991239926
+"Abacn": 1689010497261
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 6cfd4acfb72 Updating config from bot
6cfd4acfb72 is described below
commit 6cfd4acfb72427a1df0f6d0071730a898baa9576
Author: github-actions
AuthorDate: Mon Jul 10 17:35:03 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27430.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27430.json
b/scripts/ci/pr-bot/state/pr-state/pr-27430.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27430.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new af0693bce93 Updating config from bot
af0693bce93 is described below
commit af0693bce9341bd8aab2ce9ef4b315109ce67b3a
Author: github-actions
AuthorDate: Mon Jul 10 17:34:58 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27432.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27432.json
b/scripts/ci/pr-bot/state/pr-state/pr-27432.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27432.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch direct_runner_bug updated (ff3019b6912 -> 87634de3675)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch direct_runner_bug in repository https://gitbox.apache.org/repos/asf/beam.git from ff3019b6912 Fix mypy error add 87634de3675 Update sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py | 1 - 1 file changed, 1 deletion(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ec0fca5e15d Updating config from bot
ec0fca5e15d is described below
commit ec0fca5e15d6c7db06cb2e201ac93d920edf5201
Author: github-actions
AuthorDate: Mon Jul 10 16:56:42 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27431.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27431.json
b/scripts/ci/pr-bot/state/pr-state/pr-27431.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27431.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch direct_runner_bug updated (cac43167919 -> ff3019b6912)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch direct_runner_bug in repository https://gitbox.apache.org/repos/asf/beam.git from cac43167919 Apply suggestions from code review add ff3019b6912 Fix mypy error No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch master updated (88e635a2b0a -> dcd8765da76)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 88e635a2b0a [Python] Make BigQuery load job names deterministic for
retry resilience (#27384)
add dcd8765da76 Support for read from Cosmos DB Core SQL API (#25672)
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 1 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 6 +-
sdks/java/io/{azure => azure-cosmos}/build.gradle | 37 +--
.../apache/beam/sdk/io/azure/cosmos/CosmosIO.java | 307 +
.../beam/sdk/io/azure/cosmos/CosmosOptions.java| 81 ++
.../beam/sdk/io/azure/cosmos/NormalizedRange.java | 66 +
.../beam/sdk/io/azure/cosmos/package-info.java | 20 ++
.../com/azure/cosmos/examples/common/Address.java | 50
.../com/azure/cosmos/examples/common/Child.java| 68 +
.../com/azure/cosmos/examples/common/Families.java | 133 +
.../com/azure/cosmos/examples/common/Family.java | 88 ++
.../com/azure/cosmos/examples/common/Parent.java | 48
.../java/com/azure/cosmos/examples/common/Pet.java | 32 +++
.../beam/sdk/io/azure/cosmos/CosmosIOTest.java | 176
sdks/java/io/azure/README.md | 19 --
sdks/java/io/azure/build.gradle| 23 +-
.../{BlobstoreOptions.java => AzureOptions.java} | 72 +
.../sdk/io/azure/options/BlobstoreOptions.java | 39 +--
settings.gradle.kts| 1 +
19 files changed, 1112 insertions(+), 155 deletions(-)
copy sdks/java/io/{azure => azure-cosmos}/build.gradle (60%)
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/CosmosIO.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/CosmosOptions.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/NormalizedRange.java
create mode 100644
sdks/java/io/azure-cosmos/src/main/java/org/apache/beam/sdk/io/azure/cosmos/package-info.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Address.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Child.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Families.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Family.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Parent.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/com/azure/cosmos/examples/common/Pet.java
create mode 100644
sdks/java/io/azure-cosmos/src/test/java/org/apache/beam/sdk/io/azure/cosmos/CosmosIOTest.java
delete mode 100644 sdks/java/io/azure/README.md
copy
sdks/java/io/azure/src/main/java/org/apache/beam/sdk/io/azure/options/{BlobstoreOptions.java
=> AzureOptions.java} (51%)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 0d00cd85165 Updating config from bot
0d00cd85165 is described below
commit 0d00cd851656a8d7b5a3e9bec2897676414982bf
Author: github-actions
AuthorDate: Mon Jul 10 15:32:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27373.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27373.json
b/scripts/ci/pr-bot/state/pr-state/pr-27373.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27373.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch direct_runner_bug updated (a576094632f -> cac43167919)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch direct_runner_bug in repository https://gitbox.apache.org/repos/asf/beam.git from a576094632f Merge remote-tracking branch 'upstream/master' into direct_runner_bug add cac43167919 Apply suggestions from code review No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py | 3 --- 1 file changed, 3 deletions(-)
[beam] branch direct_runner_bug updated (aeb7661f272 -> a576094632f)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch direct_runner_bug in repository https://gitbox.apache.org/repos/asf/beam.git from aeb7661f272 Create empty ListBuffer when buffer is none add 22c4cc216e7 Replace empty buffer with a List/GroupBuffer add 946324ae137 set _isPlaygroundShown after _currentUnitContent (#27242) add bac98cf2075 add gcloud auth and push to registry on merge PR (#27341) add e8d8043f441 Upgrade obsolete Avro plugin to 0.16.0 (#27232) add 2c30acfe75b Consider beam_services option when determining the expansion service type add c47438802d9 Merge pull request #27345: Consider beam_services option when determining the expansion service type add 4d38d953788 updated go precommit with new condition, re-run action, pull_req_targ… (#27321) add e00e19c0887 adds GitHub CLI to runner image (#27342) add 619e41c7c0a Bump github.com/proullon/ramsql in /sdks (#27337) add 54adbcb0b9c Bump actions/setup-python from 3 to 4 (#27336) add 8b8fdcaf71c Verify script improvements (#27309) add 3457434de74 fix repo and add paths to push trigger (#27357) add 4a7a2579c70 Update Storage Write API Autosharding documentation (#27316) add dc5bdb0d1e0 Bump golang.org/x/text from 0.10.0 to 0.11.0 in /sdks (#27351) add a4141189b83 Updates PythonExternalTransform to support IPv6 addresses add aa6f0a364db Merge pull request #27332: Updates PythonExternalTransform to support IPv6 addresses add 35bdebc4c1b wrong path fix (#27359) add 1842dd2f07a Respect BigQuery insert byte size limits (#24979) add 0ce0d9acead Keep roles for gcp SA updated (#27306) add cb8adf123e7 adding glcoud cli and kubectl (#27358) add 2a1276544f6 Add Windmill support for MultimapState (#23492) add 24fa67f958b Fix windmill proto (#27361) add 9a11e28ce79 Use DOCKERHUB_USER instead of username (#27362) add 15a54a00ac0 [Go SDK][Prism] Improve failed pipeline handling (#27297) add 79fcfc6fcb5 Fix flakiness for BeamFnLoggingClientTest (#27308) add 5bc80aa2e6d Inherit Generic for TimestampedValue (#26290) add c28fa0639c2 Update 2.50 release notes to include new Kafka topicPattern feature (#27367) add a6760153543 Bump golang.org/x/oauth2 from 0.9.0 to 0.10.0 in /sdks (#27376) add cafb3da0523 Bump @grpc/grpc-js from 1.6.7 to 1.8.8 in /sdks/typescript (#27372) add 556d93365a1 Update dataflow container versions (#27381) add 44f165fedf6 Update python options in runners/spark.md. (#27334) add a5873792a36 Fix mismatched section title in Release Guide (#27383) add b529efecc9d MLTransform (#26795) add 9e6420d9f90 Arc additional pools (#27369) add 7eaef18bcbf Address failing BT read tests in #27319 (#27322) add ccf6544a636 Automatically use Docker Compose based transform service for PythonExternalTransform when needed. add f19c384af1f Merge pull request #27365: Automatically use Docker Compose based transform service for PythonExternalTransform when needed. add a12577ae68a python precommit jobs and readme (#27379) add a0c9aa2e4f7 Bump google.golang.org/grpc from 1.56.1 to 1.56.2 in /sdks (#27387) add 71cb3ca2403 Bump google.golang.org/api from 0.129.0 to 0.130.0 in /sdks (#27377) add 7ce2d00464f Revert "Replace StorageV1 client with GCS client (#25965)" (#27388) add 4ba5e43a789 Fix CombineGlobally with GlobalWindows (#26922) add f643872f0b7 Restore trailing slash accidentally removed in https://github.com/apache/beam/pull/25665 add dbf9ecce8f5 Merge pull request #27386 from Restore trailing slash in Dockerfile. add cbfeb2e5ec7 fixes rerun call in matrix jobs (#27392) add 79d210c7f01 added beam_PreCommit_Java_Examples_Dataflow job (#27233) add 4026d9e3482 Migrate "beam_PreCommit_Typescript" Jenkins job to Github Actions (#27141) add c06247bf79e Fix staged RC source name in GA (#27394) add e6cccdef7d2 Ensure CancellableQueue doesn't keep references to logically removed objects. (#27403) add ed7cbcaa0f9 temporarily disable PreCommit_Java_Examples_Dataflow.yml (#27420) add e809912fa30 Fix a typo in the GenAI notebook (#27418) add 88e635a2b0a [Python] Make BigQuery load job names deterministic for retry resilience (#27384) add a576094632f Merge remote-tracking branch 'upstream/master' into direct_runner_bug No new revisions were added by this update. Summary of changes: .github/actions/rerun-job-action/action.yml|7 + .../gh-actions-self-hosted-runners/arc/README.md | 44 +- .../arc/config/arc_autoscaler.tpl |4 +- .../arc/config/arc_deployment.tpl | 35 +- .../arc/environments/beam.env | 42 +- .github/gh-actions-self-hosted-runners/arc/gke.tf | 53 +- .github/gh-actions-self-hosted-runners/arc/helm.tf |2 +- .../arc/images/Dockerfile
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 156fb1b0cc8 Updating config from bot
156fb1b0cc8 is described below
commit 156fb1b0cc8a10db8badd5559f1bd34c77d75239
Author: github-actions
AuthorDate: Mon Jul 10 15:06:00 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27427.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27427.json
b/scripts/ci/pr-bot/state/pr-state/pr-27427.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27427.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 46b8be424eb Updating config from bot
46b8be424eb is described below
commit 46b8be424ebd1d32ac4410558e0d4124b6982a16
Author: github-actions
AuthorDate: Mon Jul 10 15:06:02 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index dd5f2675669..a101c025665 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1688645641369,
+"damccorm": 1689001558041,
"Abacn": 1688991239926
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d827da27eef Updating config from bot d827da27eef is described below commit d827da27eef5fe51e4ee89044c20df2d4af3d134 Author: github-actions AuthorDate: Mon Jul 10 14:42:45 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27413.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27413.json b/scripts/ci/pr-bot/state/pr-state/pr-27413.json index 04679178cab..b50c07fa0bd 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27413.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27413.json @@ -5,7 +5,7 @@ "io": "bvolpato" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch master updated (e809912fa30 -> 88e635a2b0a)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from e809912fa30 Fix a typo in the GenAI notebook (#27418) add 88e635a2b0a [Python] Make BigQuery load job names deterministic for retry resilience (#27384) No new revisions were added by this update. Summary of changes: .../apache_beam/io/gcp/bigquery_file_loads.py | 21 +--- .../apache_beam/io/gcp/bigquery_file_loads_test.py | 56 +++--- sdks/python/apache_beam/io/gcp/bigquery_tools.py | 2 +- 3 files changed, 44 insertions(+), 35 deletions(-)
[beam] branch release-2.49.0 updated: Ensure CancellableQueue doesn't keep references to logically removed objects. (#27403) (#27419)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch release-2.49.0
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/release-2.49.0 by this push:
new 6ce85e5f8ae Ensure CancellableQueue doesn't keep references to
logically removed objects. (#27403) (#27419)
6ce85e5f8ae is described below
commit 6ce85e5f8ae908024c2b22afd06901551ce72a49
Author: Yi Hu
AuthorDate: Mon Jul 10 10:26:49 2023 -0400
Ensure CancellableQueue doesn't keep references to logically removed
objects. (#27403) (#27419)
Co-authored-by: scwhittle
---
.../org/apache/beam/sdk/fn/CancellableQueue.java | 26 +---
.../apache/beam/sdk/fn/CancellableQueueTest.java | 28 ++
2 files changed, 46 insertions(+), 8 deletions(-)
diff --git
a/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/CancellableQueue.java
b/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/CancellableQueue.java
index 96b0f8b9a0d..77fd73f5655 100644
---
a/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/CancellableQueue.java
+++
b/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/CancellableQueue.java
@@ -22,9 +22,9 @@ import static
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Prec
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
-import javax.annotation.Nullable;
import javax.annotation.concurrent.ThreadSafe;
import org.checkerframework.checker.nullness.qual.NonNull;
+import org.checkerframework.checker.nullness.qual.Nullable;
/**
* A simplified {@link ThreadSafe} blocking queue that can be cancelled
freeing any blocked {@link
@@ -36,7 +36,7 @@ import org.checkerframework.checker.nullness.qual.NonNull;
public class CancellableQueue {
private final int capacity;
- private final Object[] elements;
+ private final @Nullable Object[] elements;
private final Lock lock;
private final Condition notFull;
private final Condition notEmpty;
@@ -86,8 +86,9 @@ public class CancellableQueue {
* must invoke {@link #cancel} if the interrupt is unrecoverable.
* @throws Exception if the queue is cancelled.
*/
+ @SuppressWarnings({"cast"})
public T take() throws Exception, InterruptedException {
-Object rval;
+T rval;
try {
lock.lockInterruptibly();
while (count == 0 && cancellationException == null) {
@@ -97,14 +98,15 @@ public class CancellableQueue {
throw cancellationException;
}
- rval = elements[takeIndex];
+ rval = (T) elements[takeIndex];
+ elements[takeIndex] = null;
takeIndex = (takeIndex + 1) % elements.length;
count -= 1;
notFull.signal();
} finally {
lock.unlock();
}
-return (T) rval;
+return rval;
}
/**
@@ -119,6 +121,7 @@ public class CancellableQueue {
try {
if (cancellationException == null) {
cancellationException = exception;
+clearElementsLocked();
}
notEmpty.signalAll();
notFull.signalAll();
@@ -127,14 +130,21 @@ public class CancellableQueue {
}
}
+ private void clearElementsLocked() {
+for (int i = takeIndex; count > 0; i = (i + 1) % elements.length) {
+ elements[i] = null;
+ --count;
+}
+addIndex = 0;
+takeIndex = 0;
+ }
+
/** Enables the queue to be re-used after it has been cancelled. */
public void reset() {
lock.lock();
try {
cancellationException = null;
- addIndex = 0;
- takeIndex = 0;
- count = 0;
+ clearElementsLocked();
} finally {
lock.unlock();
}
diff --git
a/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/CancellableQueueTest.java
b/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/CancellableQueueTest.java
index b8b577b1805..0e3ae17e50f 100644
---
a/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/CancellableQueueTest.java
+++
b/sdks/java/fn-execution/src/test/java/org/apache/beam/sdk/fn/CancellableQueueTest.java
@@ -21,7 +21,9 @@ import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.empty;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThrows;
+import static org.junit.Assert.assertTrue;
+import java.lang.ref.WeakReference;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
@@ -219,4 +221,30 @@ public class CancellableQueueTest {
queue.cancel(new RuntimeException("Second cancel exception"));
assertThrows("First cancel exception", RuntimeException.class, () ->
queue.take());
}
+
+ @Test
+ public void testMemoryReferenceOnTake() throws Exception {
+String s1 = new String("test1");
+String s2 = new String("test2");
+WeakReference weakReference1 = new WeakReference<>(s1);
+
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4c4de2fec80 Updating config from bot
4c4de2fec80 is described below
commit 4c4de2fec80da8beee8005947054ac9e78e49993
Author: github-actions
AuthorDate: Mon Jul 10 12:46:12 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
index d31f0b7886f..bccd76b130a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"Abacn": 162783170,
"AnandInguva": 1688938738642,
-"chamikaramj": 1688054778539,
+"chamikaramj": 1688993168560,
"damccorm": 1688168089096,
"johnjcasey": 1688249155619,
"jrmccluskey": 1688603128711,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c2b2990a814 Updating config from bot
c2b2990a814 is described below
commit c2b2990a81419a808acb69abb6110982e2268714
Author: github-actions
AuthorDate: Mon Jul 10 12:46:10 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27424.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27424.json
b/scripts/ci/pr-bot/state/pr-state/pr-27424.json
new file mode 100644
index 000..26e4be5dc8a
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27424.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "chamikaramj"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4563bf95615 Updating config from bot
4563bf95615 is described below
commit 4563bf9561586e49ec37a0579a325584aed07d4e
Author: github-actions
AuthorDate: Mon Jul 10 12:14:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 2178e14badf..dd5f2675669 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1688645641369,
-"Abacn": 1688659588960
+"Abacn": 1688991239926
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 350da5ff75d Updating config from bot
350da5ff75d is described below
commit 350da5ff75da9684d086c13e2a1e5d853692def7
Author: github-actions
AuthorDate: Mon Jul 10 12:14:01 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27239.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27239.json
b/scripts/ci/pr-bot/state/pr-state/pr-27239.json
index 8a92900947e..9c51466b9ee 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27239.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27239.json
@@ -1,7 +1,7 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"build": "damccorm"
+"build": "Abacn"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
