[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8ad921de618 Updating config from bot 8ad921de618 is described below commit 8ad921de6189ff22177a2886517d1d4e9ebca634 Author: github-actions AuthorDate: Thu Jul 13 05:06:26 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27479.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27479.json b/scripts/ci/pr-bot/state/pr-state/pr-27479.json index 55cd2385442..2403b9b4513 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27479.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27479.json @@ -7,5 +7,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new cd935ae5725 Updating config from bot cd935ae5725 is described below commit cd935ae5725239e92454f0d08d8b1c62bb7243f3 Author: github-actions AuthorDate: Thu Jul 13 05:06:24 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 37fefeec64d..96c76ad38af 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1689053694130, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1688967312701 +"riteshghorse": 1689224780802 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1423a383dc2 Updating config from bot
1423a383dc2 is described below
commit 1423a383dc2cf391eaaf7735e512cc3c7eaf40b2
Author: github-actions
AuthorDate: Thu Jul 13 05:06:22 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27481.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27481.json
b/scripts/ci/pr-bot/state/pr-state/pr-27481.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27481.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch nightly-refs/heads/master updated (c9fe1586254 -> 0fb46f0e28d)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from c9fe1586254 Remove ValidatesRunner_V1 suite (#26543) add 7f063ae1cb9 fixes issue with pagination when PR has more than 30 jobs (#27468) add 406e115d66b adding runner image 738c02cac (#27441) add a154c176a5a Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript add 48dacc25463 Merge pull request #27404: Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript add 843e7fdb239 Some more runner v2 cleanup (#27464) add 30b1b8c1d72 Allow use of private endpoints for Vertex AI requests (#27458) add b5486cf37a3 Remove obsolete native (dataflow) io from Beam Python. (#27444) add 75644d8f5fa Fix BigQuery batch insertion loop (#27477) add 0fb46f0e28d [#27435] Describe the Source and Sink transform and urns in the protos. (#27437) No new revisions were added by this update. Summary of changes: .github/actions/rerun-job-action/action.yml| 4 +- .../arc/environments/beam.env | 6 +- build.gradle.kts | 1 - .../beam/model/fn_execution/v1/beam_fn_api.proto | 46 +++ sdks/go/pkg/beam/io/bigqueryio/bigquery.go | 5 +- sdks/python/apache_beam/io/gcp/pubsub.py | 26 +- sdks/python/apache_beam/io/iobase.py | 43 +-- .../ml/inference/vertex_ai_inference.py| 20 +- sdks/python/apache_beam/pipeline_test.py | 53 +--- .../runners/dataflow/native_io/__init__.py | 16 - .../runners/dataflow/native_io/iobase.py | 342 - .../runners/dataflow/native_io/iobase_test.py | 203 .../apache_beam/runners/direct/direct_runner.py| 17 +- .../runners/direct/transform_evaluator.py | 82 + .../runners/worker/bundle_processor_test.py| 1 - sdks/python/test-suites/dataflow/build.gradle | 7 - sdks/python/test-suites/dataflow/common.gradle | 19 +- sdks/typescript/package-lock.json | 171 +++ sdks/typescript/package.json | 2 +- 19 files changed, 231 insertions(+), 833 deletions(-) delete mode 100644 sdks/python/apache_beam/runners/dataflow/native_io/__init__.py delete mode 100644 sdks/python/apache_beam/runners/dataflow/native_io/iobase.py delete mode 100644 sdks/python/apache_beam/runners/dataflow/native_io/iobase_test.py
[beam] branch dependabot/go_modules/sdks/google.golang.org/api-0.131.0 created (now 7d9ed9610c6)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/google.golang.org/api-0.131.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 7d9ed9610c6 Bump google.golang.org/api from 0.130.0 to 0.131.0 in /sdks No new revisions were added by this update.
[beam] branch asf-site updated: Publishing website 2023/07/13 04:15:58 at commit 0fb46f0
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new eb27f11357b Publishing website 2023/07/13 04:15:58 at commit 0fb46f0 eb27f11357b is described below commit eb27f11357bd27523f84303377caf754e8fa2c8a Author: jenkins AuthorDate: Thu Jul 13 04:15:58 2023 + Publishing website 2023/07/13 04:15:58 at commit 0fb46f0 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 4825955adbe..4b904bfd1f7 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/categories/blog/2023-07-12T13:16:36-07:00/blog/2023-07-12T13:16:36-07:00/categories/2023-07-12T13:16:36-07:00/blog/managing-beam-dependencies-in-java/2023-07-12T13:16:36-07:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/categories/blog/2023-07-12T19:37:03-07:00/blog/2023-07-12T19:37:03-07:00/categories/2023-07-12T19:37:03-07:00/blog/managing-beam-dependencies-in-java/2023-07-12T19:37:03-07:00 [...] \ No newline at end of file
[beam] branch master updated: [#27435] Describe the Source and Sink transform and urns in the protos. (#27437)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 0fb46f0e28d [#27435] Describe the Source and Sink transform and urns
in the protos. (#27437)
0fb46f0e28d is described below
commit 0fb46f0e28d4ba14a66889fed50d540d13042289
Author: Robert Burke
AuthorDate: Wed Jul 12 19:37:03 2023 -0700
[#27435] Describe the Source and Sink transform and urns in the protos.
(#27437)
* [#27435] Describe source and sink transforms.
* Add underscores.
* re-order because
* Move to fn_api.proto
* rm space
* Describe the enum messages.
* import beam_urn correctly
* robertwb's pr comments & suggestions
-
Co-authored-by: lostluck <13907733+lostl...@users.noreply.github.com>
---
.../beam/model/fn_execution/v1/beam_fn_api.proto | 46 ++
1 file changed, 46 insertions(+)
diff --git
a/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
b/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
index efa37ea996e..66d144ab231 100644
---
a/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
+++
b/model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto
@@ -46,6 +46,52 @@ import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/duration.proto";
+
+// Describes transforms necessary to execute Beam over the FnAPI but are
+// implementation details rather than part of the core model.
+message FnApiTransforms {
+ enum Runner {
+// DataSource is a Root Transform, and a source of data for downstream
+// transforms in the same ProcessBundleDescriptor.
+// It represents a stream of values coming in from an external source/over
+// a data channel, typically from the runner. It's not the PCollection
itself
+// but a description of how to get the portion of the PCollection for a
given
+// bundle.
+//
+// The DataSource transform is implemented in each SDK and not explicitly
+// provided during pipeline construction. A runner inserts the transform
+// in ProcessBundleDescriptors to indicate where the bundle
+// can retrieve data for an associated ProcessBundleRequest.
+// Data for the same request will be retrieved with the matching
instruction ID,
+// and transform ID determined by the runner.
+//
+// The DataSource transform will take a stream of bytes from the remote
+// source for the matching instruction ID and decode them as windowed
+// values using the provided coder ID, which must be a windowed value
coder.
+//
+// Payload: RemoteGrpcPort
+DATA_SOURCE = 0 [(org.apache.beam.model.pipeline.v1.beam_urn) =
"beam:runner:source:v1"];
+
+// DataSink is a transform that sends PCollection elements to a remote
+// port using the Data API.
+//
+// The DataSink transform is implemented in each SDK and not explicitly
+// provided during pipeline construction. A runner inserts the transform in
+// ProcessBundleDescriptors to indicate where the bundle can send
+// data for each associated ProcessBundleRequest. Data for the same
+// request will be sent with the matching instruction ID and transform ID.
+// Each PCollection that exits the ProcessBundleDescriptor subgraph will
have
+// it's own DataSink, keyed by a transform ID determined by the runner.
+//
+// The DataSink will take in a stream of elements for a given instruction
ID
+// and encode them for transmission to the remote sink. The coder ID must
be
+// for a windowed value coder.
+//
+// Payload: RemoteGrpcPort
+DATA_SINK = 1 [(org.apache.beam.model.pipeline.v1.beam_urn) =
"beam:runner:sink:v1"];
+ }
+}
+
// A descriptor for connecting to a remote port using the Beam Fn Data API.
// Allows for communication between two environments (for example between the
// runner and the SDK).
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c79f98944a2 Updating config from bot c79f98944a2 is described below commit c79f98944a2799a695211fa4b9d886df9b0c4f49 Author: github-actions AuthorDate: Thu Jul 13 01:35:44 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index f07171a7cf9..0d1f79bc329 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -6,7 +6,7 @@ "pabloem": 1689190474793, "Abacn": 1689077653560, "ahmedabu98": 1689203171737, -"bvolpato": 1688896598481, +"bvolpato": 1689212138308, "manavgarg": 1689186972147 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new beec9378189 Updating config from bot beec9378189 is described below commit beec9378189da5b517cb50f7aee3af04f7814374 Author: github-actions AuthorDate: Thu Jul 13 01:35:42 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index a3b5efc19d1..bb6257e8fcc 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -9,7 +9,7 @@ "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1689087964242, -"jrmccluskey": 1689086132282, +"jrmccluskey": 1689212138307, "riteshghorse": 1689174359250, "liferoad": 1689186972147 }
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 95e03a1da5b Updating config from bot
95e03a1da5b is described below
commit 95e03a1da5ba60e8a0d768fd1be94991ec1041c0
Author: github-actions
AuthorDate: Thu Jul 13 01:35:40 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27479.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27479.json
b/scripts/ci/pr-bot/state/pr-state/pr-27479.json
new file mode 100644
index 000..55cd2385442
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27479.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "jrmccluskey",
+"io": "bvolpato"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Fix BigQuery batch insertion loop (#27477)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 75644d8f5fa Fix BigQuery batch insertion loop (#27477)
75644d8f5fa is described below
commit 75644d8f5faf12ad4259880ab1352def8ecc0905
Author: Mahmud Ridwan
AuthorDate: Thu Jul 13 05:33:42 2023 +0600
Fix BigQuery batch insertion loop (#27477)
---
sdks/go/pkg/beam/io/bigqueryio/bigquery.go | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/sdks/go/pkg/beam/io/bigqueryio/bigquery.go
b/sdks/go/pkg/beam/io/bigqueryio/bigquery.go
index 4ca64be8780..12beacd4a01 100644
--- a/sdks/go/pkg/beam/io/bigqueryio/bigquery.go
+++ b/sdks/go/pkg/beam/io/bigqueryio/bigquery.go
@@ -337,10 +337,9 @@ func (f *writeFn) ProcessElement(ctx context.Context, _
int, iter func(*beam.X)
}
data = nil
size = writeOverheadBytes
- } else {
- data = append(data, reflect.ValueOf(val.(any)))
- size += current
}
+ data = append(data, reflect.ValueOf(val.(any)))
+ size += current
}
if len(data) == 0 {
return nil
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f5c59f98294 Updating config from bot
f5c59f98294 is described below
commit f5c59f98294d109b992fce65a3eba28e3e2df40f
Author: github-actions
AuthorDate: Wed Jul 12 23:06:14 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27474.json | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27474.json
b/scripts/ci/pr-bot/state/pr-state/pr-27474.json
index 242a48d7d3b..4f67ed6553e 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-27474.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27474.json
@@ -1,7 +1,10 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "AnandInguva",
+"io": "ahmedabu98"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 43a965520e9 Updating config from bot 43a965520e9 is described below commit 43a965520e93a385cc1119562594cd5d2343533d Author: github-actions AuthorDate: Wed Jul 12 23:06:18 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index 4cb167ca126..f07171a7cf9 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -5,7 +5,7 @@ "johnjcasey": 1689017692295, "pabloem": 1689190474793, "Abacn": 1689077653560, -"ahmedabu98": 1688585680741, +"ahmedabu98": 1689203171737, "bvolpato": 1688896598481, "manavgarg": 1689186972147 }
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4f538a7acbf Updating config from bot
4f538a7acbf is described below
commit 4f538a7acbf3683f1b749f6a414d1f3fefb61254
Author: github-actions
AuthorDate: Wed Jul 12 23:06:16 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 01e1c340014..a3b5efc19d1 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -1,7 +1,7 @@
{
"label": "python",
"dateOfLastReviewAssignment": {
-"AnandInguva": 1689055617055,
+"AnandInguva": 1689203171737,
"yeandy": 1665802753763,
"TheNeuralBit": 1667896849319,
"ryanthompson591": 1670002443548,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 706e8e70ec6 Updating config from bot 706e8e70ec6 is described below commit 706e8e70ec6e9f472b8a827e41e699395c8ff9eb Author: github-actions AuthorDate: Wed Jul 12 22:35:24 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27477.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27477.json b/scripts/ci/pr-bot/state/pr-state/pr-27477.json index d994c98344d..bfa316fff79 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27477.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27477.json @@ -7,5 +7,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch asf-site updated: Publishing website 2023/07/12 22:17:26 at commit b5486cf
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 23041943e75 Publishing website 2023/07/12 22:17:26 at commit b5486cf 23041943e75 is described below commit 23041943e75c39bef04b7423ffc87d3071d22ad1 Author: jenkins AuthorDate: Wed Jul 12 22:17:26 2023 + Publishing website 2023/07/12 22:17:26 at commit b5486cf --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 1d84d383de9..4825955adbe 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/categories/blog/2023-07-12T12:03:55-04:00/blog/2023-07-12T12:03:55-04:00/categories/2023-07-12T12:03:55-04:00/blog/managing-beam-dependencies-in-java/2023-07-12T12:03:55-04:00 [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/categories/blog/2023-07-12T13:16:36-07:00/blog/2023-07-12T13:16:36-07:00/categories/2023-07-12T13:16:36-07:00/blog/managing-beam-dependencies-in-java/2023-07-12T13:16:36-07:00 [...] \ No newline at end of file
[beam] branch users/damccorm/v2-removal updated (57ee522ee54 -> 69045265ca2)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/v2-removal in repository https://gitbox.apache.org/repos/asf/beam.git from 57ee522ee54 Add changes.md entry add 69045265ca2 Lint No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/transforms/combinefn_lifecycle_test.py | 2 -- 1 file changed, 2 deletions(-)
[beam-starter-go] branch main updated: use latest version of go
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git The following commit(s) were added to refs/heads/main by this push: new fe2937e use latest version of go new e8b683f Merge pull request #17 from case-k-git/feature/update-go-version fe2937e is described below commit fe2937e847d660b5ae3f212c9e68d66515934c04 Author: case-k-git AuthorDate: Mon Jul 3 08:22:31 2023 +0900 use latest version of go --- go.mod | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/go.mod b/go.mod index f4e9106..372599a 100644 --- a/go.mod +++ b/go.mod @@ -8,7 +8,7 @@ module example.com/my_app -go 1.18 +go 1.20 require github.com/apache/beam/sdks/v2 v2.48.2
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5f272971ce5 Updating config from bot
5f272971ce5 is described below
commit 5f272971ce53c74f6425d5219bb91f8d1376b226
Author: github-actions
AuthorDate: Wed Jul 12 20:34:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27472.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27472.json
b/scripts/ci/pr-bot/state/pr-state/pr-27472.json
new file mode 100644
index 000..8ad90f2eea6
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27472.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "kennknowles"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 59a7eb88b91 Updating config from bot 59a7eb88b91 is described below commit 59a7eb88b9187a2cfddc2517667320d98bd972e7 Author: github-actions AuthorDate: Wed Jul 12 20:34:57 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index ff0ad8a0258..2459791f77e 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -7,7 +7,7 @@ "damccorm": 1689129180115, "johnjcasey": 1689133059377, "jrmccluskey": 1688603128711, -"kennknowles": 1688771157829, +"kennknowles": 1689194093691, "lostluck": 1688794478849, "riteshghorse": 1688846765068, "robertwb": 1688857585947,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 32c4621c0fb Updating config from bot
32c4621c0fb is described below
commit 32c4621c0fb28075c2d0cc5ff94d327a304ede83
Author: github-actions
AuthorDate: Wed Jul 12 20:34:51 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27474.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27474.json
b/scripts/ci/pr-bot/state/pr-state/pr-27474.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27474.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Remove obsolete native (dataflow) io from Beam Python. (#27444)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new b5486cf37a3 Remove obsolete native (dataflow) io from Beam Python.
(#27444)
b5486cf37a3 is described below
commit b5486cf37a31863da9ce4837b6547628031d1bd3
Author: Robert Bradshaw
AuthorDate: Wed Jul 12 13:16:36 2023 -0700
Remove obsolete native (dataflow) io from Beam Python. (#27444)
This was only needed for Runner v1.
---
sdks/python/apache_beam/io/gcp/pubsub.py | 26 +-
sdks/python/apache_beam/io/iobase.py | 43 +--
sdks/python/apache_beam/pipeline_test.py | 53 +---
.../runners/dataflow/native_io/__init__.py | 16 -
.../runners/dataflow/native_io/iobase.py | 342 -
.../runners/dataflow/native_io/iobase_test.py | 203
.../apache_beam/runners/direct/direct_runner.py| 17 +-
.../runners/direct/transform_evaluator.py | 82 +
.../runners/worker/bundle_processor_test.py| 1 -
9 files changed, 45 insertions(+), 738 deletions(-)
diff --git a/sdks/python/apache_beam/io/gcp/pubsub.py
b/sdks/python/apache_beam/io/gcp/pubsub.py
index 8cee8acfebb..af58006d6e7 100644
--- a/sdks/python/apache_beam/io/gcp/pubsub.py
+++ b/sdks/python/apache_beam/io/gcp/pubsub.py
@@ -39,9 +39,9 @@ from typing import Optional
from typing import Tuple
from apache_beam import coders
+from apache_beam.io import iobase
from apache_beam.io.iobase import Read
from apache_beam.io.iobase import Write
-from apache_beam.runners.dataflow.native_io import iobase as dataflow_io
from apache_beam.transforms import Flatten
from apache_beam.transforms import Map
from apache_beam.transforms import PTransform
@@ -261,6 +261,7 @@ class ReadFromPubSub(PTransform):
timestamp_attribute=timestamp_attribute)
def expand(self, pvalue):
+# TODO(BEAM-27443): Apply a proper transform rather than Read.
pcoll = pvalue.pipeline | Read(self._source)
pcoll.element_type = bytes
if self.with_attributes:
@@ -423,7 +424,8 @@ def parse_subscription(full_subscription):
return project, subscription_name
-class _PubSubSource(dataflow_io.NativeSource):
+# TODO(BEAM-27443): Remove (or repurpose as a proper PTransform).
+class _PubSubSource(iobase.SourceBase):
"""Source for a Cloud Pub/Sub topic or subscription.
This ``NativeSource`` is overridden by a native Pubsub implementation.
@@ -460,11 +462,6 @@ class _PubSubSource(dataflow_io.NativeSource):
if subscription:
self.project, self.subscription_name = parse_subscription(subscription)
- @property
- def format(self):
-"""Source format name required for remote execution."""
-return 'pubsub'
-
def display_data(self):
return {
'id_label': DisplayDataItem(self.id_label,
@@ -480,14 +477,15 @@ class _PubSubSource(dataflow_io.NativeSource):
label='Timestamp Attribute').drop_if_none(),
}
- def reader(self):
-raise NotImplementedError
+ def default_output_coder(self):
+return self.coder
def is_bounded(self):
return False
-class _PubSubSink(dataflow_io.NativeSink):
+# TODO(BEAM-27443): Remove in favor of a proper WriteToPubSub transform.
+class _PubSubSink(object):
"""Sink for a Cloud Pub/Sub topic.
This ``NativeSource`` is overridden by a native Pubsub implementation.
@@ -505,14 +503,6 @@ class _PubSubSink(dataflow_io.NativeSink):
self.project, self.topic_name = parse_topic(topic)
- @property
- def format(self):
-"""Sink format name required for remote execution."""
-return 'pubsub'
-
- def writer(self):
-raise NotImplementedError
-
class PubSubSourceDescriptor(NamedTuple):
"""A PubSub source descriptor for ``MultipleReadFromPubSub```
diff --git a/sdks/python/apache_beam/io/iobase.py
b/sdks/python/apache_beam/io/iobase.py
index 6d75d520af5..e15205ead4d 100644
--- a/sdks/python/apache_beam/io/iobase.py
+++ b/sdks/python/apache_beam/io/iobase.py
@@ -101,6 +101,9 @@ class SourceBase(HasDisplayData, urns.RunnerApiFn):
"""
urns.RunnerApiFn.register_pickle_urn(python_urns.PICKLED_SOURCE)
+ def default_output_coder(self):
+raise NotImplementedError
+
def is_bounded(self):
# type: () -> bool
raise NotImplementedError
@@ -923,11 +926,8 @@ class Read(ptransform.PTransform):
def _infer_output_coder(self, input_type=None, input_coder=None):
# type: (...) -> Optional[coders.Coder]
-from apache_beam.runners.dataflow.native_io import iobase as dataflow_io
-if isinstance(self.source, BoundedSource):
+if isinstance(self.source, SourceBase):
return self.source.default_output_coder()
-elif isinstance(self.source, dataflow_io.NativeSource):
- return self.source.coder
else:
return None
@@ -941,18 +941,17 @@ class Read(ptr
[beam] branch users/damccorm/v2-removal updated (aa735cec94e -> 57ee522ee54)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/v2-removal in repository https://gitbox.apache.org/repos/asf/beam.git from aa735cec94e Finish removing v1/v2 splits add 57ee522ee54 Add changes.md entry No new revisions were added by this update. Summary of changes: CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch users/damccorm/v2-removal created (now aa735cec94e)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/v2-removal in repository https://gitbox.apache.org/repos/asf/beam.git at aa735cec94e Finish removing v1/v2 splits This branch includes the following new commits: new aa735cec94e Finish removing v1/v2 splits The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Finish removing v1/v2 splits
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/v2-removal
in repository https://gitbox.apache.org/repos/asf/beam.git
commit aa735cec94e520fb895c22694df82f335b83eac6
Author: Danny McCormick
AuthorDate: Wed Jul 12 16:01:46 2023 -0400
Finish removing v1/v2 splits
---
...stCommit_Python_ValidatesRunner_Dataflow.groovy | 4 ++--
.../runinference_metrics/pipeline/options.py | 1 -
.../apache_beam/io/gcp/bigquery_read_it_test.py| 5 +---
.../python/apache_beam/options/pipeline_options.py | 3 +--
.../runners/dataflow/dataflow_metrics_test.py | 1 -
.../transforms/combinefn_lifecycle_test.py | 22 --
sdks/python/scripts/run_integration_test.sh| 27 --
sdks/python/test-suites/dataflow/build.gradle | 8 +++
sdks/python/test-suites/dataflow/common.gradle | 11 -
sdks/python/test-suites/gradle.properties | 4 ++--
10 files changed, 10 insertions(+), 76 deletions(-)
diff --git
a/.test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow.groovy
b/.test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow.groovy
index 13e78617d2c..db052a0046c 100644
--- a/.test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow.groovy
+++ b/.test-infra/jenkins/job_PostCommit_Python_ValidatesRunner_Dataflow.groovy
@@ -36,8 +36,8 @@
PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_VR_Dataflow', 'Run Python
steps {
gradle {
rootBuildScriptDir(commonJobProperties.checkoutDir)
-
tasks(':sdks:python:test-suites:dataflow:validatesRunnerBatchTestsV2')
-
tasks(':sdks:python:test-suites:dataflow:validatesRunnerStreamingTestsV2')
+ tasks(':sdks:python:test-suites:dataflow:validatesRunnerBatchTests')
+
tasks(':sdks:python:test-suites:dataflow:validatesRunnerStreamingTests')
switches('-PuseWheelDistribution')
commonJobProperties.setGradleSwitches(delegate)
}
diff --git
a/sdks/python/apache_beam/examples/inference/runinference_metrics/pipeline/options.py
b/sdks/python/apache_beam/examples/inference/runinference_metrics/pipeline/options.py
index b32200ed733..1966100430e 100644
---
a/sdks/python/apache_beam/examples/inference/runinference_metrics/pipeline/options.py
+++
b/sdks/python/apache_beam/examples/inference/runinference_metrics/pipeline/options.py
@@ -61,7 +61,6 @@ def get_pipeline_options(
flags = [
"--experiment=worker_accelerator=type:nvidia-tesla-p4;count:1;"\
"install-nvidia-driver",
-"--experiment=use_runner_v2",
]
dataflow_options.update({
"sdk_container_image": cfg.DOCKER_IMG,
diff --git a/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py
b/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py
index 248e0849cdd..0672cd5ea16 100644
--- a/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py
+++ b/sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py
@@ -792,10 +792,7 @@ class ReadAllBQTests(BigQueryReadIntegrationTests):
@skip(['PortableRunner', 'FlinkRunner'])
@pytest.mark.it_postcommit
def test_read_queries(self):
-# TODO(https://github.com/apache/beam/issues/20610): Remove experiment when
-# tests run on r_v2.
-args = self.args + ["--experiments=use_runner_v2"]
-with beam.Pipeline(argv=args) as p:
+with beam.Pipeline(argv=self.args) as p:
result = (
p
| beam.Create([
diff --git a/sdks/python/apache_beam/options/pipeline_options.py
b/sdks/python/apache_beam/options/pipeline_options.py
index d56b464e71c..d36b335ae29 100644
--- a/sdks/python/apache_beam/options/pipeline_options.py
+++ b/sdks/python/apache_beam/options/pipeline_options.py
@@ -1151,8 +1151,7 @@ class DebugOptions(PipelineOptions):
help=(
'Number of threads per worker to use on the runner. If left '
'unspecified, the runner will compute an appropriate number of '
-'threads to use. Currently only enabled for DataflowRunner when '
-'experiment \'use_runner_v2\' is enabled.'))
+'threads to use.'))
def add_experiment(self, experiment):
# pylint: disable=access-member-before-definition
diff --git a/sdks/python/apache_beam/runners/dataflow/dataflow_metrics_test.py
b/sdks/python/apache_beam/runners/dataflow/dataflow_metrics_test.py
index 06e1585ffc4..86e71f9c1ed 100644
--- a/sdks/python/apache_beam/runners/dataflow/dataflow_metrics_test.py
+++ b/sdks/python/apache_beam/runners/dataflow/dataflow_metrics_test.py
@@ -503,7 +503,6 @@ class TestDataflowMetrics(unittest.TestCase):
self.ONLY_COUNTERS_LIST)
pipeline_options = PipelineOptions([
-'--experiments=use_runner_v2',
'--experiments=use_portable_job_submission',
'--temp_location=gs://any-location/temp',
'--project=dummy_project',
diff --git a/sdks/python/apache_beam
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c418b0dd84e Updating config from bot
c418b0dd84e is described below
commit c418b0dd84e6cdf413eeab011f2c935a681207a5
Author: github-actions
AuthorDate: Wed Jul 12 19:34:38 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index b3d0af35880..37fefeec64d 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1688965554372,
+"lostluck": 1689190474793,
"jrmccluskey": 1689053694130,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e19700ad1ab Updating config from bot
e19700ad1ab is described below
commit e19700ad1abad145ee1432d1907cc8638d7215b7
Author: github-actions
AuthorDate: Wed Jul 12 19:34:39 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index e1f07c9b50a..4cb167ca126 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"chamikaramj": 1689042646484,
"johnjcasey": 1689017692295,
-"pabloem": 1688472842104,
+"pabloem": 1689190474793,
"Abacn": 1689077653560,
"ahmedabu98": 1688585680741,
"bvolpato": 1688896598481,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e029800de4c Updating config from bot
e029800de4c is described below
commit e029800de4cf63194e1981cec295d23e15c3721a
Author: github-actions
AuthorDate: Wed Jul 12 19:34:36 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27477.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27477.json
b/scripts/ci/pr-bot/state/pr-state/pr-27477.json
new file mode 100644
index 000..d994c98344d
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27477.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck",
+"io": "pabloem"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Allow use of private endpoints for Vertex AI requests (#27458)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 30b1b8c1d72 Allow use of private endpoints for Vertex AI requests
(#27458)
30b1b8c1d72 is described below
commit 30b1b8c1d72243b51104deff435539c7dee3fd3c
Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com>
AuthorDate: Wed Jul 12 15:28:12 2023 -0400
Allow use of private endpoints for Vertex AI requests (#27458)
---
.../apache_beam/ml/inference/vertex_ai_inference.py | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/sdks/python/apache_beam/ml/inference/vertex_ai_inference.py
b/sdks/python/apache_beam/ml/inference/vertex_ai_inference.py
index 34fd6d27448..a0e0d9d3f8f 100644
--- a/sdks/python/apache_beam/ml/inference/vertex_ai_inference.py
+++ b/sdks/python/apache_beam/ml/inference/vertex_ai_inference.py
@@ -66,6 +66,7 @@ class VertexAIModelHandlerJSON(ModelHandler[Any,
project: str,
location: str,
experiment: Optional[str] = None,
+ network: Optional[str] = None,
**kwargs):
"""Implementation of the ModelHandler interface for Vertex AI.
**NOTE:** This API and its implementation are under development and
@@ -73,19 +74,30 @@ class VertexAIModelHandlerJSON(ModelHandler[Any,
Unlike other ModelHandler implementations, this does not load the model
being used onto the worker and instead makes remote queries to a
Vertex AI endpoint. In that way it functions more like a mid-pipeline
-IO. At present this implementation only supports public endpoints with
-a maximum request size of 1.5 MB.
+IO. Public Vertex AI endpoints have a maximum request size of 1.5 MB.
+If you wish to make larger requests and use a private endpoint, provide
+the Compute Engine network you wish to use.
+
Args:
endpoint_id: the numerical ID of the Vertex AI endpoint to query
project: the GCP project name where the endpoint is deployed
location: the GCP location where the endpoint is deployed
- experiment (Optional): experiment label to apply to the queries
+ experiment: optional. experiment label to apply to the
+queries
+ network: optional. the full name of the Compute Engine
+network the endpoint is deployed on; used for private
+endpoints only.
+Ex: "projects/12345/global/networks/myVPC"
"""
self._env_vars = kwargs.get('env_vars', {})
# TODO: support the full list of options for aiplatform.init()
# See
https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform#google_cloud_aiplatform_init
-aiplatform.init(project=project, location=location, experiment=experiment)
+aiplatform.init(
+project=project,
+location=location,
+experiment=experiment,
+network=network)
# Check for liveness here but don't try to actually store the endpoint
# in the class yet
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 42b806acf38 Updating config from bot 42b806acf38 is described below commit 42b806acf381722afe8e8c171f7643d5d8d424d6 Author: github-actions AuthorDate: Wed Jul 12 18:36:15 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 54f79cd9098..01e1c340014 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -11,6 +11,6 @@ "damccorm": 1689087964242, "jrmccluskey": 1689086132282, "riteshghorse": 1689174359250, -"liferoad": 1689042646483 +"liferoad": 1689186972147 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f5657f805d5 Updating config from bot f5657f805d5 is described below commit f5657f805d571029a533828be7971e7818d909d6 Author: github-actions AuthorDate: Wed Jul 12 18:36:17 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index d7d2eab1f52..e1f07c9b50a 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -7,6 +7,6 @@ "Abacn": 1689077653560, "ahmedabu98": 1688585680741, "bvolpato": 1688896598481, -"manavgarg": 1688137558599 +"manavgarg": 1689186972147 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4f15e0493c9 Updating config from bot
4f15e0493c9 is described below
commit 4f15e0493c92eef4768c3f5c29d540786f1bd544
Author: github-actions
AuthorDate: Wed Jul 12 18:36:14 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27471.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27471.json
b/scripts/ci/pr-bot/state/pr-state/pr-27471.json
new file mode 100644
index 000..18f3e1a6b7d
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27471.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "liferoad",
+"io": "manavgarg"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6d2dfb6f2e4 Updating config from bot 6d2dfb6f2e4 is described below commit 6d2dfb6f2e4cd0807556ffc2f4149cdbfe3c350f Author: github-actions AuthorDate: Wed Jul 12 17:11:14 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27436.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27436.json b/scripts/ci/pr-bot/state/pr-state/pr-27436.json index 76d5fbcc8d6..1368af41e91 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27436.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27436.json @@ -4,7 +4,7 @@ "python": "riteshghorse" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a06c2df2f96 Updating config from bot
a06c2df2f96 is described below
commit a06c2df2f96137f70eabfd0d71ac042c156204e9
Author: github-actions
AuthorDate: Wed Jul 12 17:06:19 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 4003008a53a..934578cf9d8 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1689075311928,
+"damccorm": 1689181574820,
"Abacn": 1689174359250
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e8f114892ea Updating config from bot
e8f114892ea is described below
commit e8f114892ea8895024cfe16d85055e145fe976c4
Author: github-actions
AuthorDate: Wed Jul 12 17:06:17 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27466.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27466.json
b/scripts/ci/pr-bot/state/pr-state/pr-27466.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27466.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch users/damccorm/v2-cleanup deleted (was 8e87f9bfb4d)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/v2-cleanup in repository https://gitbox.apache.org/repos/asf/beam.git was 8e87f9bfb4d Syntax The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated: Some more runner v2 cleanup (#27464)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 843e7fdb239 Some more runner v2 cleanup (#27464)
843e7fdb239 is described below
commit 843e7fdb239e0a61b47e39ce635c37221cd09f4e
Author: Danny McCormick
AuthorDate: Wed Jul 12 12:03:55 2023 -0400
Some more runner v2 cleanup (#27464)
* Some more runner v2 cleanup
* Fix suffix
* Remove duplicate tasks
* Remove duplicate tasks
* Syntax
---
build.gradle.kts | 1 -
sdks/python/test-suites/dataflow/build.gradle | 7 ---
sdks/python/test-suites/dataflow/common.gradle | 19 ---
3 files changed, 4 insertions(+), 23 deletions(-)
diff --git a/build.gradle.kts b/build.gradle.kts
index 2ffbc765a79..d202d9c76fb 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -472,7 +472,6 @@ tasks.register("pythonPreCommit") {
tasks.register("pythonPreCommitIT") {
dependsOn(":sdks:python:test-suites:tox:pycommon:preCommitPyCommon")
dependsOn(":sdks:python:test-suites:dataflow:preCommitIT")
- dependsOn(":sdks:python:test-suites:dataflow:preCommitIT_V2")
}
tasks.register("pythonDocsPreCommit") {
diff --git a/sdks/python/test-suites/dataflow/build.gradle
b/sdks/python/test-suites/dataflow/build.gradle
index 53f1f51dbdd..50d35774ffc 100644
--- a/sdks/python/test-suites/dataflow/build.gradle
+++ b/sdks/python/test-suites/dataflow/build.gradle
@@ -30,13 +30,6 @@ task preCommitIT {
}
}
-task preCommitIT_V2{
- getVersionsAsList('dataflow_precommit_it_task_py_versions').each {
-
dependsOn.add(":sdks:python:test-suites:dataflow:py${getVersionSuffix(it)}:preCommitIT_batch_V2")
-
dependsOn.add(":sdks:python:test-suites:dataflow:py${getVersionSuffix(it)}:preCommitIT_streaming_V2")
- }
-}
-
task mongodbioIT {
getVersionsAsList('dataflow_mongodbio_it_task_py_versions').each {
dependsOn.add(":sdks:python:test-suites:dataflow:py${getVersionSuffix(it)}:mongodbioIT")
diff --git a/sdks/python/test-suites/dataflow/common.gradle
b/sdks/python/test-suites/dataflow/common.gradle
index 36fc75f2e41..44257b09c01 100644
--- a/sdks/python/test-suites/dataflow/common.gradle
+++ b/sdks/python/test-suites/dataflow/common.gradle
@@ -69,9 +69,8 @@ def basicPytestOpts = [
"--log-cli-level=INFO", //log level
]
-def preCommitIT(String runScriptsDir, String envdir, Boolean streaming,
Boolean runnerV2, String pythonSuffix) {
- def suffix = runnerV2 ? '_V2' : ''
- suffix = streaming ? "_streaming$suffix" : "_batch$suffix"
+def preCommitIT(String runScriptsDir, String envdir, Boolean streaming, String
pythonSuffix) {
+ def suffix = streaming ? "_streaming" : "_batch"
task "preCommitIT${suffix}" {
dependsOn 'initializeForDataflowJob'
@@ -97,9 +96,6 @@ def preCommitIT(String runScriptsDir, String envdir, Boolean
streaming, Boolean
if (streaming){
argMap.put("streaming", "true")
-argMap.put("runner_v2", "true")
- } else if (runnerV2) {
-argMap.put("runner_v2", "true")
}
def cmdArgs = mapToArgString(argMap)
@@ -111,21 +107,14 @@ def preCommitIT(String runScriptsDir, String envdir,
Boolean streaming, Boolean
}
}
-preCommitIT(runScriptsDir, envdir, false, false, pythonVersionSuffix)
-preCommitIT(runScriptsDir, envdir, true, false, pythonVersionSuffix)
-preCommitIT(runScriptsDir, envdir, false, true, pythonVersionSuffix)
-preCommitIT(runScriptsDir, envdir, true, true, pythonVersionSuffix)
+preCommitIT(runScriptsDir, envdir, false, pythonVersionSuffix)
+preCommitIT(runScriptsDir, envdir, true, pythonVersionSuffix)
task preCommitIT{
dependsOn preCommitIT_batch
dependsOn preCommitIT_streaming
}
-task preCommitIT_V2{
- dependsOn preCommitIT_batch_V2
- dependsOn preCommitIT_streaming_V2
-}
-
task postCommitIT {
dependsOn 'initializeForDataflowJob'
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7c95f125355 Updating config from bot 7c95f125355 is described below commit 7c95f12535560cdbf4ebbd2818e83603c172b3b0 Author: github-actions AuthorDate: Wed Jul 12 15:35:18 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27464.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27464.json b/scripts/ci/pr-bot/state/pr-state/pr-27464.json index 4947f013b51..393fd03e2a8 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27464.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27464.json @@ -7,5 +7,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1ef56cda24c Updating config from bot
1ef56cda24c is described below
commit 1ef56cda24c96dfe1c76eb4084f457a15789f93a
Author: github-actions
AuthorDate: Wed Jul 12 15:06:06 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 8b3e31f4410..4003008a53a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1689075311928,
-"Abacn": 1689081030937
+"Abacn": 1689174359250
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 3f8ab11ca89 Updating config from bot 3f8ab11ca89 is described below commit 3f8ab11ca8986084a15bad6442fe9ca6de8e6547 Author: github-actions AuthorDate: Wed Jul 12 15:06:09 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27436.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27436.json b/scripts/ci/pr-bot/state/pr-state/pr-27436.json index c43704aa814..76d5fbcc8d6 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27436.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27436.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2cd4915fc2d Updating config from bot 2cd4915fc2d is described below commit 2cd4915fc2d32752201bdb71d430d0e3fad5a35c Author: github-actions AuthorDate: Wed Jul 12 15:06:04 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 737a9282112..54f79cd9098 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -10,7 +10,7 @@ "y1chi": 1667002607045, "damccorm": 1689087964242, "jrmccluskey": 1689086132282, -"riteshghorse": 1689024882446, +"riteshghorse": 1689174359250, "liferoad": 1689042646483 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e6e963f65ca Updating config from bot
e6e963f65ca is described below
commit e6e963f65caf8fbc249b78c8e497cd1b6b785d91
Author: github-actions
AuthorDate: Wed Jul 12 15:06:01 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27464.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27464.json
b/scripts/ci/pr-bot/state/pr-state/pr-27464.json
new file mode 100644
index 000..4947f013b51
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27464.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "riteshghorse",
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam-starter-java] branch dependabot/maven/org.apache.maven.plugins-maven-surefire-plugin-3.1.2 deleted (was efbfa1a)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch dependabot/maven/org.apache.maven.plugins-maven-surefire-plugin-3.1.2 in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git was efbfa1a Bump maven-surefire-plugin from 3.0.0-M8 to 3.1.2 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam-starter-java] branch main updated (ec49027 -> 6004184)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git from ec49027 Merge pull request #77: Bump actions/checkout from 3.5.2 to 3.5.3 add efbfa1a Bump maven-surefire-plugin from 3.0.0-M8 to 3.1.2 add 6004184 Merge pull request #76: Bump maven-surefire-plugin from 3.0.0-M8 to 3.1.2 No new revisions were added by this update. Summary of changes: pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam-starter-java] branch dependabot/github_actions/actions/checkout-3.5.3 deleted (was 7b14527)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch dependabot/github_actions/actions/checkout-3.5.3 in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git was 7b14527 Bump actions/checkout from 3.5.2 to 3.5.3 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam-starter-java] branch main updated (532e2be -> ec49027)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git from 532e2be Merge pull request #71 from apache/dependabot/maven/org.apache.maven.plugins-maven-assembly-plugin-3.6.0 add 7b14527 Bump actions/checkout from 3.5.2 to 3.5.3 new ec49027 Merge pull request #77: Bump actions/checkout from 3.5.2 to 3.5.3 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/test.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[beam-starter-java] 01/01: Merge pull request #77: Bump actions/checkout from 3.5.2 to 3.5.3
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git commit ec49027c943007f122454515a0a0b0b0dd1533b0 Merge: 532e2be 7b14527 Author: Kenn Knowles AuthorDate: Wed Jul 12 10:13:49 2023 -0400 Merge pull request #77: Bump actions/checkout from 3.5.2 to 3.5.3 .github/workflows/test.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 9d03c05981a Updating config from bot 9d03c05981a is described below commit 9d03c05981ab6d2843ca465c2fabdfad4b439b57 Author: github-actions AuthorDate: Wed Jul 12 14:00:51 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-27459.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27459.json b/scripts/ci/pr-bot/state/pr-state/pr-27459.json index f6cbd919240..473b870bb5b 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-27459.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-27459.json @@ -4,7 +4,7 @@ "no-matching-label": "damccorm" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch dependabot/npm_and_yarn/sdks/typescript/protobufjs-7.2.4 deleted (was a154c176a5a)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch dependabot/npm_and_yarn/sdks/typescript/protobufjs-7.2.4 in repository https://gitbox.apache.org/repos/asf/beam.git was a154c176a5a Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (406e115d66b -> 48dacc25463)
This is an automated email from the ASF dual-hosted git repository. kenn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 406e115d66b adding runner image 738c02cac (#27441) add a154c176a5a Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript new 48dacc25463 Merge pull request #27404: Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/typescript/package-lock.json | 171 -- sdks/typescript/package.json | 2 +- 2 files changed, 111 insertions(+), 62 deletions(-)
[beam] 01/01: Merge pull request #27404: Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript
This is an automated email from the ASF dual-hosted git repository. kenn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 48dacc25463d7c0075c159144e87bebfae61479b Merge: 406e115d66b a154c176a5a Author: Kenneth Knowles AuthorDate: Wed Jul 12 10:00:08 2023 -0400 Merge pull request #27404: Bump protobufjs from 6.11.3 to 7.2.4 in /sdks/typescript sdks/typescript/package-lock.json | 171 -- sdks/typescript/package.json | 2 +- 2 files changed, 111 insertions(+), 62 deletions(-)
[beam] branch master updated: adding runner image 738c02cac (#27441)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 406e115d66b adding runner image 738c02cac (#27441)
406e115d66b is described below
commit 406e115d66b3f20276b380c1ba35ea5b39f531b9
Author: Vlado Djerek <20559691+volatilemolo...@users.noreply.github.com>
AuthorDate: Wed Jul 12 15:58:42 2023 +0200
adding runner image 738c02cac (#27441)
* adding runner image 738c02cac
* change runner and node max
* bump main pool to 16 max
---
.github/gh-actions-self-hosted-runners/arc/environments/beam.env | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/.github/gh-actions-self-hosted-runners/arc/environments/beam.env
b/.github/gh-actions-self-hosted-runners/arc/environments/beam.env
index 48566e940a7..d99f6f3f20a 100644
--- a/.github/gh-actions-self-hosted-runners/arc/environments/beam.env
+++ b/.github/gh-actions-self-hosted-runners/arc/environments/beam.env
@@ -31,11 +31,12 @@ deploy_webhook = "true"
runner_group = "beam"
main_runner = {
name = "main-runner"
+runner_image =
"us-central1-docker.pkg.dev/apache-beam-testing/beam-github-actions/beam-arc-runner:738c02cac4d6a88faa3dd02606d50f1d2efed5e7"
machine_type = "e2-standard-16"
min_node_count = "1"
-max_node_count = "7"
+max_node_count = "16"
min_replicas = "1"
-max_replicas = "45"
+max_replicas = "128"
webhook_scaling = true
disk_size_gb = 200
requests = {
@@ -46,6 +47,7 @@ main_runner = {
additional_runner_pools = [{
name = "small-runner"
machine_type = "e2-standard-2"
+runner_image =
"us-central1-docker.pkg.dev/apache-beam-testing/beam-github-actions/beam-arc-runner:738c02cac4d6a88faa3dd02606d50f1d2efed5e7"
min_node_count = "1"
max_node_count = "10"
min_replicas = "1"
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ec029f3912c Updating config from bot
ec029f3912c is described below
commit ec029f3912cea81e8597f1d924120e534024747b
Author: github-actions
AuthorDate: Wed Jul 12 13:49:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-27461.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-27461.json
b/scripts/ci/pr-bot/state/pr-state/pr-27461.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-27461.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch users/damccorm/v2-cleanup updated: Syntax
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/v2-cleanup
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/users/damccorm/v2-cleanup by
this push:
new 8e87f9bfb4d Syntax
8e87f9bfb4d is described below
commit 8e87f9bfb4de1e320761b91f5761775b1e035318
Author: Danny McCormick
AuthorDate: Wed Jul 12 09:49:32 2023 -0400
Syntax
---
sdks/python/test-suites/dataflow/common.gradle | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sdks/python/test-suites/dataflow/common.gradle
b/sdks/python/test-suites/dataflow/common.gradle
index ab6b33ab028..44257b09c01 100644
--- a/sdks/python/test-suites/dataflow/common.gradle
+++ b/sdks/python/test-suites/dataflow/common.gradle
@@ -70,7 +70,7 @@ def basicPytestOpts = [
]
def preCommitIT(String runScriptsDir, String envdir, Boolean streaming, String
pythonSuffix) {
- suffix = streaming ? "_streaming" : "_batch"
+ def suffix = streaming ? "_streaming" : "_batch"
task "preCommitIT${suffix}" {
dependsOn 'initializeForDataflowJob'
[beam] branch master updated (c9fe1586254 -> 7f063ae1cb9)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from c9fe1586254 Remove ValidatesRunner_V1 suite (#26543) add 7f063ae1cb9 fixes issue with pagination when PR has more than 30 jobs (#27468) No new revisions were added by this update. Summary of changes: .github/actions/rerun-job-action/action.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
[beam] branch users/damccorm/v2-cleanup updated (77e7a4e1b48 -> f3fc6c64fa2)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/v2-cleanup in repository https://gitbox.apache.org/repos/asf/beam.git from 77e7a4e1b48 Remove duplicate tasks add f3fc6c64fa2 Remove duplicate tasks No new revisions were added by this update. Summary of changes: sdks/python/test-suites/dataflow/common.gradle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
