[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new a9b9116215a Updating config from bot a9b9116215a is described below commit a9b9116215a951398c35f939615db978c8e6ed0b Author: github-actions AuthorDate: Wed May 25 07:04:42 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b8fe844264d Updating config from bot b8fe844264d is described below commit b8fe844264db828ead75abe6b66c0d3f4d14ffbc Author: github-actions AuthorDate: Wed May 25 07:57:14 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 5a3d11fdf77 Updating config from bot 5a3d11fdf77 is described below commit 5a3d11fdf7771f986c2ff7c591c66b905ef03e0d Author: github-actions AuthorDate: Wed May 25 08:06:23 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 64d36f58380 Updating config from bot 64d36f58380 is described below commit 64d36f583801b44a028caaf9a427754522872431 Author: github-actions AuthorDate: Wed May 25 08:39:48 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 0a07a3b41ae Updating config from bot 0a07a3b41ae is described below commit 0a07a3b41aef15fc3f6ee79ac09e03e3db4a3fdc Author: github-actions AuthorDate: Wed May 25 09:04:44 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1133600e1bd Updating config from bot 1133600e1bd is described below commit 1133600e1bdfb6ab7ee2935f6c4e4a5d7d45f770 Author: github-actions AuthorDate: Wed May 25 09:36:10 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 596fa2bfa3b Updating config from bot 596fa2bfa3b is described below commit 596fa2bfa3b3e724ff0bfd5208758c8ae94c9a31 Author: github-actions AuthorDate: Wed May 25 10:05:27 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 34d3583dd34 Updating config from bot 34d3583dd34 is described below commit 34d3583dd34331f1b2f5bfb80850cb0f0e61e478 Author: github-actions AuthorDate: Wed May 25 10:38:35 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4a826a65abd Updating config from bot 4a826a65abd is described below commit 4a826a65abdf00eec383b2759ec5379ec9a00d40 Author: github-actions AuthorDate: Wed May 25 11:04:52 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 47a3cba0d80 Updating config from bot 47a3cba0d80 is described below commit 47a3cba0d80e2d99470d95518b4695f4753ae374 Author: github-actions AuthorDate: Wed May 25 11:34:52 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4e71ec9a17d Updating config from bot 4e71ec9a17d is described below commit 4e71ec9a17d4d80dc18560cf8b05cba9db3427a4 Author: github-actions AuthorDate: Wed May 25 12:06:44 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1351309c616 Updating config from bot
1351309c616 is described below
commit 1351309c616eb45dd9acc02d87098a9c5e296469
Author: github-actions
AuthorDate: Wed May 25 12:46:29 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17752.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17752.json
b/scripts/ci/pr-bot/state/pr-state/pr-17752.json
new file mode 100644
index 000..1c2c2e31975
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17752.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d8149c9d509 Updating config from bot
d8149c9d509 is described below
commit d8149c9d509f69e7bae941d2eedfb375e7f477b9
Author: github-actions
AuthorDate: Wed May 25 12:46:31 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 994a52de266..6865a0c9123 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1653052548992,
-"jrmccluskey": 1652940304367,
+"jrmccluskey": 1653482787841,
"youngoli": 1653024973545,
"damccorm": 1652940309956,
"riteshghorse": 1652992565365
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new eb4e956dc30 Updating config from bot eb4e956dc30 is described below commit eb4e956dc3086829e7f306a2950409ece12e3ced Author: github-actions AuthorDate: Wed May 25 12:46:33 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f34a36724d9 Updating config from bot f34a36724d9 is described below commit f34a36724d9d46ab114e0ade50340c84aebaa35c Author: github-actions AuthorDate: Wed May 25 13:22:35 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 5e33732139d Updating config from bot 5e33732139d is described below commit 5e33732139d3ef9f811e64c02d3f3abc7ec2255d Author: github-actions AuthorDate: Wed May 25 13:46:32 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d54edf7539b Updating config from bot d54edf7539b is described below commit d54edf7539be58bc913bfe15a8962745bdb22538 Author: github-actions AuthorDate: Wed May 25 14:06:03 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d5a6c685e7c Updating config from bot d5a6c685e7c is described below commit d5a6c685e7c772bb4b8da497bec5362062418a74 Author: github-actions AuthorDate: Wed May 25 14:35:41 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f18db8d7781 Updating config from bot
f18db8d7781 is described below
commit f18db8d7781e0320f45a4356ce39d3dbc690f75f
Author: github-actions
AuthorDate: Wed May 25 14:35:35 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17752.json | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17752.json
b/scripts/ci/pr-bot/state/pr-state/pr-17752.json
index 1c2c2e31975..269c2419ccc 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-17752.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17752.json
@@ -1,10 +1,10 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"go": "jrmccluskey"
+"go": "youngoli"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
- "committerAssigned": false
+ "committerAssigned": true
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 39fd9d5d99c Updating config from bot
39fd9d5d99c is described below
commit 39fd9d5d99c7137032a3befa331e92454468be7d
Author: github-actions
AuthorDate: Wed May 25 14:35:39 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 6865a0c9123..d6e17043447 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"lostluck": 1653052548992,
"jrmccluskey": 1653482787841,
-"youngoli": 1653024973545,
+"youngoli": 1653489334025,
"damccorm": 1652940309956,
"riteshghorse": 1652992565365
}
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8d590276b94 Updating config from bot 8d590276b94 is described below commit 8d590276b94a67e5dcb96212672912071c75ff8f Author: github-actions AuthorDate: Wed May 25 15:05:10 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 987c10e8492 Updating config from bot 987c10e8492 is described below commit 987c10e8492393a8025bfc578466c6600cf6113b Author: github-actions AuthorDate: Wed May 25 15:39:01 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 16d8a464507 Updating config from bot
16d8a464507 is described below
commit 16d8a464507c52f0b2bd3ff5d906987c162cd61a
Author: github-actions
AuthorDate: Wed May 25 15:38:57 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17754.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17754.json
b/scripts/ci/pr-bot/state/pr-state/pr-17754.json
new file mode 100644
index 000..2deda2c6f85
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17754.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 5049da2d22f Updating config from bot 5049da2d22f is described below commit 5049da2d22f1a809f846503de61af5f530730aa5 Author: github-actions AuthorDate: Wed May 25 15:38:59 2022 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index d6e17043447..9c2a7f6c7e4 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -4,7 +4,7 @@ "lostluck": 1653052548992, "jrmccluskey": 1653482787841, "youngoli": 1653489334025, -"damccorm": 1652940309956, +"damccorm": 1653493136196, "riteshghorse": 1652992565365 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new cd9e48e4d45 Updating config from bot cd9e48e4d45 is described below commit cd9e48e4d45022ceb726aa4b7c32940ecbc2445f Author: github-actions AuthorDate: Wed May 25 15:43:46 2022 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-17754.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17754.json b/scripts/ci/pr-bot/state/pr-state/pr-17754.json index 2deda2c6f85..7c0d6dd803d 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-17754.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-17754.json @@ -4,7 +4,7 @@ "go": "damccorm" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new fbf6e4d0d79 Updating config from bot fbf6e4d0d79 is described below commit fbf6e4d0d79f1bcaad64d8e0aa838cc80ce88705 Author: github-actions AuthorDate: Wed May 25 16:04:56 2022 + Updating config from bot
[beam] branch master updated: [BEAM-14139] Remove unused Flink 1.11 directory (#17750)
This is an automated email from the ASF dual-hosted git repository.
bhulette pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 57354042908 [BEAM-14139] Remove unused Flink 1.11 directory (#17750)
57354042908 is described below
commit 57354042908ab86742018eea002c4fd7faaa5a6a
Author: Kyle Weaver
AuthorDate: Wed May 25 09:27:14 2022 -0700
[BEAM-14139] Remove unused Flink 1.11 directory (#17750)
---
.../runners/flink/streaming/StreamSources.java | 91 --
1 file changed, 91 deletions(-)
diff --git
a/runners/flink/1.11/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
b/runners/flink/1.11/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
deleted file mode 100644
index 3452c50fc29..000
---
a/runners/flink/1.11/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
+++ /dev/null
@@ -1,91 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.beam.runners.flink.streaming;
-
-import org.apache.flink.api.dag.Transformation;
-import org.apache.flink.runtime.operators.testutils.MockEnvironmentBuilder;
-import org.apache.flink.streaming.api.functions.source.SourceFunction;
-import org.apache.flink.streaming.api.operators.AbstractStreamOperator;
-import org.apache.flink.streaming.api.operators.Output;
-import org.apache.flink.streaming.api.operators.StreamSource;
-import org.apache.flink.streaming.api.transformations.OneInputTransformation;
-import org.apache.flink.streaming.runtime.streamrecord.StreamRecord;
-import org.apache.flink.streaming.runtime.streamstatus.StreamStatus;
-import org.apache.flink.streaming.runtime.streamstatus.StreamStatusMaintainer;
-import org.apache.flink.streaming.runtime.tasks.OperatorChain;
-import org.apache.flink.streaming.runtime.tasks.StreamTask;
-
-/**
- * {@link StreamSource} utilities, that bridge incompatibilities between Flink
releases.
- *
- * This change is becauses RecordWriter is wrapped in RecordWriterDelegate
in 1.10, please refer
- * to
https://github.com/apache/flink/commit/2c8b4ef572f05bf4740b7e204af1e5e709cd945c
for more
- * details.
- */
-public class StreamSources {
-
- /**
- * Backward compatibility helper for {@link OneInputTransformation}
`getInput` method, that has
- * been removed in Flink 1.12.
- *
- * @param source Source to get single input from.
- * @return Input transformation.
- */
- public static Transformation getOnlyInput(OneInputTransformation
source) {
-return source.getInput();
- }
-
- public static > void run(
- StreamSource streamSource,
- Object lockingObject,
- Output> collector)
- throws Exception {
-streamSource.run(
-lockingObject,
-new TestStreamStatusMaintainer(),
-collector,
-createOperatorChain(streamSource));
- }
-
- private static OperatorChain
createOperatorChain(AbstractStreamOperator operator) {
-return new OperatorChain<>(
-operator.getContainingTask(),
-StreamTask.createRecordWriterDelegate(
-operator.getOperatorConfig(), new
MockEnvironmentBuilder().build()));
- }
-
- /** StreamStatusMaintainer was removed in Flink 1.14. */
- private static final class TestStreamStatusMaintainer implements
StreamStatusMaintainer {
-StreamStatus currentStreamStatus = StreamStatus.ACTIVE;
-
-@Override
-public void toggleStreamStatus(StreamStatus streamStatus) {
- if (!currentStreamStatus.equals(streamStatus)) {
-currentStreamStatus = streamStatus;
- }
-}
-
-@Override
-public StreamStatus getStreamStatus() {
- return currentStreamStatus;
-}
- }
-
- /** The emitWatermarkStatus method was added in Flink 1.14, so we need to
wrap Output. */
- public interface OutputWrapper extends Output {}
-}
[beam] branch master updated: [BEAM-14044] Allow ModelLoader to forward BatchElements args (#17527)
This is an automated email from the ASF dual-hosted git repository.
bhulette pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 0a6fa95a04d [BEAM-14044] Allow ModelLoader to forward BatchElements
args (#17527)
0a6fa95a04d is described below
commit 0a6fa95a04dc9485510e3bd034513d566155f61a
Author: zwestrick <101210340+zwestr...@users.noreply.github.com>
AuthorDate: Wed May 25 09:37:43 2022 -0700
[BEAM-14044] Allow ModelLoader to forward BatchElements args (#17527)
* Updates ModelLoader to allow defining arguments to BatchElements
* Update base.py
* Adds unit test for batch arg forwarding
* Fixes run_inference_base -> base
* lint changes
* lint changes
* fmt fix
* lint changes
---
sdks/python/apache_beam/ml/inference/base.py | 7 ++-
sdks/python/apache_beam/ml/inference/base_test.py | 22 ++
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/sdks/python/apache_beam/ml/inference/base.py
b/sdks/python/apache_beam/ml/inference/base.py
index 3c7e6fe0e3d..49753c4e7a3 100644
--- a/sdks/python/apache_beam/ml/inference/base.py
+++ b/sdks/python/apache_beam/ml/inference/base.py
@@ -38,6 +38,7 @@ from typing import Any
from typing import Generic
from typing import Iterable
from typing import List
+from typing import Mapping
from typing import TypeVar
import apache_beam as beam
@@ -82,6 +83,10 @@ class ModelLoader(Generic[T]):
"""Returns an implementation of InferenceRunner for this model."""
raise NotImplementedError(type(self))
+ def batch_elements_kwargs(self) -> Mapping[str, Any]:
+"""Returns kwargs suitable for beam.BatchElements."""
+return {}
+
class RunInference(beam.PTransform):
"""An extensible transform for running inferences."""
@@ -95,7 +100,7 @@ class RunInference(beam.PTransform):
return (
pcoll
# TODO(BEAM-14044): Hook into the batching DoFn APIs.
-| beam.BatchElements()
+| beam.BatchElements(**self._model_loader.batch_elements_kwargs())
| beam.ParDo(_RunInferenceDoFn(self._model_loader, self._clock)))
diff --git a/sdks/python/apache_beam/ml/inference/base_test.py
b/sdks/python/apache_beam/ml/inference/base_test.py
index 55936f63ed4..384ee9426d0 100644
--- a/sdks/python/apache_beam/ml/inference/base_test.py
+++ b/sdks/python/apache_beam/ml/inference/base_test.py
@@ -72,6 +72,21 @@ class ExtractInferences(beam.DoFn):
yield prediction_result.inference
+class FakeInferenceRunnerNeedsBigBatch(FakeInferenceRunner):
+ def run_inference(self, batch, unused_model):
+if len(batch) < 100:
+ raise ValueError('Unexpectedly small batch')
+return batch
+
+
+class FakeLoaderWithBatchArgForwarding(FakeModelLoader):
+ def get_inference_runner(self):
+return FakeInferenceRunnerNeedsBigBatch()
+
+ def batch_elements_kwargs(self):
+return {'min_batch_size': }
+
+
class RunInferenceBaseTest(unittest.TestCase):
def test_run_inference_impl_simple_examples(self):
with TestPipeline() as pipeline:
@@ -142,6 +157,13 @@ class RunInferenceBaseTest(unittest.TestCase):
self.assertEqual(load_model_latency.result.count, 1)
self.assertEqual(load_model_latency.result.mean, 50)
+ def test_forwards_batch_args(self):
+examples = list(range(100))
+with TestPipeline() as pipeline:
+ pcoll = pipeline | 'start' >> beam.Create(examples)
+ actual = pcoll | base.RunInference(FakeLoaderWithBatchArgForwarding())
+ assert_that(actual, equal_to(examples), label='assert:inferences')
+
if __name__ == '__main__':
unittest.main()
[beam] branch master updated: [BEAM-14481] Remove unnecessary context (#17737)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 15a5abceaa5 [BEAM-14481] Remove unnecessary context (#17737) 15a5abceaa5 is described below commit 15a5abceaa58ef63cc07d599c2a140680025b205 Author: Brian Hulette AuthorDate: Wed May 25 09:38:25 2022 -0700 [BEAM-14481] Remove unnecessary context (#17737) --- .../apache_beam/runners/worker/bundle_processor.py | 25 +++--- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/sdks/python/apache_beam/runners/worker/bundle_processor.py b/sdks/python/apache_beam/runners/worker/bundle_processor.py index 1f6239a6dc6..0d993dd5413 100644 --- a/sdks/python/apache_beam/runners/worker/bundle_processor.py +++ b/sdks/python/apache_beam/runners/worker/bundle_processor.py @@ -191,19 +191,18 @@ class DataInputOperation(RunnerIOOperation): self.started = False def setup(self): -with self.scoped_start_state: - super().setup() - # We must do this manually as we don't have a spec or spec.output_coders. - self.receivers = [ - operations.ConsumerSet.create( - self.counter_factory, - self.name_context.step_name, - 0, - self.consumer, - self.windowed_coder, - self.get_output_batch_converter(), - self._get_runtime_performance_hints()) - ] +super().setup() +# We must do this manually as we don't have a spec or spec.output_coders. +self.receivers = [ +operations.ConsumerSet.create( +self.counter_factory, +self.name_context.step_name, +0, +self.consumer, +self.windowed_coder, +self.get_output_batch_converter(), +self._get_runtime_performance_hints()) +] def start(self): # type: () -> None
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 00bc553cd68 Updating config from bot 00bc553cd68 is described below commit 00bc553cd684dd49251ba98db81814fa8d7a6ec2 Author: github-actions AuthorDate: Wed May 25 16:44:20 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 230e3b2143a Updating config from bot
230e3b2143a is described below
commit 230e3b2143a464eb62dbb89ddeff0dff4d726871
Author: github-actions
AuthorDate: Wed May 25 16:44:16 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17755.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17755.json
b/scripts/ci/pr-bot/state/pr-state/pr-17755.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17755.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 643e01546f4 Updating config from bot 643e01546f4 is described below commit 643e01546f4b8b8b8dd4e527b76b5d97deedd6a7 Author: github-actions AuthorDate: Wed May 25 16:56:46 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d6395176b21 Updating config from bot d6395176b21 is described below commit d6395176b21c850030220f01ee8069bc7e921ee8 Author: github-actions AuthorDate: Wed May 25 16:59:41 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c260f8b7f3e Updating config from bot c260f8b7f3e is described below commit c260f8b7f3e7980900b6cccbc2190d7fb3822c16 Author: github-actions AuthorDate: Wed May 25 17:05:30 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 11051e8447f Updating config from bot 11051e8447f is described below commit 11051e8447fa7df8cec4cd999185871e05b37afa Author: github-actions AuthorDate: Wed May 25 17:09:18 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 99c70d83d59 Updating config from bot 99c70d83d59 is described below commit 99c70d83d59a5e7a5703966cb2f95d567abfdd33 Author: github-actions AuthorDate: Wed May 25 17:09:16 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 51fb74cf43c Updating config from bot
51fb74cf43c is described below
commit 51fb74cf43cd7bdd7c5c33bbae7339ccc6f995a6
Author: github-actions
AuthorDate: Wed May 25 17:20:18 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17748.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17748.json
b/scripts/ci/pr-bot/state/pr-state/pr-17748.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17748.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6c696b9f898 Updating config from bot 6c696b9f898 is described below commit 6c696b9f8985ddc8cf19238ac9a949cc8fab429a Author: github-actions AuthorDate: Wed May 25 17:32:09 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 3130110a409 Updating config from bot 3130110a409 is described below commit 3130110a4099c0c04a1b4ecb073e69ba8bd95ec7 Author: github-actions AuthorDate: Wed May 25 17:40:16 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 06c60e2d2d1 Updating config from bot 06c60e2d2d1 is described below commit 06c60e2d2d116e98f76d7a17a395f09ffb2545a3 Author: github-actions AuthorDate: Wed May 25 17:40:18 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 2e3714db168 Updating config from bot
2e3714db168 is described below
commit 2e3714db168489fc5677cf13009a5f1352ca31ae
Author: github-actions
AuthorDate: Wed May 25 17:40:21 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17747.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17747.json
b/scripts/ci/pr-bot/state/pr-state/pr-17747.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17747.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6c562352766 Updating config from bot 6c562352766 is described below commit 6c562352766de811f30e143cc5a2cf552c448d50 Author: github-actions AuthorDate: Wed May 25 17:40:22 2022 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 9c2a7f6c7e4..dc1392d48bb 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1653482787841, "youngoli": 1653489334025, "damccorm": 1653493136196, -"riteshghorse": 1652992565365 +"riteshghorse": 1653500420008 } } \ No newline at end of file
[beam] branch master updated: [BEAM-9324] Fix incompatibility of direct runner with cython (#17728)
This is an automated email from the ASF dual-hosted git repository. bhulette pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new a73f32d5a07 [BEAM-9324] Fix incompatibility of direct runner with cython (#17728) a73f32d5a07 is described below commit a73f32d5a075a145d643f4311c55274374692f4e Author: Yi Hu AuthorDate: Wed May 25 13:48:07 2022 -0400 [BEAM-9324] Fix incompatibility of direct runner with cython (#17728) * [BEAM-9324] Fix incompatibility of direct runner with cython * documented workaround for future clean up --- .../apache_beam/runners/worker/operations.py | 36 +++--- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/sdks/python/apache_beam/runners/worker/operations.py b/sdks/python/apache_beam/runners/worker/operations.py index f46a4a18369..09706fe7187 100644 --- a/sdks/python/apache_beam/runners/worker/operations.py +++ b/sdks/python/apache_beam/runners/worker/operations.py @@ -78,9 +78,7 @@ try: except ImportError: class FakeCython(object): -@staticmethod -def cast(type, value): - return value +compiled = False globals()['cython'] = FakeCython() @@ -93,6 +91,22 @@ SdfSplitResultsPrimary = Tuple['DoOperation', 'SplitResultPrimary'] SdfSplitResultsResidual = Tuple['DoOperation', 'SplitResultResidual'] +# TODO(BEAM-9324) Remove these workarounds once upgraded to Cython 3 +def _cast_to_operation(value): + if cython.compiled: +return cython.cast(Operation, value) + else: +return value + + +# TODO(BEAM-9324) Remove these workarounds once upgraded to Cython 3 +def _cast_to_receiver(value): + if cython.compiled: +return cython.cast(Receiver, value) + else: +return value + + class ConsumerSet(Receiver): """A ConsumerSet represents a graph edge between two Operation nodes. @@ -307,7 +321,7 @@ class GeneralPurposeConsumerSet(ConsumerSet): self.update_counters_start(windowed_value) for consumer in self.element_consumers: - cython.cast(Operation, consumer).process(windowed_value) + _cast_to_operation(consumer).process(windowed_value) # TODO: Do this branching when contstructing ConsumerSet if self.has_batch_consumers: @@ -324,10 +338,10 @@ class GeneralPurposeConsumerSet(ConsumerSet): for wv in windowed_batch.as_windowed_values( self.producer_batch_converter.explode_batch): for consumer in self.element_consumers: - cython.cast(Operation, consumer).process(wv) + _cast_to_operation(consumer).process(wv) for consumer in self.passthrough_batch_consumers: - cython.cast(Operation, consumer).process_batch(windowed_batch) + _cast_to_operation(consumer).process_batch(windowed_batch) for (consumer_batch_converter, consumers) in self.other_batch_consumers.items(): @@ -342,7 +356,7 @@ class GeneralPurposeConsumerSet(ConsumerSet): "This is very inefficient, consider re-structuring your pipeline " "or adding a DoFn to directly convert between these types.", InefficientExecutionWarning) -cython.cast(Operation, consumer).process_batch( +_cast_to_operation(consumer).process_batch( windowed_batch.with_values( consumer_batch_converter.produce_batch( self.producer_batch_converter.explode_batch( @@ -358,13 +372,13 @@ class GeneralPurposeConsumerSet(ConsumerSet): for windowed_batch in WindowedBatch.from_windowed_values( self._batched_elements, produce_fn=batch_converter.produce_batch): for consumer in consumers: - cython.cast(Operation, consumer).process_batch(windowed_batch) + _cast_to_operation(consumer).process_batch(windowed_batch) for consumer in self.passthrough_batch_consumers: for windowed_batch in WindowedBatch.from_windowed_values( self._batched_elements, produce_fn=self.producer_batch_converter.produce_batch): -cython.cast(Operation, consumer).process_batch(windowed_batch) +_cast_to_operation(consumer).process_batch(windowed_batch) self._batched_elements = [] @@ -495,7 +509,7 @@ class Operation(object): """Finish operation.""" for receiver in self.receivers: - cython.cast(Receiver, receiver).flush() + _cast_to_receiver(receiver).flush() def teardown(self): # type: () -> None @@ -511,7 +525,7 @@ class Operation(object): def output(self, windowed_value, output_index=0): # type: (WindowedValue, int) -> None -cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value) +_cast_to_receiver(self.receivers[output_index]).receive(windowed_value) def add_receiver(self, operation, output_index=0): # type: (Operation, int) -> None
[beam] branch master updated: [BEAM-14503] Add support for Flink 1.15 (#17739)
This is an automated email from the ASF dual-hosted git repository.
ibzib pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 24decb50cf3 [BEAM-14503] Add support for Flink 1.15 (#17739)
24decb50cf3 is described below
commit 24decb50cf3e14e8cce4dd9d82b8963bf57a1805
Author: Julien Tournay
AuthorDate: Wed May 25 19:59:04 2022 +0200
[BEAM-14503] Add support for Flink 1.15 (#17739)
* Flink 1.15 runner
* Ignore Flink 1.11
---
gradle.properties | 2 +-
.../streaming/ProcessingTimeCallbackCompat.java| 22 +++
.../beam/runners/flink/MiniClusterCompat.java | 29 ++
runners/flink/1.15/build.gradle| 34 +
.../flink/1.15/job-server-container/build.gradle | 26 +
runners/flink/1.15/job-server/build.gradle | 31 +++
.../streaming/ProcessingTimeCallbackCompat.java| 22 +++
.../beam/runners/flink/MiniClusterCompat.java | 30 +++
runners/flink/flink_runner.gradle | 44 --
.../flink/FlinkStreamingTransformTranslators.java | 4 +-
.../wrappers/streaming/DoFnOperator.java | 3 +-
.../streaming/io/UnboundedSourceWrapper.java | 4 +-
.../flink/FlinkRequiresStableInputTest.java| 2 +-
.../beam/runners/flink/FlinkSavepointTest.java | 2 +-
.../python/apache_beam/options/pipeline_options.py | 2 +-
settings.gradle.kts| 4 ++
16 files changed, 240 insertions(+), 21 deletions(-)
diff --git a/gradle.properties b/gradle.properties
index f880631cd68..d1528eb2cd0 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -37,5 +37,5 @@ javaVersion=1.8
docker_image_default_repo_root=apache
docker_image_default_repo_prefix=beam_
-flink_versions=1.12,1.13,1.14
+flink_versions=1.12,1.13,1.14,1.15
diff --git
a/runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ProcessingTimeCallbackCompat.java
b/runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ProcessingTimeCallbackCompat.java
new file mode 100644
index 000..a494fec01dd
--- /dev/null
+++
b/runners/flink/1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ProcessingTimeCallbackCompat.java
@@ -0,0 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.runners.flink.translation.wrappers.streaming;
+
+import org.apache.flink.streaming.runtime.tasks.ProcessingTimeCallback;
+
+public interface ProcessingTimeCallbackCompat extends ProcessingTimeCallback {}
diff --git
a/runners/flink/1.12/src/test/java/org/apache/beam/runners/flink/MiniClusterCompat.java
b/runners/flink/1.12/src/test/java/org/apache/beam/runners/flink/MiniClusterCompat.java
new file mode 100644
index 000..1bbcd0159b1
--- /dev/null
+++
b/runners/flink/1.12/src/test/java/org/apache/beam/runners/flink/MiniClusterCompat.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.runners.flink;
+
+import java.util.concurrent.CompletableFuture;
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.runtime.minicluster.MiniCluster;
+
+public class MiniClusterCompat {
+ public static CompletableF
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new fce2538cfe1 Updating config from bot
fce2538cfe1 is described below
commit fce2538cfe1fab7a5402531edf2f227858c2675a
Author: github-actions
AuthorDate: Wed May 25 18:04:53 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17755.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17755.json
b/scripts/ci/pr-bot/state/pr-state/pr-17755.json
index 242a48d7d3b..8ea94810368 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-17755.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17755.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new dfcb1b0e390 Updating config from bot dfcb1b0e390 is described below commit dfcb1b0e390c0e5fc6694cc6248ff3ceae916c9c Author: github-actions AuthorDate: Wed May 25 18:04:58 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b4fcb22dfe5 Updating config from bot
b4fcb22dfe5 is described below
commit b4fcb22dfe5b3e0a8f24eff4137c39e9781cd261
Author: github-actions
AuthorDate: Wed May 25 18:04:55 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index dc1392d48bb..c94c6405c2a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1653052548992,
+"lostluck": 1653501892515,
"jrmccluskey": 1653482787841,
"youngoli": 1653489334025,
"damccorm": 1653493136196,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2d3803f816d Updating config from bot 2d3803f816d is described below commit 2d3803f816d146be3141296af58f817d72170a7a Author: github-actions AuthorDate: Wed May 25 18:07:10 2022 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-17755.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17755.json b/scripts/ci/pr-bot/state/pr-state/pr-17755.json index 8ea94810368..ae9024f9d4e 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-17755.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-17755.json @@ -4,7 +4,7 @@ "go": "lostluck" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] annotated tag sdks/v2.39.0 updated (23bb12f6651 -> 03a4fe1b224)
This is an automated email from the ASF dual-hosted git repository. yichi pushed a change to annotated tag sdks/v2.39.0 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag sdks/v2.39.0 was modified! *** from 23bb12f6651 (commit) to 03a4fe1b224 (tag) tagging 308a5339ecd2870bf0f334380df81235dcf5b2ac (tag) length 145 bytes by Yichi Zhang on Wed May 25 18:14:02 2022 + - Log - Go SDK v2.39.0 -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEDbn/BLXZZ5A6KEJwId4WYClx63wFAmKOchoACgkQId4WYClx 63wPXhAAlO3O5RC3Tta1tk4IgQt/zcstwFx6cgewGlCjpvQmtd3eXxrRdiSwtHg4 d4V+HXC/e0YB/Jj1ZDABG0Wv3H9iutyO6voTI6Nh7VFtMkjg1SDNigQjl5MBvyaZ LNSZMeMO9CAju/472ZjA4y1rKg0dCBe06oHAS6zO9H+FRpPSo6inh3gGGEb+2mf5 TgA6vA1YMtzRxPF28goA+xd36F5Rexo58Acig/Ep16UNWHPKGml8fpN3odAYVDGZ c/RVTUz0fV8MncTRGErcBzLfSY8XHequr7npdKFI0d79vTAM7kMDrFUL3P+tScJv GhVnmdjZlxpYL3kMCMgy+fc2o+AW1s9sopGYH827Uv2lXyECkbjtO5dI6rEgcTVt 1Ni5U3IE9TLnGy9VBAlgaammkVDlp2Zh7ln3FMTE10w3Gn8sQXV1TfKxTmNg8qiw jcfabtibHeAPfKyDdTKe+sc0xBhQjFJfjxi5kHYkmZuoJdUlUNE5pJ8SeIZzOrvg QvuEeF6wvVyv58lTvxdI+5wtJn6+nkhoXPitHcsq/EqjingcG6TMvnNr6AbYGSAS mTYj03uIvpQDMaK/KzJrCqZ0t0Q/XBwGZT09UVaOUg/586J2Jsl/Lo3JhrcvsQUU C4+aJ6QIjxiUR6HPvhNQfmKg1Z5hoxNGcLSRssOZJcFX2po6/bk= =sfzd -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[beam] annotated tag v2.39.0 updated (23bb12f6651 -> 0f07d6be013)
This is an automated email from the ASF dual-hosted git repository. yichi pushed a change to annotated tag v2.39.0 in repository https://gitbox.apache.org/repos/asf/beam.git *** WARNING: tag v2.39.0 was modified! *** from 23bb12f6651 (commit) to 0f07d6be013 (tag) tagging 308a5339ecd2870bf0f334380df81235dcf5b2ac (tag) length 145 bytes by Yichi Zhang on Wed May 25 18:15:42 2022 + - Log - Tag for 2.39.0 release -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEDbn/BLXZZ5A6KEJwId4WYClx63wFAmKOcpIACgkQId4WYClx 63zdcBAAosnz1sqg/ImaXnWJ8DCUEoekLFsLK7OIVjIfEonP/PKMVuOIx0kZejdG OdQlLwcmWJVsJmyG1p/W7kwuz88YZOeFxS5KwDXJ1PBvsproYrFKSH31gIrLpu5T WvI1ueC/D/RFbHVSpZy/RKXMIne9+t0GNQNcyKSIp4PtndjEH6ogGMfFcQ+Au3ET Jjaehbm+Jho7Ocb+dj2k6U9kNpaN5M1FhJ9pNgGEs87s4doKOmN2ZAUZakZOmgKp 0maiPxiIWkY2K7JMabk7Z0YBwHJ5JgLkkSrbBHgMIzYujlr3+voaXeRlL68oqmbd o48wD1eJFfr1JZkKgkpJiCNdh3LyA+XLTc2kFcleAR5hOZSiqbAHAnliQVhg1YYD ylVxOF9h9LyKeyPYL8zv0rvJETdFyM7TMjlHJu8z918iLa6uxndG5hTut8vK1Ac8 8TBLpIk72AjZgPvAfpCPlLOPwPcpBRdHB/9b4uH1wzNfdnBDx6UEDchA/XWUemSC Jvn7pzLweyT8VxWkoXSCY2s/5X5zDg493yx7jfpps+bsoDCYAOp37iGbyTcHRAx5 bmYuDxyioCmFJaCrYaQhdHcN6gKA3NR0Fm9W9qAeXXAoFMysIcypkae+tw+3OF0V iXlEo/cVMjG9vlCqhCvmen9cKHKM0+1qQsriso+/WIV4K15GiA8= =Nv73 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[beam] branch master updated: Update Beam website to release 2.39.0 (#17690)
This is an automated email from the ASF dual-hosted git repository. yichi pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 83c85a57993 Update Beam website to release 2.39.0 (#17690) 83c85a57993 is described below commit 83c85a5799377eee0410f4c492965749155370d1 Author: Yichi Zhang AuthorDate: Wed May 25 11:28:33 2022 -0700 Update Beam website to release 2.39.0 (#17690) --- CHANGES.md | 31 +++- website/www/site/config.toml | 2 +- website/www/site/content/en/blog/beam-2.39.0.md| 185 + .../www/site/content/en/get-started/downloads.md | 8 + website/www/site/data/authors.yml | 3 + 5 files changed, 221 insertions(+), 8 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 42c0592ec97..5eca4ec80d2 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -85,11 +85,13 @@ * ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)). -# [2.39.0] - Unreleased, Cut +# [2.39.0] - 2022-05-25 ## Highlights -* Watermark estimation is now supported in the Go SDK ([BEAM-11105](https://issues.apache.org/jira/browse/BEAM-11105)) +* Watermark estimation is now supported in the Go SDK ([BEAM-11105](https://issues.apache.org/jira/browse/BEAM-11105)). +* Support for impersonation credentials added to dataflow runner in the Java and Python SDK ([BEAM-14014](https://issues.apache.org/jira/browse/BEAM-14014)). +* Implemented Apache PulsarIO ([BEAM-8218](https://issues.apache.org/jira/browse/BEAM-8218)). ## I/Os @@ -97,11 +99,27 @@ * JmsIO introduces the ability to write to dynamic topics (Java) ([BEAM-16308](https://issues.apache.org/jira/browse/BEAM-16308)). * A `topicNameMapper` must be set to extract the topic name from the input value. * A `valueMapper` must be set to convert the input value to JMS message. +* Reduce number of threads spawned by BigqueryIO StreamingInserts ( + [BEAM-14283](https://issues.apache.org/jira/browse/BEAM-14283)). +* Implemented Apache PulsarIO ([BEAM-8218](https://issues.apache.org/jira/browse/BEAM-8218)). + ## New Features / Improvements +* Support for flink scala 2.12, because most of the libraries support version 2.12 onwards. ([beam-14386](https://issues.apache.org/jira/browse/BEAM-14386)) * 'Manage Clusters' JupyterLab extension added for users to configure usage of Dataproc clusters managed by Interactive Beam (Python) ([BEAM-14130](https://issues.apache.org/jira/browse/BEAM-14130)). * Pipeline drain support added for Go SDK ([BEAM-11106](https://issues.apache.org/jira/browse/BEAM-11106)). **Note: this feature is not yet fully validated and should be treated as experimental in this release.** +* `DataFrame.unstack()`, `DataFrame.pivot() ` and `Series.unstack()` + implemented for DataFrame API ([BEAM-13948](https://issues.apache.org/jira/browse/BEAM-13948), [BEAM-13966](https://issues.apache.org/jira/browse/BEAM-13966)). +* Support for impersonation credentials added to dataflow runner in the Java and Python SDK ([BEAM-14014](https://issues.apache.org/jira/browse/BEAM-14014)). +* Implemented Jupyterlab extension for managing Dataproc clusters ([BEAM-14130](https://issues.apache.org/jira/browse/BEAM-14130)). +* ExternalPythonTransform API added for easily invoking Python transforms from + Java ([BEAM-14143](https://issues.apache.org/jira/browse/BEAM-14143)). +* Added Add support for Elasticsearch 8.x ([BEAM-14003](https://issues.apache.org/jira/browse/BEAM-14003)). +* Shard aware Kinesis record aggregation (AWS Sdk v2), ([BEAM-14104](https://issues.apache.org/jira/browse/BEAM-14104)). +* Upgrade to ZetaSQL 2022.04.1 ([BEAM-14348](https://issues.apache.org/jira/browse/BEAM-14348)). +* Fixed ReadFromBigQuery cannot be used with the interactive runner ([BEAM-14112](https://issues.apache.org/jira/browse/BEAM-14112)). + ## Breaking Changes @@ -113,20 +131,19 @@ .withValueMapper(new TextMessageMapper()); ``` * Coders in Python are expected to inherit from Coder. ([BEAM-14351](https://issues.apache.org/jira/browse/BEAM-14351)). +* New abstract method `metadata()` added to io.filesystem.FileSystem in the + Python SDK. ([BEAM-14314](https://issues.apache.org/jira/browse/BEAM-14314)) ## Deprecations -* Support for flink scala 2.12, because most of the libraries support version 2.12 onwards. ([beam-14386](https://issues.apache.org/jira/browse/BEAM-14386)) * Flink 1.11 is no longer supported ([BEAM-14139](https://issues.apache.org/jira/browse/BEAM-14139)). * Python 3.6 is no longer supported ([BEAM-13657](https://issues.apache.org/jira/browse/BEAM-13657)). ## Bugfixes -* Fixed X (Java/Python) ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)). - -## Known Issues +* Fixed Java Spanner IO NPE when ProjectID not specified in template executions (Java) ([BEAM-14405](https://issues.apach
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2aace18625d Updating config from bot 2aace18625d is described below commit 2aace18625d0674e5c914b6244a07fc14765f1ad Author: github-actions AuthorDate: Wed May 25 18:39:32 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b2610619c96 Updating config from bot b2610619c96 is described below commit b2610619c96f1b76eafd2c2445795ff5cb8ebb49 Author: github-actions AuthorDate: Wed May 25 19:04:50 2022 + Updating config from bot
[beam] branch master updated: [BEAM-14509] Add several flags to dataflow runner (#17752)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 2039b39f96f [BEAM-14509] Add several flags to dataflow runner (#17752)
2039b39f96f is described below
commit 2039b39f96ff96e0899a830e560c02c6c8e17acb
Author: Danny McCormick
AuthorDate: Wed May 25 15:23:08 2022 -0400
[BEAM-14509] Add several flags to dataflow runner (#17752)
---
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 112 +
sdks/go/pkg/beam/runners/dataflow/dataflow_test.go | 39 ++-
.../pkg/beam/runners/dataflow/dataflowlib/job.go | 21 +++-
3 files changed, 123 insertions(+), 49 deletions(-)
diff --git a/sdks/go/pkg/beam/runners/dataflow/dataflow.go
b/sdks/go/pkg/beam/runners/dataflow/dataflow.go
index 33b4b0fc66d..eff46bca3f3 100644
--- a/sdks/go/pkg/beam/runners/dataflow/dataflow.go
+++ b/sdks/go/pkg/beam/runners/dataflow/dataflow.go
@@ -51,26 +51,32 @@ import (
// TODO(herohde) 5/16/2017: the Dataflow flags should match the other SDKs.
var (
- endpoint = flag.String("dataflow_endpoint", "", "Dataflow
endpoint (optional).")
- stagingLocation = flag.String("staging_location", "", "GCS staging
location (required).")
- image= flag.String("worker_harness_container_image",
"", "Worker harness container image (required).")
- labels = flag.String("labels", "", "JSON-formatted
map[string]string of job labels (optional).")
- serviceAccountEmail = flag.String("service_account_email", "",
"Service account email (optional).")
- numWorkers = flag.Int64("num_workers", 0, "Number of workers
(optional).")
- maxNumWorkers= flag.Int64("max_num_workers", 0, "Maximum number
of workers during scaling (optional).")
- diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root disk
for VMs, in GB (optional).")
- diskType = flag.String("disk_type", "", "Type of root disk
for VMs (optional).")
- autoscalingAlgorithm = flag.String("autoscaling_algorithm", "",
"Autoscaling mode to use (optional).")
- zone = flag.String("zone", "", "GCP zone (optional)")
- network = flag.String("network", "", "GCP network
(optional)")
- subnetwork = flag.String("subnetwork", "", "GCP subnetwork
(optional)")
- noUsePublicIPs = flag.Bool("no_use_public_ips", false, "Workers
must not use public IP addresses (optional)")
- tempLocation = flag.String("temp_location", "", "Temp location
(optional)")
- machineType = flag.String("worker_machine_type", "", "GCE
machine type (optional)")
- minCPUPlatform = flag.String("min_cpu_platform", "", "GCE minimum
cpu platform (optional)")
- workerJar= flag.String("dataflow_worker_jar", "", "Dataflow
worker jar (optional)")
- workerRegion = flag.String("worker_region", "", "Dataflow
worker region (optional)")
- workerZone = flag.String("worker_zone", "", "Dataflow worker
zone (optional)")
+ endpoint = flag.String("dataflow_endpoint", "", "Dataflow
endpoint (optional).")
+ stagingLocation= flag.String("staging_location", "", "GCS
staging location (required).")
+ image = flag.String("worker_harness_container_image",
"", "Worker harness container image (optional).")
+ labels = flag.String("labels", "", "JSON-formatted
map[string]string of job labels (optional).")
+ serviceAccountEmail= flag.String("service_account_email", "",
"Service account email (optional).")
+ numWorkers = flag.Int64("num_workers", 0, "Number of
workers (optional).")
+ workerHarnessThreads = flag.Int64("number_of_worker_harness_threads",
0, "The number of threads per each worker harness process (optional).")
+ maxNumWorkers = flag.Int64("max_num_workers", 0, "Maximum
number of workers during scaling (optional).")
+ diskSizeGb = flag.Int64("disk_size_gb", 0, "Size of root
disk for VMs, in GB (optional).")
+ diskType = flag.String("disk_type", "", "Type of root
disk for VMs (optional).")
+ autoscalingAlgorithm = flag.String("autoscaling_algorithm", "",
"Autoscaling mode to use (optional).")
+ zone = flag.String("zone", "", "GCP zone (optional)")
+ kmsKey = flag.String("dataflow_kms_key", "", "The Cloud
KMS key identifier used to encrypt data at rest (optional).")
+ network= flag.String("network", "", "GCP network
(optional)")
+ subnetwork = flag.String("subnetwork", "", "GCP subnetwork
(optional)")
+ noUsePublicIPs = flag.Bool("no_use_public_ips", f
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 695535c1409 Updating config from bot 695535c1409 is described below commit 695535c140950e1ebf7abd7f97c5e814fe6554e8 Author: github-actions AuthorDate: Wed May 25 19:28:01 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d18ce0be1ca Updating config from bot d18ce0be1ca is described below commit d18ce0be1ca2cf1c3b3cc6ee7b72c5640525ef9c Author: github-actions AuthorDate: Wed May 25 19:33:58 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 7e1136ce8a5 Updating config from bot 7e1136ce8a5 is described below commit 7e1136ce8a54354dbc84fffcb95ca7c985f1eaab Author: github-actions AuthorDate: Wed May 25 19:46:17 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new eb2e4086e47 Updating config from bot eb2e4086e47 is described below commit eb2e4086e47e5f19e4aea38df2472764c4d2d923 Author: github-actions AuthorDate: Wed May 25 19:49:54 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 66b4772eb03 Updating config from bot 66b4772eb03 is described below commit 66b4772eb03592b36a90c847cada6e42c2e85aba Author: github-actions AuthorDate: Wed May 25 20:04:49 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2cda9f8e664 Updating config from bot 2cda9f8e664 is described below commit 2cda9f8e6644a8bebb44ed3df255fc9886c939ce Author: github-actions AuthorDate: Wed May 25 20:16:45 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c24428dd2d0 Updating config from bot c24428dd2d0 is described below commit c24428dd2d0f86b006f84f2364eec5ad88219bde Author: github-actions AuthorDate: Wed May 25 20:30:24 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8a1198bb4df Updating config from bot 8a1198bb4df is described below commit 8a1198bb4df770f7c03bfdf8690e0b3614d5208f Author: github-actions AuthorDate: Wed May 25 20:36:57 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 0f19d27f0b0 Updating config from bot 0f19d27f0b0 is described below commit 0f19d27f0b0c388e9e5e8b81f9ae3b53357f51f5 Author: github-actions AuthorDate: Wed May 25 20:37:44 2022 + Updating config from bot
[beam] branch master updated: [BEAM-14494] Fix publish_docker_images.sh (#17756)
This is an automated email from the ASF dual-hosted git repository.
yichi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 3130e9c02f8 [BEAM-14494] Fix publish_docker_images.sh (#17756)
3130e9c02f8 is described below
commit 3130e9c02f82d919f8975ab3e40d6efd653ea44b
Author: Yichi Zhang
AuthorDate: Wed May 25 13:42:40 2022 -0700
[BEAM-14494] Fix publish_docker_images.sh (#17756)
---
release/src/main/scripts/publish_docker_images.sh | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/release/src/main/scripts/publish_docker_images.sh
b/release/src/main/scripts/publish_docker_images.sh
index 0339f9c627d..b189c9c1bfd 100755
--- a/release/src/main/scripts/publish_docker_images.sh
+++ b/release/src/main/scripts/publish_docker_images.sh
@@ -40,21 +40,21 @@ RC_VERSION="rc${RC_NUM}"
echo "Confirming Release and RC version==="
echo "We are using ${RC_VERSION} to push docker images for ${RELEASE}."
echo "Publishing the following images:"
-IMAGES=$(docker images --filter
"reference=apache/beam_*:${RELEASE}_${RC_VERSION}" --format "{{.Repository}}")
+IMAGES=$(docker images --filter
"reference=apache/beam_*:${RELEASE}${RC_VERSION}" --format "{{.Repository}}")
echo "${IMAGES}"
echo "Do you want to proceed? [y|N]"
read confirmation
if [[ $confirmation = "y" ]]; then
echo "${IMAGES}" | while read IMAGE; do
# Pull verified RC from dockerhub.
-docker pull "${IMAGE}:${RELEASE}_${RC_VERSION}"
+docker pull "${IMAGE}:${RELEASE}${RC_VERSION}"
# Tag with ${RELEASE} and push to dockerhub.
-docker tag "${IMAGE}:${RELEASE}_${RC_VERSION}" "${IMAGE}:${RELEASE}"
+docker tag "${IMAGE}:${RELEASE}${RC_VERSION}" "${IMAGE}:${RELEASE}"
docker push "${IMAGE}:${RELEASE}"
# Tag with latest and push to dockerhub.
-docker tag "${IMAGE}:${RELEASE}_${RC_VERSION}" "${IMAGE}:latest"
+docker tag "${IMAGE}:${RELEASE}${RC_VERSION}" "${IMAGE}:latest"
docker push "${IMAGE}:latest"
done
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8aa50cba4b5 Updating config from bot 8aa50cba4b5 is described below commit 8aa50cba4b59770d95c124858c88a83a99fdb3df Author: github-actions AuthorDate: Wed May 25 21:04:47 2022 + Updating config from bot
[beam] 01/01: Merge pull request #17568 [BEAM-14426] Allow skipping of any output when writing an empty PCollection.
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 4cf268e64c7847276251e7dcad3e1d58866c8234 Merge: 3130e9c02f8 a2c26cd37d9 Author: Robert Bradshaw AuthorDate: Wed May 25 14:27:12 2022 -0700 Merge pull request #17568 [BEAM-14426] Allow skipping of any output when writing an empty PCollection. .../main/java/org/apache/beam/sdk/io/TextIO.java | 14 .../java/org/apache/beam/sdk/io/WriteFiles.java| 17 ++ .../org/apache/beam/sdk/io/TextIOWriteTest.java| 38 -- sdks/python/apache_beam/io/filebasedsink.py| 4 ++- sdks/python/apache_beam/io/iobase.py | 14 +--- sdks/python/apache_beam/io/textio.py | 14 +--- sdks/python/apache_beam/io/textio_test.py | 20 +++- 7 files changed, 108 insertions(+), 13 deletions(-)
[beam] branch master updated (3130e9c02f8 -> 4cf268e64c7)
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 3130e9c02f8 [BEAM-14494] Fix publish_docker_images.sh (#17756) add 59c7a119eb0 [BEAM-14426] Allow skipping of any output when writing an empty PCollection. add a2c26cd37d9 Add skip_if_empty attribute to base class to fix test. new 4cf268e64c7 Merge pull request #17568 [BEAM-14426] Allow skipping of any output when writing an empty PCollection. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../main/java/org/apache/beam/sdk/io/TextIO.java | 14 .../java/org/apache/beam/sdk/io/WriteFiles.java| 17 ++ .../org/apache/beam/sdk/io/TextIOWriteTest.java| 38 -- sdks/python/apache_beam/io/filebasedsink.py| 4 ++- sdks/python/apache_beam/io/iobase.py | 14 +--- sdks/python/apache_beam/io/textio.py | 14 +--- sdks/python/apache_beam/io/textio_test.py | 20 +++- 7 files changed, 108 insertions(+), 13 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new eba4b7bfce8 Updating config from bot eba4b7bfce8 is described below commit eba4b7bfce885d6a16fa248a707de3f39b13734c Author: github-actions AuthorDate: Wed May 25 21:35:39 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 1b8df70861c Updating config from bot 1b8df70861c is described below commit 1b8df70861c0f471f8910bb3c520ea35a3233a48 Author: github-actions AuthorDate: Wed May 25 22:04:53 2022 + Updating config from bot
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f00f68098df Updating config from bot f00f68098df is described below commit f00f68098df4ec2a5266e2ae876501eb489bd811 Author: github-actions AuthorDate: Wed May 25 22:36:57 2022 + Updating config from bot
[beam] branch master updated (4cf268e64c7 -> 45f2dd1e710)
This is an automated email from the ASF dual-hosted git repository. danoliveira pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 4cf268e64c7 Merge pull request #17568 [BEAM-14426] Allow skipping of any output when writing an empty PCollection. add f450881f3f7 Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks new 45f2dd1e710 Merge pull request #17720: Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: sdks/go.mod | 4 ++-- sdks/go.sum | 7 +-- 2 files changed, 7 insertions(+), 4 deletions(-)
[beam] 01/01: Merge pull request #17720: Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks
This is an automated email from the ASF dual-hosted git repository. danoliveira pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 45f2dd1e7100c6616aedc9b0c2c333a88ea30839 Merge: 4cf268e64c7 f450881f3f7 Author: Daniel Oliveira AuthorDate: Wed May 25 16:03:10 2022 -0700 Merge pull request #17720: Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks sdks/go.mod | 4 ++-- sdks/go.sum | 7 +-- 2 files changed, 7 insertions(+), 4 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 60741b4e0c8 Updating config from bot 60741b4e0c8 is described below commit 60741b4e0c84e77c319e1210e3408a23f5503291 Author: github-actions AuthorDate: Wed May 25 23:04:46 2022 + Updating config from bot
[beam] branch dependabot/go_modules/sdks/google.golang.org/api-0.81.0 updated (90bd653f302 -> 46ca24304b3)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/go_modules/sdks/google.golang.org/api-0.81.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 90bd653f302 Bump google.golang.org/api from 0.76.0 to 0.81.0 in /sdks
add 3e683606d9a [BEAM-14496] Ensure that precombine is inheriting one of
the timestamps output values (#17729)
add 57354042908 [BEAM-14139] Remove unused Flink 1.11 directory (#17750)
add 0a6fa95a04d [BEAM-14044] Allow ModelLoader to forward BatchElements
args (#17527)
add 15a5abceaa5 [BEAM-14481] Remove unnecessary context (#17737)
add a73f32d5a07 [BEAM-9324] Fix incompatibility of direct runner with
cython (#17728)
add 24decb50cf3 [BEAM-14503] Add support for Flink 1.15 (#17739)
add 83c85a57993 Update Beam website to release 2.39.0 (#17690)
add 2039b39f96f [BEAM-14509] Add several flags to dataflow runner (#17752)
add 3130e9c02f8 [BEAM-14494] Fix publish_docker_images.sh (#17756)
add 59c7a119eb0 [BEAM-14426] Allow skipping of any output when writing an
empty PCollection.
add a2c26cd37d9 Add skip_if_empty attribute to base class to fix test.
add 4cf268e64c7 Merge pull request #17568 [BEAM-14426] Allow skipping of
any output when writing an empty PCollection.
add f450881f3f7 Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in
/sdks
add 45f2dd1e710 Merge pull request #17720: Bump
cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks
add 46ca24304b3 Bump google.golang.org/api from 0.76.0 to 0.81.0 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (90bd653f302)
\
N -- N -- N
refs/heads/dependabot/go_modules/sdks/google.golang.org/api-0.81.0 (46ca24304b3)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 31 +++-
gradle.properties | 2 +-
release/src/main/scripts/publish_docker_images.sh | 8 +-
.../runners/flink/streaming/StreamSources.java | 91 --
.../streaming/ProcessingTimeCallbackCompat.java} | 6 +-
.../beam/runners/flink/MiniClusterCompat.java} | 16 +-
runners/flink/{1.14 => 1.15}/build.gradle | 8 +-
.../job-server-container/build.gradle | 0
.../flink/{1.12 => 1.15}/job-server/build.gradle | 2 +-
.../streaming/ProcessingTimeCallbackCompat.java} | 6 +-
.../beam/runners/flink/MiniClusterCompat.java} | 17 +-
runners/flink/flink_runner.gradle | 44 +++--
.../flink/FlinkStreamingTransformTranslators.java | 4 +-
.../wrappers/streaming/DoFnOperator.java | 3 +-
.../streaming/io/UnboundedSourceWrapper.java | 4 +-
.../flink/FlinkRequiresStableInputTest.java| 2 +-
.../beam/runners/flink/FlinkSavepointTest.java | 2 +-
sdks/go.mod| 2 +-
sdks/go.sum| 4 +-
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 112 -
sdks/go/pkg/beam/runners/dataflow/dataflow_test.go | 39 -
.../pkg/beam/runners/dataflow/dataflowlib/job.go | 21 ++-
.../main/java/org/apache/beam/sdk/io/TextIO.java | 14 ++
.../java/org/apache/beam/sdk/io/WriteFiles.java| 17 ++
.../org/apache/beam/sdk/io/TextIOWriteTest.java| 38 -
.../beam/fn/harness/PrecombineGroupingTable.java | 26 ++-
.../fn/harness/PrecombineGroupingTableTest.java| 28
sdks/python/apache_beam/io/filebasedsink.py| 4 +-
sdks/python/apache_beam/io/iobase.py | 14 +-
sdks/python/apache_beam/io/textio.py | 14 +-
sdks/python/apache_beam/io/textio_test.py | 20 ++-
sdks/python/apache_beam/ml/inference/base.py | 7 +-
sdks/python/apache_beam/ml/inference/base_test.py | 22 +++
.../python/apache_beam/options/pipeline_options.py | 2 +-
.../apache_beam/runners/worker/bundle_processor.py | 25 ++-
.../apache_beam/runners/worker/operations.py | 36 ++--
settings.gradle.kts| 4 +
website/www/site/config.toml | 2 +-
website/www/site/content/en/blog/beam-2.39.0.md| 185 +
.../www/site/content/en/get-started/downloads.md | 8 +
website/www/site/data/authors.yml | 3 +
41 file
[beam] branch master updated (45f2dd1e710 -> fd854635552)
This is an automated email from the ASF dual-hosted git repository. danoliveira pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 45f2dd1e710 Merge pull request #17720: Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks add 9bcc1ce0279 Fix 2.38.0 download page. add fd854635552 Merge pull request #17759: Fix 2.38.0 download page. No new revisions were added by this update. Summary of changes: website/www/site/content/en/get-started/downloads.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 43a5d83c540 Updating config from bot
43a5d83c540 is described below
commit 43a5d83c540a656c8373347d20b2bb7ef64e2cd6
Author: github-actions
AuthorDate: Wed May 25 23:36:13 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-17751.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-17751.json
b/scripts/ci/pr-bot/state/pr-state/pr-17751.json
index 242a48d7d3b..1c2c2e31975 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-17751.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-17751.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f65a8449c63 Updating config from bot
f65a8449c63 is described below
commit f65a8449c63501f71769f63d1fd09252bf427a3b
Author: github-actions
AuthorDate: Wed May 25 23:36:17 2022 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index c94c6405c2a..0501ec8c3dc 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1653501892515,
-"jrmccluskey": 1653482787841,
+"jrmccluskey": 1653521771922,
"youngoli": 1653489334025,
"damccorm": 1653493136196,
"riteshghorse": 1653500420008
[beam] branch asf-site updated: Publishing website 2022/05/26 04:17:22 at commit fd85463
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 661a369adde Publishing website 2022/05/26 04:17:22 at commit fd85463 661a369adde is described below commit 661a369added3aabe1fff771eae399e0734a6b60 Author: jenkins AuthorDate: Thu May 26 04:17:23 2022 + Publishing website 2022/05/26 04:17:22 at commit fd85463 --- website/generated-content/get-started/downloads/index.html | 6 +++--- website/generated-content/get-started/index.xml| 6 +++--- website/generated-content/sitemap.xml | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/website/generated-content/get-started/downloads/index.html b/website/generated-content/get-started/downloads/index.html index c9e86c088f0..19931a7d8fd 100644 --- a/website/generated-content/get-started/downloads/index.html +++ b/website/generated-content/get-started/downloads/index.html @@ -53,9 +53,9 @@ may change at any point and are not guaranteed to remain compatible across versi versions denoted 0.x.y.Releases2.39.0 (2022-05-25)Official https://www.apache.org/dyn/closer.cgi/beam/2.39.0/apache-beam-2.39.0-source-release.zip>source code download. https://downloads.apache.org/beam/2.39.0/apache-beam-2.39.0-source-release.zip.sha512>SHA-512. https://downloads.apache.org/beam/2.39.0/apache-beam-2.39.0-source-release.zip.asc>signature.https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351169";>Release notes -Blog post.2.38.0 (2022-04-20)Official https://www.apache.org/dyn/closer.cgi/beam/2.38.0/apache-beam-2.38.0-source-release.zip>source code download. -https://downloads.apache.org/beam/2.38.0/apache-beam-2.38.0-source-release.zip.sha512>SHA-512. -https://downloads.apache.org/beam/2.38.0/apache-beam-2.38.0-source-release.zip.asc>signature.https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351169";>Release notes +Blog post.2.38.0 (2022-04-20)Official https://archive.apache.org/dist/beam/2.38.0/apache-beam-2.38.0-source-release.zip>source code download. +https://archive.apache.org/dist/beam/2.38.0/apache-beam-2.38.0-source-release.zip.sha512>SHA-512. +https://archive.apache.org/dist/beam/2.38.0/apache-beam-2.38.0-source-release.zip.asc>signature.https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351169";>Release notes Blog post.2.37.0 (2022-03-04)Official https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip>source code download. https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip.sha512>SHA-512. https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip.asc>signature.https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351168";>Release notes diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml index d3fb575f7bd..c6f439ecd48 100644 --- a/website/generated-content/get-started/index.xml +++ b/website/generated-content/get-started/index.xml @@ -2125,9 +2125,9 @@ versions denoted0.x.y.2.38.0 (2022-04-20)
-Official source code download. -SHA-512. -signature.
+Official source code download. +SHA-512. +signature.
2.37.0 (2022-03-04)
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index c171a02e96a..560cb55d689 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9"; xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.39.0/2022-05-25T11:28:33-07:00/categories/blog/2022-05-25T11:28:33-07:00/blog/2022-05-25T11:28:33-07:00/categories/2022-05-25T11:28:33-07:
[beam] branch nightly-refs/heads/master updated (d50f1b1790a -> fd854635552)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from d50f1b1790a [BEAM-14129] Clean up PubsubLiteIO by removing options
that no longer apply (#17169)
add 3e683606d9a [BEAM-14496] Ensure that precombine is inheriting one of
the timestamps output values (#17729)
add 57354042908 [BEAM-14139] Remove unused Flink 1.11 directory (#17750)
add 0a6fa95a04d [BEAM-14044] Allow ModelLoader to forward BatchElements
args (#17527)
add 15a5abceaa5 [BEAM-14481] Remove unnecessary context (#17737)
add a73f32d5a07 [BEAM-9324] Fix incompatibility of direct runner with
cython (#17728)
add 24decb50cf3 [BEAM-14503] Add support for Flink 1.15 (#17739)
add 83c85a57993 Update Beam website to release 2.39.0 (#17690)
add 2039b39f96f [BEAM-14509] Add several flags to dataflow runner (#17752)
add 3130e9c02f8 [BEAM-14494] Fix publish_docker_images.sh (#17756)
add 59c7a119eb0 [BEAM-14426] Allow skipping of any output when writing an
empty PCollection.
add a2c26cd37d9 Add skip_if_empty attribute to base class to fix test.
add 4cf268e64c7 Merge pull request #17568 [BEAM-14426] Allow skipping of
any output when writing an empty PCollection.
add f450881f3f7 Bump cloud.google.com/go/storage from 1.22.0 to 1.22.1 in
/sdks
add 45f2dd1e710 Merge pull request #17720: Bump
cloud.google.com/go/storage from 1.22.0 to 1.22.1 in /sdks
add 9bcc1ce0279 Fix 2.38.0 download page.
add fd854635552 Merge pull request #17759: Fix 2.38.0 download page.
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 31 +++-
gradle.properties | 2 +-
release/src/main/scripts/publish_docker_images.sh | 8 +-
.../runners/flink/streaming/StreamSources.java | 91 --
.../streaming/ProcessingTimeCallbackCompat.java} | 6 +-
.../beam/runners/flink/MiniClusterCompat.java} | 16 +-
runners/flink/{1.14 => 1.15}/build.gradle | 8 +-
.../job-server-container/build.gradle | 0
.../flink/{1.12 => 1.15}/job-server/build.gradle | 2 +-
.../streaming/ProcessingTimeCallbackCompat.java} | 6 +-
.../beam/runners/flink/MiniClusterCompat.java} | 17 +-
runners/flink/flink_runner.gradle | 44 +++--
.../flink/FlinkStreamingTransformTranslators.java | 4 +-
.../wrappers/streaming/DoFnOperator.java | 3 +-
.../streaming/io/UnboundedSourceWrapper.java | 4 +-
.../flink/FlinkRequiresStableInputTest.java| 2 +-
.../beam/runners/flink/FlinkSavepointTest.java | 2 +-
sdks/go.mod| 4 +-
sdks/go.sum| 7 +-
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 112 -
sdks/go/pkg/beam/runners/dataflow/dataflow_test.go | 39 -
.../pkg/beam/runners/dataflow/dataflowlib/job.go | 21 ++-
.../main/java/org/apache/beam/sdk/io/TextIO.java | 14 ++
.../java/org/apache/beam/sdk/io/WriteFiles.java| 17 ++
.../org/apache/beam/sdk/io/TextIOWriteTest.java| 38 -
.../beam/fn/harness/PrecombineGroupingTable.java | 26 ++-
.../fn/harness/PrecombineGroupingTableTest.java| 28
sdks/python/apache_beam/io/filebasedsink.py| 4 +-
sdks/python/apache_beam/io/iobase.py | 14 +-
sdks/python/apache_beam/io/textio.py | 14 +-
sdks/python/apache_beam/io/textio_test.py | 20 ++-
sdks/python/apache_beam/ml/inference/base.py | 7 +-
sdks/python/apache_beam/ml/inference/base_test.py | 22 +++
.../python/apache_beam/options/pipeline_options.py | 2 +-
.../apache_beam/runners/worker/bundle_processor.py | 25 ++-
.../apache_beam/runners/worker/operations.py | 36 ++--
settings.gradle.kts| 4 +
website/www/site/config.toml | 2 +-
website/www/site/content/en/blog/beam-2.39.0.md| 185 +
.../www/site/content/en/get-started/downloads.md | 14 +-
website/www/site/data/authors.yml | 3 +
41 files changed, 650 insertions(+), 254 deletions(-)
delete mode 100644
runners/flink/1.11/src/test/java/org/apache/beam/runners/flink/streaming/StreamSources.java
copy
runners/flink/{src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/package-info.java
=>
1.12/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/ProcessingTimeCallbackCompat.java}
(85%)
copy
runners/flink/{src/test/java/org/apache/beam/runners/flink/RemoteMiniCluster.java
=> 1.12/src/test/java/org/apache/beam/runners/flink/MiniClusterCompat.java}
(70%)
copy runners/flink/{1.14 => 1.15}/build.gradle (84%)
copy runners/flink/{1.12 => 1.15}/job-server-container/build.gradle (100%)
copy
