[beam] branch asf-site updated: Publishing website 2023/04/15 05:07:13 at commit f5f7a47
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 74e67cdbbba Publishing website 2023/04/15 05:07:13 at commit f5f7a47 74e67cdbbba is described below commit 74e67cdbbbaf5880fb4e50ce3cf790a2a7e77b62 Author: jenkins AuthorDate: Sat Apr 15 05:07:14 2023 + Publishing website 2023/04/15 05:07:13 at commit f5f7a47 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 4adc268f96f..e3de27333a8 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.46.0/2023-04-14T17:23:30-04:00/categories/blog/2023-04-14T17:23:30-04:00/blog/2023-04-14T17:23:30-04:00/categories/2023-04-14T17:23:30-04:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.46.0/2023-04-14T19:24:25-07:00/categories/blog/2023-04-14T19:24:25-07:00/blog/2023-04-14T19:24:25-07:00/categories/2023-04-14T19:24:25-07:00/catego [...] \ No newline at end of file
[beam] branch master updated: Merge pull request #26284: Fix GroupIntoBatches hold
This is an automated email from the ASF dual-hosted git repository.
reuvenlax pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new f5f7a471321 Merge pull request #26284: Fix GroupIntoBatches hold
f5f7a471321 is described below
commit f5f7a471321c903174b452a1982c5183a79ac6bc
Author: Reuven Lax
AuthorDate: Fri Apr 14 19:24:25 2023 -0700
Merge pull request #26284: Fix GroupIntoBatches hold
---
.../java/org/apache/beam/sdk/transforms/GroupIntoBatches.java | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java
index 311a3dac6ca..3616cc2e59f 100644
---
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java
+++
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/GroupIntoBatches.java
@@ -580,6 +580,7 @@ public class GroupIntoBatches
timerTs,
minBufferedTs);
bufferingTimer.clear();
+holdTimer.clear();
}
}
@@ -593,13 +594,18 @@ public class GroupIntoBatches
@StateId(NUM_BYTES_IN_BATCH_ID) CombiningState
storedBatchSizeBytes,
@StateId(TIMER_TIMESTAMP) ValueState timerTs,
@StateId(MIN_BUFFERED_TS) CombiningState
minBufferedTs,
-@TimerId(END_OF_BUFFERING_ID) Timer bufferingTimer) {
+@TimerId(END_OF_BUFFERING_ID) Timer bufferingTimer,
+@TimerId(TIMER_HOLD_ID) Timer holdTimer) {
LOG.debug(
"*** END OF BUFFERING *** for timer timestamp {} with buffering
duration {}",
timestamp,
maxBufferingDuration);
flushBatch(
receiver, key, batch, storedBatchSize, storedBatchSizeBytes,
timerTs, minBufferedTs);
+ // Generally this is a noop, since holdTimer is not set if
bufferingTimer is set. However we
+ // delete the holdTimer
+ // here in order to allow users to modify this policy on pipeline update.
+ holdTimer.clear();
}
@OnWindowExpiration
[beam] branch master updated: Add ValidatesContainer tests with installing release candidates(RCs) (#26156)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new e86486dd6aa Add ValidatesContainer tests with installing release
candidates(RCs) (#26156)
e86486dd6aa is described below
commit e86486dd6aaac6641e5571fd7417be4eea5d7ce5
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Fri Apr 14 21:23:06 2023 -0400
Add ValidatesContainer tests with installing release candidates(RCs)
(#26156)
Co-authored-by: tvalentyn
---
...ommit_Python_ValidatesContainer_Dataflow.groovy | 22 +++
sdks/python/container/Dockerfile | 2 --
sdks/python/container/common.gradle| 3 ++-
sdks/python/container/run_generate_requirements.sh | 31 +-
sdks/python/test-suites/dataflow/common.gradle | 8 +-
5 files changed, 49 insertions(+), 17 deletions(-)
diff --git
a/.test-infra/jenkins/job_PostCommit_Python_ValidatesContainer_Dataflow.groovy
b/.test-infra/jenkins/job_PostCommit_Python_ValidatesContainer_Dataflow.groovy
index bc43ecb366f..25409644866 100644
---
a/.test-infra/jenkins/job_PostCommit_Python_ValidatesContainer_Dataflow.groovy
+++
b/.test-infra/jenkins/job_PostCommit_Python_ValidatesContainer_Dataflow.groovy
@@ -43,3 +43,25 @@
PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_ValCont',
}
}
}
+
+PostcommitJobBuilder.postCommitJob('beam_PostCommit_Py_ValCont_with_RC',
+'Run Python RC Dataflow ValidatesContainer', 'Google Cloud Dataflow Runner
Python ValidatesContainer Tests with RC Dependencies', this) {
+ description('Runs Python ValidatesContainer suite on the Dataflow runner
by installing Release Candidates.')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ publishers {
+archiveJunit('**/pytest*.xml')
+ }
+
+ // Execute shell command to test Python SDK.
+ steps {
+gradle {
+ rootBuildScriptDir(commonJobProperties.checkoutDir)
+ tasks(':sdks:python:test-suites:dataflow:validatesContainerTests')
+ switches('-PtestRCDependencies=true')
+ commonJobProperties.setGradleSwitches(delegate)
+}
+ }
+}
diff --git a/sdks/python/container/Dockerfile b/sdks/python/container/Dockerfile
index 8dcfd5b74dd..ff3b4484950 100644
--- a/sdks/python/container/Dockerfile
+++ b/sdks/python/container/Dockerfile
@@ -109,5 +109,3 @@ COPY --from=third_party_licenses
/opt/apache/beam/third_party_licenses /opt/apac
RUN if [ "$pull_licenses" != "true" ] ; then \
rm -rf /opt/apache/beam/third_party_licenses ; \
fi
-
-
diff --git a/sdks/python/container/common.gradle
b/sdks/python/container/common.gradle
index c10211950f8..48c059ae5ec 100644
--- a/sdks/python/container/common.gradle
+++ b/sdks/python/container/common.gradle
@@ -32,13 +32,14 @@ dependencies {
def generatePythonRequirements = tasks.register("generatePythonRequirements") {
dependsOn ':sdks:python:sdist'
+ def pipExtraOptions = project.hasProperty("testRCDependencies") ? "--pre" :
""
def runScriptsPath =
"${rootDir}/sdks/python/container/run_generate_requirements.sh"
doLast {
exec {
executable 'sh'
args '-c', "cd ${rootDir} && ${runScriptsPath} " +
"${project.ext.pythonVersion} " +
- "${files(configurations.sdkSourceTarball.files).singleFile}"
+ "${files(configurations.sdkSourceTarball.files).singleFile} " +
"${pipExtraOptions}"
}
}
}
diff --git a/sdks/python/container/run_generate_requirements.sh
b/sdks/python/container/run_generate_requirements.sh
index 55173bb8cd9..fd222107dba 100755
--- a/sdks/python/container/run_generate_requirements.sh
+++ b/sdks/python/container/run_generate_requirements.sh
@@ -16,7 +16,7 @@
#limitations under the License.
#
-# Generates requirements files, which list PyPI depenedncies to install in
+# Generates requirements files, which list PyPI dependencies to install in
# Apache Beam Python SDK container images. To generate the list,
# we use two sources of information:
# 1) Requirements of Apache Beam itself, as defined by setup.py.
@@ -30,21 +30,25 @@
# You will need Python interpreters for all versions supported by Beam, see:
# https://s.apache.org/beam-python-dev-wiki
-if [[ $# != 2 ]]; then
+if [[ $# -lt 2 ]]; then
printf "Example usage: \n$>
./sdks/python/container/run_generate_requirements.sh 3.8 "
- printf "\n\twhere 3.8 is the Python major.minor version."
+ printf "\n\where 3.8 is the Python major.minor version."
exit 1
fi
PY_VERSION=$1
SDK_TARBALL=$2
+# Use the PIP_EXTRA_OPTIONS environment variable to pass additional flags to
the pip install command.
+# For example, you can include the --pre flag in $PIP_EXTRA_OPTIONS to
download
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 60d4bbb6beb Updating config from bot 60d4bbb6beb is described below commit 60d4bbb6beb087a82f9a927d7927b91e8f5b9a26 Author: github-actions AuthorDate: Sat Apr 15 00:50:17 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-26289.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26289.json b/scripts/ci/pr-bot/state/pr-state/pr-26289.json index fa3809f24dd..a14d93d95e3 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-26289.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-26289.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 17a5e65e66c Updating config from bot
17a5e65e66c is described below
commit 17a5e65e66cd56d3145e092462febe946bfac2bb
Author: github-actions
AuthorDate: Sat Apr 15 00:20:38 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26289.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26289.json
b/scripts/ci/pr-bot/state/pr-state/pr-26289.json
new file mode 100644
index 000..fa3809f24dd
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26289.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "kennknowles"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e384e717b22 Updating config from bot
e384e717b22 is described below
commit e384e717b22ec1c7d4e69fa998b75592c9afa552
Author: github-actions
AuthorDate: Sat Apr 15 00:20:40 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 5edaac3249f..1ae562aa3c0 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -2,7 +2,7 @@
"label": "java",
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
-"kennknowles": 1681407270922,
+"kennknowles": 1681518036229,
"robertwb": 1681504477670,
"kileys": 1674428678843,
"apilloud": 1678822446183,
[beam] branch asf-site updated: Publishing website 2023/04/14 22:16:59 at commit b9f27f9
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new f49834201a5 Publishing website 2023/04/14 22:16:59 at commit b9f27f9 f49834201a5 is described below commit f49834201a50dd5909e6cd85ba336067c248d56c Author: jenkins AuthorDate: Fri Apr 14 22:16:59 2023 + Publishing website 2023/04/14 22:16:59 at commit b9f27f9 --- .../documentation/sdks/python-machine-learning/index.html | 2 +- website/generated-content/sitemap.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/website/generated-content/documentation/sdks/python-machine-learning/index.html b/website/generated-content/documentation/sdks/python-machine-learning/index.html index 88d5f7c52f3..04aa8a31a89 100644 --- a/website/generated-content/documentation/sdks/python-machine-learning/index.html +++ b/website/generated-content/documentation/sdks/python-machine-learning/index.html @@ -90,7 +90,7 @@ with pipeline as p: If you need to use this object explicitly, include the following line in your pipeline to import the object:from apache_beam.ml.inference.base import PredictionResult For more information, see the https://github.com/apache/beam/blob/master/sdks/python/apache_beam/ml/inference/base.py#L65>PredictionResult documentation.Run a machine learning pipelineFor detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference>Example RunInference API pipelines on GitHub.Automatic model refreshTo automatically update the models used with the RunInference PTransform without stopping the Beam pipeline, pass a https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata>ModelMetadata< [...] -an update to the side input. This could happen with global windowed side inputs with data driven triggers, such as AfterCount and AfterProcessingTime. Until the side input is updated, emit the default or initial model ID that is used to pass the respective ModelHandler as a side input.Beam Java SDK supportThe RunInference API is available with the Beam Java SDK versions 2.41.0 and later through Apache Beam [...] +an update to the side input. This could happen with global windowed side inputs with data driven triggers, such as AfterCount and AfterProcessingTime. Until the side input is updated, emit the default or initial model ID that is used to pass the respective ModelHandler as a side input.Beam Java SDK supportThe RunInference API is available with the Beam Java SDK versions 2.41.0 and later through Apache Beam [...] BatchElements PTransforms. For an example, see our https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_language_modeling.py>language modeling example.Related linksRunInference transformshttps://github.com/apache/beam/tree/master/sdks/python/apache_ [...] Pydochttps://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/python/transforms/RunInference.html>https://beam.apache.org/images/logos/sdks/java.png width=20px height=30px alt=Javadoc> Javadochttp://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.46.0/2023-04-14T12:06:17-04:00/categories/blog/2023-04-14T12:06:17-04:00/blog/2023-04-14T12:06:17-04:00/categories/2023-04-14T12:06:17-04:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.46.0/2023-04-14T17:23:30-04:00/categories/blog/2023-04-14T17:23:30-04:00/blog/2023-04-14T17:23:30-04:00/categories/2023-04-14T17:23:30-04:00/catego [...] \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 68f270ed10e Updating config from bot
68f270ed10e is described below
commit 68f270ed10e8076ab1550076e9cdf1c4ca55311c
Author: github-actions
AuthorDate: Fri Apr 14 21:34:37 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26287.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26287.json
b/scripts/ci/pr-bot/state/pr-state/pr-26287.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26287.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: DLQ support in RunInference (#26261)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new b9f27f9da2e DLQ support in RunInference (#26261) b9f27f9da2e is described below commit b9f27f9da2e63b564feecaeb593d7b12783192b0 Author: Danny McCormick AuthorDate: Fri Apr 14 17:23:30 2023 -0400 DLQ support in RunInference (#26261) * DLQ support in RunInference * Doc example * Comment * CHANGES.md --- CHANGES.md| 2 +- sdks/python/apache_beam/ml/inference/base.py | 76 +++ sdks/python/apache_beam/ml/inference/base_test.py | 28 - 3 files changed, 92 insertions(+), 14 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 3113c875005..7be9752517d 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -64,7 +64,7 @@ ## New Features / Improvements -* X feature added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)). +* Dead letter queue support added to RunInference in Python ([#24209](https://github.com/apache/beam/issues/24209)). ## Breaking Changes diff --git a/sdks/python/apache_beam/ml/inference/base.py b/sdks/python/apache_beam/ml/inference/base.py index 50056107702..5bc3bb31557 100644 --- a/sdks/python/apache_beam/ml/inference/base.py +++ b/sdks/python/apache_beam/ml/inference/base.py @@ -339,6 +339,7 @@ class RunInference(beam.PTransform[beam.PCollection[ExampleT], self._metrics_namespace = metrics_namespace self._model_metadata_pcoll = model_metadata_pcoll self._enable_side_input_loading = self._model_metadata_pcoll is not None +self._with_exception_handling = False # TODO(BEAM-14046): Add and link to help documentation. @classmethod @@ -368,20 +369,71 @@ class RunInference(beam.PTransform[beam.PCollection[ExampleT], # batching DoFn APIs. | beam.BatchElements(**self._model_handler.batch_elements_kwargs())) +run_inference_pardo = beam.ParDo( +_RunInferenceDoFn( +self._model_handler, +self._clock, +self._metrics_namespace, +self._enable_side_input_loading), +self._inference_args, +beam.pvalue.AsSingleton( +self._model_metadata_pcoll, +) if self._enable_side_input_loading else None).with_resource_hints( +**resource_hints) + +if self._with_exception_handling: + run_inference_pardo = run_inference_pardo.with_exception_handling( + exc_class=self._exc_class, + use_subprocess=self._use_subprocess, + threshold=self._threshold) + return ( batched_elements_pcoll -| 'BeamML_RunInference' >> ( -beam.ParDo( -_RunInferenceDoFn( -self._model_handler, -self._clock, -self._metrics_namespace, -self._enable_side_input_loading), -self._inference_args, -beam.pvalue.AsSingleton( -self._model_metadata_pcoll, -) if self._enable_side_input_loading else -None).with_resource_hints(**resource_hints))) +| 'BeamML_RunInference' >> run_inference_pardo) + + def with_exception_handling( + self, *, exc_class=Exception, use_subprocess=False, threshold=1): +"""Automatically provides a dead letter output for skipping bad records. +This can allow a pipeline to continue successfully rather than fail or +continuously throw errors on retry when bad elements are encountered. + +This returns a tagged output with two PCollections, the first being the +results of successfully processing the input PCollection, and the second +being the set of bad batches of records (those which threw exceptions +during processing) along with information about the errors raised. + +For example, one would write:: + +good, bad = RunInference( + maybe_error_raising_model_handler +).with_exception_handling() + +and `good` will be a PCollection of PredictionResults and `bad` will +contain a tuple of all batches that raised exceptions, along with their +corresponding exception. + + +Args: + exc_class: An exception class, or tuple of exception classes, to catch. + Optional, defaults to 'Exception'. + use_subprocess: Whether to execute the DoFn logic in a subprocess. This + allows one to recover from errors that can crash the calling process + (e.g. from an underlying library causing a segfault), but is + slower as elements and results must cross a process boundary. Note + that this starts up a long-running process that is used to handle + all the elements (until hard failure, which should be rare) rather + than a new process per
[beam] branch master updated (0bf9d55ff41 -> 82699bed49c)
This is an automated email from the ASF dual-hosted git repository. chamikara pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 0bf9d55ff41 Download codecov uploader since codecov has been removed from pypi (#26282) add 82699bed49c Adds a reference to new Java RunInference example (#26285) No new revisions were added by this update. Summary of changes: .../www/site/content/en/documentation/sdks/python-machine-learning.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 8bf919029db Updating config from bot
8bf919029db is described below
commit 8bf919029db6acdd7d3f9b6c8f7bd8a26087eb9a
Author: github-actions
AuthorDate: Fri Apr 14 20:34:43 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 0ede052cb3d..aa66917ae1e 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"chamikaramj": 1681407270922,
"johnjcasey": 1681425285966,
-"pabloem": 1681241693255,
+"pabloem": 1681504477670,
"Abacn": 1681428879964,
"ahmedabu98": 1681324530501
}
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 052046e7fb7 Updating config from bot
052046e7fb7 is described below
commit 052046e7fb72b5617abd335c956e1d1d581d2535
Author: github-actions
AuthorDate: Fri Apr 14 20:34:41 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 9eb2e7ec110..5edaac3249f 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
"kennknowles": 1681407270922,
-"robertwb": 1681382072223,
+"robertwb": 1681504477670,
"kileys": 1674428678843,
"apilloud": 1678822446183,
"Abacn": 1681428879964
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9b34e48b9cb Updating config from bot
9b34e48b9cb is described below
commit 9b34e48b9cb7c9f2e17c940b0a6759bfb3f43f52
Author: github-actions
AuthorDate: Fri Apr 14 20:34:39 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26286.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26286.json
b/scripts/ci/pr-bot/state/pr-state/pr-26286.json
new file mode 100644
index 000..1d58554d958
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26286.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "robertwb",
+"io": "pabloem"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
svn commit: r61285 - in /dev/beam/vendor/beam-vendor-grpc-1_54_0: ./ 0.1/
Author: yhu Date: Fri Apr 14 20:19:25 2023 New Revision: 61285 Log: Source release for vendor grpc 1.54.0 Added: dev/beam/vendor/beam-vendor-grpc-1_54_0/ dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/ dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip (with props) dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.asc dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.sha512 Added: dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip == Binary file - no diff available. Propchange: dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.asc == --- dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.asc (added) +++ dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.asc Fri Apr 14 20:19:25 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIBHsk2MD2aHbZi7hy2l0yBcEBcsFAmQ5tBsACgkQy2l0yBcE +BctERg/7BFTeMsSPgwDGJDLMd2XknLcfdnGjA/F7aKI3xpDEzPaKveR5SC+t0r24 +7ayrmCXPj46PESrGVLIo1J7xK+1P804joC3T7SWB5WS8muUlZ65PF+UcOV/CEASo +5mj8uGkz7yXCxgIUGNO+eIyUAU5+GHyfnHEqu+Y22NScK8WQWnLBVgCjCZ9h8xba +SyS+jOJFrfUqCdBaXuujo1jRx5uD+ZnQWV4IDrmQxyXb80KwBp22aq32nRKixzdf +7+buVM//8XKbLyBj6EoD2huS3c8OzxQ7hMzSTg7LV/8H8aMmwVIs1n6plYrz/EJn +N68IIHUwlV4iNQXMhGp2U6XXxr05hqkphsRrwZeSCbWkHfdiCBfMSOE+rowy/Uz4 +XOdglDnSbxMbeyE7jzkGtqQPXVtJozZx03Cn5zLsZrYBVOtijgW/ZfGKVA1FGrpC +OqQGJV7IesPX5BboySylJZh/Xsapnn0+sxyCl+zIVp+zA6ZAXyd/aUqnDuIeeG79 +TJi8+gwwM4F9yVXaCHX6KYV8vdjJZw83FbNGAK+eAQ+qfXoxJLOTA3/VQQ9P3/xk +gxXxO6yNSEYsmX7L7bTPElprntDaTWoAWSDfTp/5lv3CfnEIxbnCgjaP6T9LCSD+ +k5p5zpBEqi09IrbbdvLQr3GHOiQ9M6s/wAD5hkeVVgk8TyaFUuM= +=c/ap +-END PGP SIGNATURE- Added: dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.sha512 == --- dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.sha512 (added) +++ dev/beam/vendor/beam-vendor-grpc-1_54_0/0.1/apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip.sha512 Fri Apr 14 20:19:25 2023 @@ -0,0 +1 @@ +7435fb18110a72eb4d48afc9aec8b785852ce3e0d37917ea0c7fedf5150d053e8626ae20666c4aa63057e419f2af1e1cc017ed6f16860553dd2730461d750be1 apache-beam-a38d9b94a738e4c488e7339ae3710fd5e1dc119e-source-release.zip
[beam] branch master updated (a38d9b94a73 -> 0bf9d55ff41)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from a38d9b94a73 Vendor grpc 1.54.0 (#26271) add 0bf9d55ff41 Download codecov uploader since codecov has been removed from pypi (#26282) No new revisions were added by this update. Summary of changes: sdks/python/tox.ini | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-)
[beam] branch master updated: Vendor grpc 1.54.0 (#26271)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new a38d9b94a73 Vendor grpc 1.54.0 (#26271)
a38d9b94a73 is described below
commit a38d9b94a738e4c488e7339ae3710fd5e1dc119e
Author: Yi Hu
AuthorDate: Fri Apr 14 15:06:52 2023 -0400
Vendor grpc 1.54.0 (#26271)
---
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 1 +
.../apache/beam/gradle/GrpcVendoring_1_54_0.groovy | 204 +
settings.gradle.kts| 1 +
vendor/README.md | 8 +-
vendor/grpc-1_54_0/build.gradle| 37
5 files changed, 247 insertions(+), 4 deletions(-)
diff --git
a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
index 9f8537aed3f..60f01846709 100644
--- a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
+++ b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
@@ -815,6 +815,7 @@ class BeamModulePlugin implements Plugin {
testcontainers_gcloud :
"org.testcontainers:gcloud:$testcontainers_version",
testcontainers_rabbitmq :
"org.testcontainers:rabbitmq:$testcontainers_version",
vendored_grpc_1_48_1:
"org.apache.beam:beam-vendor-grpc-1_48_1:0.1",
+vendored_grpc_1_54_0:
"org.apache.beam:beam-vendor-grpc-1_54_0:0.1",
vendored_guava_26_0_jre :
"org.apache.beam:beam-vendor-guava-26_0-jre:0.1",
vendored_calcite_1_28_0 :
"org.apache.beam:beam-vendor-calcite-1_28_0:0.2",
woodstox_core_asl :
"org.codehaus.woodstox:woodstox-core-asl:4.4.1",
diff --git
a/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_54_0.groovy
b/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_54_0.groovy
new file mode 100644
index 000..ddb317b8929
--- /dev/null
+++
b/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_54_0.groovy
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.gradle
+
+/**
+ * Utilities for working with our vendored version of gRPC.
+ *
+ * To update:
+ * 1. Determine the set of io.grpc libraries we want to include, most likely a
superset of
+ *of the previous vendored gRPC version.
+ * 2. Use mvn dependency:tree and https://search.maven.org/search?q=g:io.grpc
+ *to determine dependency tree. You may need to search for optional
dependencies
+ *and determine if they are needed (e.g. conscrypt).
+ * 3. Validate built artifacts by running linkage tool
+ *
(https://github.com/apache/beam/tree/master/vendor#how-to-validate-the-vendored-dependencies)
+ *and unit and integration tests in a PR (e.g.
https://github.com/apache/beam/pull/16460,
+ *https://github.com/apache/beam/pull/16459)
+ */
+class GrpcVendoring_1_54_0 {
+ static def grpc_version = "1.54.0"
+
+ // See
https://github.com/grpc/grpc-java/blob/v1.54.0/gradle/libs.versions.toml
+ // or https://search.maven.org/search?q=g:io.grpc%201.54.0
+ static def guava_version = "31.1-jre"
+ static def protobuf_version = "3.21.7"
+ static def gson_version = "2.9.0"
+ static def google_auth_version = "1.4.0"
+ static def opencensus_version = "0.31.0"
+ static def conscrypt_version = "2.5.2"
+ static def proto_google_common_protos_version = "2.9.0"
+ static def netty_version = "4.1.87.Final"
+ static def netty_tcnative_version = "2.0.56.Final"
+
+ /** Returns the list of implementation time dependencies. */
+ static List dependencies() {
+return [
+ "com.google.guava:guava:$guava_version",
+ "com.google.protobuf:protobuf-java:$protobuf_version",
+ "com.google.protobuf:protobuf-java-util:$protobuf_version",
+ "com.google.code.gson:gson:$gson_version",
+ "io.grpc:grpc-auth:$grpc_version",
+ "io.grpc:grpc-core:$grpc_version",
+
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 14ea01550a9 Updating config from bot 14ea01550a9 is described below commit 14ea01550a9fbcb875eaadaed26b13efdbd21957 Author: github-actions AuthorDate: Fri Apr 14 19:05:37 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 57c60ed44fb..815c6b7af0c 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1681405537629, +"tvalentyn": 1681499133723, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1681488398413,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 430fe07b570 Updating config from bot
430fe07b570 is described below
commit 430fe07b57015d325b582da63d9dae3ea13c6745
Author: github-actions
AuthorDate: Fri Apr 14 19:05:35 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26282.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26282.json
b/scripts/ci/pr-bot/state/pr-state/pr-26282.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26282.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Refactor DoFnOp.FutureCollectorImpl to a top level class in SamzaRunner (#26274)
This is an automated email from the ASF dual-hosted git repository.
xinyu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 78b5ffbb82c Refactor DoFnOp.FutureCollectorImpl to a top level class
in SamzaRunner (#26274)
78b5ffbb82c is described below
commit 78b5ffbb82c7220c4a0c6c4755f23daccb53a159
Author: Katie Liu
AuthorDate: Fri Apr 14 11:36:27 2023 -0700
Refactor DoFnOp.FutureCollectorImpl to a top level class in SamzaRunner
(#26274)
---
.../apache/beam/runners/samza/runtime/DoFnOp.java | 74
.../runners/samza/runtime/FutureCollectorImpl.java | 99 ++
.../runners/samza/runtime/AsyncDoFnRunnerTest.java | 2 +-
.../samza/runtime/FutureCollectorImplTest.java | 6 +-
4 files changed, 103 insertions(+), 78 deletions(-)
diff --git
a/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/DoFnOp.java
b/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/DoFnOp.java
index c693754b5b9..5ee763fca34 100644
---
a/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/DoFnOp.java
+++
b/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/DoFnOp.java
@@ -26,9 +26,7 @@ import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.ServiceLoader;
-import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
-import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Function;
import org.apache.beam.model.pipeline.v1.RunnerApi;
import org.apache.beam.runners.core.DoFnRunner;
@@ -46,7 +44,6 @@ import
org.apache.beam.runners.fnexecution.provisioning.JobInfo;
import org.apache.beam.runners.samza.SamzaExecutionContext;
import org.apache.beam.runners.samza.SamzaPipelineOptions;
import org.apache.beam.runners.samza.util.DoFnUtils;
-import org.apache.beam.runners.samza.util.FutureUtils;
import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.DoFnSchemaInformation;
@@ -484,77 +481,6 @@ public class DoFnOp implements Op {
windowedValue.getPane()));
}
- static class FutureCollectorImpl implements FutureCollector {
-private final AtomicBoolean collectorSealed;
-private CompletionStage>> outputFuture;
-
-FutureCollectorImpl() {
- outputFuture = CompletableFuture.completedFuture(new ArrayList<>());
- collectorSealed = new AtomicBoolean(true);
-}
-
-@Override
-public void add(CompletionStage> element) {
- checkState(
- !collectorSealed.get(),
- "Cannot add element to an unprepared collector. Make sure prepare()
is invoked before adding elements.");
-
- // We need synchronize guard against scenarios when watermark/finish
bundle trigger outputs.
- synchronized (this) {
-outputFuture =
-outputFuture.thenCombine(
-element,
-(collection, event) -> {
- collection.add(event);
- return collection;
-});
- }
-}
-
-@Override
-public void addAll(CompletionStage>>
elements) {
- checkState(
- !collectorSealed.get(),
- "Cannot add elements to an unprepared collector. Make sure prepare()
is invoked before adding elements.");
-
- synchronized (this) {
-outputFuture = FutureUtils.combineFutures(outputFuture, elements);
- }
-}
-
-@Override
-public void discard() {
- collectorSealed.compareAndSet(false, true);
-
- synchronized (this) {
-outputFuture = CompletableFuture.completedFuture(new ArrayList<>());
- }
-}
-
-@Override
-public CompletionStage>> finish() {
- /*
- * We can ignore the results here because its okay to call finish
without invoking prepare. It will be a no-op
- * and an empty collection will be returned.
- */
- collectorSealed.compareAndSet(false, true);
-
- synchronized (this) {
-final CompletionStage>>
sealedOutputFuture = outputFuture;
-outputFuture = CompletableFuture.completedFuture(new ArrayList<>());
-return sealedOutputFuture;
- }
-}
-
-@Override
-public void prepare() {
- boolean isCollectorSealed = collectorSealed.compareAndSet(true, false);
- checkState(
- isCollectorSealed,
- "Failed to prepare the collector. Collector needs to be sealed
before prepare() is invoked.");
-}
- }
-
/**
* Factory class to create an {@link
org.apache.beam.runners.core.DoFnRunners.OutputManager} that
* emits values to the main output only, which is a single {@link
diff --git
a/runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/FutureCollectorImpl.java
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 3bd41de11a8 Updating config from bot
3bd41de11a8 is described below
commit 3bd41de11a8a9f740051087a2b0375bb861f9aac
Author: github-actions
AuthorDate: Fri Apr 14 18:24:23 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26284.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26284.json
b/scripts/ci/pr-bot/state/pr-state/pr-26284.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26284.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a9b8d0eb69b Updating config from bot
a9b8d0eb69b is described below
commit a9b8d0eb69b3b162c9e1c05b21eda5e48f2b93e1
Author: github-actions
AuthorDate: Fri Apr 14 18:23:08 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26283.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26283.json
b/scripts/ci/pr-bot/state/pr-state/pr-26283.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26283.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9ff58f12c69 Updating config from bot
9ff58f12c69 is described below
commit 9ff58f12c69ede08fa6c73a7e659a5b181206d6e
Author: github-actions
AuthorDate: Fri Apr 14 17:56:24 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25823.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25823.json
b/scripts/ci/pr-bot/state/pr-state/pr-25823.json
index 6272b4e797d..ee5d1b9f8a7 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-25823.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25823.json
@@ -3,7 +3,7 @@
"reviewersAssignedForLabels": {
"python": "AnandInguva"
},
- "nextAction": "Reviewers",
+ "nextAction": "Author",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] 01/01: Replace deprecated codecov package with codecov uploader
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/codecov
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 1366324ce2ec0a67575c667f3f475a4d466fa6a6
Author: Danny McCormick
AuthorDate: Fri Apr 14 13:08:20 2023 -0400
Replace deprecated codecov package with codecov uploader
---
sdks/python/tox.ini | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/sdks/python/tox.ini b/sdks/python/tox.ini
index b45ac9e3c1e..f01d8ba64c2 100644
--- a/sdks/python/tox.ini
+++ b/sdks/python/tox.ini
@@ -98,14 +98,17 @@ commands =
bash {toxinidir}/scripts/run_pytest.sh {envname} "{posargs}"
[testenv:py38-cloudcoverage]
deps =
- codecov
pytest-cov==3.0.0
passenv = GIT_* BUILD_* ghprb* CHANGE_ID BRANCH_NAME JENKINS_* CODECOV_*
extras = test,gcp,interactive,dataframe,aws
commands =
-rm .coverage
bash {toxinidir}/scripts/run_pytest.sh {envname} "{posargs}"
"--cov-report=xml --cov=. --cov-append"
- codecov -F python
+
+ curl -Os https://uploader.codecov.io/v0.1.0_4653/linux/codecov
+ chmod +x codecov
+ ./codecov -F python
+ rm ./codecov
[testenv:py37-lint]
# Don't set TMPDIR to avoid "AF_UNIX path too long" errors in pylint.
[beam] branch users/damccorm/codecov created (now 1366324ce2e)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/codecov in repository https://gitbox.apache.org/repos/asf/beam.git at 1366324ce2e Replace deprecated codecov package with codecov uploader This branch includes the following new commits: new 1366324ce2e Replace deprecated codecov package with codecov uploader The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] branch master updated: Autosharding support for Java is now fixed on Dataflow. Un-sickbaying GroupIntoBatches tests, fixes #25675. (#26207)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new f1fba08b335 Autosharding support for Java is now fixed on Dataflow.
Un-sickbaying GroupIntoBatches tests, fixes #25675. (#26207)
f1fba08b335 is described below
commit f1fba08b335c9aa9726728b2d52dd144b78f7153
Author: Daniel Arn
AuthorDate: Fri Apr 14 13:04:05 2023 -0400
Autosharding support for Java is now fixed on Dataflow. Un-sickbaying
GroupIntoBatches tests, fixes #25675. (#26207)
---
runners/google-cloud-dataflow-java/build.gradle | 3 ---
1 file changed, 3 deletions(-)
diff --git a/runners/google-cloud-dataflow-java/build.gradle
b/runners/google-cloud-dataflow-java/build.gradle
index d3dcb75e246..517249de76f 100644
--- a/runners/google-cloud-dataflow-java/build.gradle
+++ b/runners/google-cloud-dataflow-java/build.gradle
@@ -400,9 +400,6 @@ task validatesRunnerStreaming {
excludedTests: [
// TODO(https://github.com/apache/beam/issues/21472)
'org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState',
- // GroupIntoBatches.withShardedKey not supported on streaming runner v1
- // https://github.com/apache/beam/issues/22592
-
'org.apache.beam.sdk.transforms.GroupIntoBatchesTest.testWithShardedKeyInGlobalWindow',
]
))
}
[beam] branch release-2.47.0 updated: [Cherry Pick #26243] Disable debezium xlang against samza (#26247)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch release-2.47.0
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/release-2.47.0 by this push:
new cb6b34620b6 [Cherry Pick #26243] Disable debezium xlang against samza
(#26247)
cb6b34620b6 is described below
commit cb6b34620b62fbe38d0ee71afe2eb36546d5a90d
Author: Robert Burke
AuthorDate: Fri Apr 14 09:41:47 2023 -0700
[Cherry Pick #26243] Disable debezium xlang against samza (#26247)
Co-authored-by: lostluck <13907733+lostl...@users.noreply.github.com>
---
sdks/go/test/integration/integration.go | 2 ++
1 file changed, 2 insertions(+)
diff --git a/sdks/go/test/integration/integration.go
b/sdks/go/test/integration/integration.go
index c13d8b16692..b71bbb3fe88 100644
--- a/sdks/go/test/integration/integration.go
+++ b/sdks/go/test/integration/integration.go
@@ -187,6 +187,8 @@ var samzaFilters = []string{
"TestMapStateClear",
"TestSetState",
"TestSetStateClear",
+ // TODO(https://github.com/apache/beam/issues/26126): Java runner issue
(AcitveBundle has no regsitered handler)
+ "TestDebeziumIO_BasicRead",
}
var sparkFilters = []string{
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new ab704f6fec6 Updating config from bot ab704f6fec6 is described below commit ab704f6fec6888f845fada43d9d261bc34fdca13 Author: github-actions AuthorDate: Fri Apr 14 16:37:15 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-25843.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25843.json b/scripts/ci/pr-bot/state/pr-state/pr-25843.json index d26f23226c7..79cd2be6f38 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-25843.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-25843.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 359ffa2d4b2 Updating config from bot 359ffa2d4b2 is described below commit 359ffa2d4b2aae43e4e0e61ccc38e61cdaa8edb4 Author: github-actions AuthorDate: Fri Apr 14 16:06:41 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 8f1e2850731..57c60ed44fb 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -8,7 +8,7 @@ "tvalentyn": 1681405537629, "pabloem": 1681281324703, "y1chi": 1667002607045, -"damccorm": 1681252473662, +"damccorm": 1681488398413, "jrmccluskey": 1681425285966 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 7ed6cfba082 Updating config from bot
7ed6cfba082 is described below
commit 7ed6cfba08271f3f404cbb027b8c4ee7d718a546
Author: github-actions
AuthorDate: Fri Apr 14 16:06:40 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26090.json | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26090.json
b/scripts/ci/pr-bot/state/pr-state/pr-26090.json
index 6272b4e797d..24d0e10a64c 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-26090.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26090.json
@@ -1,10 +1,10 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"python": "AnandInguva"
+"python": "damccorm"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
- "committerAssigned": false
+ "committerAssigned": true
}
\ No newline at end of file
[beam] branch master updated (fb545ea6e1d -> 4a20ea417f0)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from fb545ea6e1d Add outputs to notebook (#26269) add 4a20ea417f0 Allow notebooks to be picked up by tooling (#26281) No new revisions were added by this update. Summary of changes: .../notebooks/beam-ml/run_custom_inference.ipynb | 68 +-- examples/notebooks/beam-ml/tfma_beam.ipynb | 200 ++--- 2 files changed, 226 insertions(+), 42 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5cfe8fbf3e0 Updating config from bot
5cfe8fbf3e0 is described below
commit 5cfe8fbf3e0e0c1ebc6cf8dc586bad542d9c2917
Author: github-actions
AuthorDate: Fri Apr 14 16:02:08 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26281.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26281.json
b/scripts/ci/pr-bot/state/pr-state/pr-26281.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26281.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch users/damccorm/outputs updated (f94e08a56c0 -> 62c53f74ca7)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/outputs in repository https://gitbox.apache.org/repos/asf/beam.git from f94e08a56c0 commas add 62c53f74ca7 Add dummy outputs to tfma notebook No new revisions were added by this update. Summary of changes: examples/notebooks/beam-ml/tfma_beam.ipynb | 200 + 1 file changed, 175 insertions(+), 25 deletions(-)
[beam] branch users/damccorm/outputs updated (f3263bdf61d -> f94e08a56c0)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/outputs in repository https://gitbox.apache.org/repos/asf/beam.git from f3263bdf61d Add outputs to custom inference notebook add f94e08a56c0 commas No new revisions were added by this update. Summary of changes: .../notebooks/beam-ml/run_custom_inference.ipynb | 34 +++--- 1 file changed, 17 insertions(+), 17 deletions(-)
[beam] 01/01: Add outputs to custom inference notebook
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/outputs
in repository https://gitbox.apache.org/repos/asf/beam.git
commit f3263bdf61d861ca7409f1b30a962ed9ad1e4b78
Author: Danny McCormick
AuthorDate: Fri Apr 14 11:51:03 2023 -0400
Add outputs to custom inference notebook
---
.../notebooks/beam-ml/run_custom_inference.ipynb | 68 --
1 file changed, 51 insertions(+), 17 deletions(-)
diff --git a/examples/notebooks/beam-ml/run_custom_inference.ipynb
b/examples/notebooks/beam-ml/run_custom_inference.ipynb
index 33402dd324f..316f459d314 100644
--- a/examples/notebooks/beam-ml/run_custom_inference.ipynb
+++ b/examples/notebooks/beam-ml/run_custom_inference.ipynb
@@ -96,7 +96,9 @@
},
{
"cell_type": "markdown",
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4cc",
+ },
"source": [
"## Learn about `spaCy`\n",
"\n",
@@ -108,7 +110,9 @@
{
"cell_type": "code",
"execution_count": 3,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4cd",
+ },
"outputs": [],
"source": [
"import spacy\n",
@@ -119,7 +123,9 @@
{
"cell_type": "code",
"execution_count": 4,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4ce",
+ },
"outputs": [],
"source": [
"# Add text strings.\n",
@@ -132,7 +138,9 @@
{
"cell_type": "code",
"execution_count": 5,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4cf",
+ },
"outputs": [],
"source": [
"# Check which entities spaCy can recognize.\n",
@@ -142,7 +150,9 @@
{
"cell_type": "code",
"execution_count": 6,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4d0",
+ },
"outputs": [
{
"name": "stdout",
@@ -163,7 +173,9 @@
{
"cell_type": "code",
"execution_count": 7,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4d1",
+ },
"outputs": [
{
"data": {
@@ -207,7 +219,9 @@
{
"cell_type": "code",
"execution_count": 8,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e0",
+ },
"outputs": [
{
"data": {
@@ -254,7 +268,9 @@
},
{
"cell_type": "markdown",
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e1",
+ },
"source": [
"## Create a model handler\n",
"\n",
@@ -264,7 +280,9 @@
{
"cell_type": "code",
"execution_count": 9,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e2",
+ },
"outputs": [
{
"data": {
@@ -304,7 +322,9 @@
{
"cell_type": "code",
"execution_count": 10,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e3",
+ },
"outputs": [],
"source": [
"# Define `SpacyModelHandler` to load the model and perform the
inference.\n",
@@ -369,7 +389,9 @@
{
"cell_type": "code",
"execution_count": 11,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e4",
+ },
"outputs": [
{
"name": "stdout",
@@ -394,7 +416,9 @@
},
{
"cell_type": "markdown",
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e5",
+ },
"source": [
"## Use `KeyedModelHandler` to handle keyed data\n",
"\n",
@@ -404,7 +428,9 @@
{
"cell_type": "code",
"execution_count": 12,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e6",
+ },
"outputs": [],
"source": [
"# You can use these text strings with keys to distinguish
examples.\n",
@@ -417,7 +443,9 @@
{
"cell_type": "code",
"execution_count": 13,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e7",
+ },
"outputs": [],
"source": [
"from apache_beam.runners.interactive.interactive_runner import
InteractiveRunner\n",
@@ -441,7 +469,9 @@
{
"cell_type": "code",
"execution_count": 14,
- "metadata": {},
+ "metadata": {
+"id": "7f841596-f217-46d2-b64e-1952db4de4e8",
+ },
"outputs": [
{
"data": {
@@ -480,7 +510,9 @@
{
"cell_type": "code",
"execution_count": 15,
- "metadata": {},
+ "metadata": {
+"id":
[beam] branch users/damccorm/outputs created (now f3263bdf61d)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/outputs in repository https://gitbox.apache.org/repos/asf/beam.git at f3263bdf61d Add outputs to custom inference notebook This branch includes the following new commits: new f3263bdf61d Add outputs to custom inference notebook The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] branch master updated (241e40f2e9e -> fb545ea6e1d)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 241e40f2e9e Merge pull request #25930: Optimize counters by reducing allocations add fb545ea6e1d Add outputs to notebook (#26269) No new revisions were added by this update. Summary of changes: .../beam-ml/automatic_model_refresh.ipynb | 1078 +++- 1 file changed, 600 insertions(+), 478 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 0ea67353aaf Updating config from bot
0ea67353aaf is described below
commit 0ea67353aaf72ca88c52baa58ea3f99b4a81ca18
Author: github-actions
AuthorDate: Fri Apr 14 13:45:51 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-26279.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-26279.json
b/scripts/ci/pr-bot/state/pr-state/pr-26279.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-26279.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam-starter-java] branch main updated (4dc8d7a -> 77c8e6a)
This is an automated email from the ASF dual-hosted git repository. austin pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-java.git from 4dc8d7a Bump actions/checkout from 3.3.0 to 3.5.2 (#59) add 77c8e6a Get tests passing, and recent version (#65) No new revisions were added by this update. Summary of changes: build.gradle | 6 +++--- build.sbt| 2 +- pom.xml | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-)
