date:20240213

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new e3de133ab91 Updating config from bot
e3de133ab91 is described below

commit e3de133ab91b0660d0622998afc656c5b94feefd
Author: github-actions 
AuthorDate: Wed Feb 14 05:06:30 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 275de8b85b0..862cfdbef3e 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
 {
   "label": "go",
   "dateOfLastReviewAssignment": {
-"lostluck": 1707282345430,
+"lostluck": 1707887188122,
 "jrmccluskey": 1707282348860,
 "youngoli": 1657688896155,
 "damccorm": 1680501930289,

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new b43c0b8315c Updating config from bot
b43c0b8315c is described below

commit b43c0b8315cce4210581513e0e7ea4a75bbb8f27
Author: github-actions 
AuthorDate: Wed Feb 14 05:06:29 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30308.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30308.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30308.json
new file mode 100644
index 000..8ea94810368
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30308.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"go": "lostluck"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 54712428296 Updating config from bot
54712428296 is described below

commit 54712428296e68abe344b2d23ce773e2c0fdbc01
Author: github-actions 
AuthorDate: Wed Feb 14 05:06:26 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30309.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30309.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30309.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30309.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 765b2e099d7 Updating config from bot
765b2e099d7 is described below

commit 765b2e099d7abdf237f7e64989730c824e91518f
Author: github-actions 
AuthorDate: Wed Feb 14 05:06:21 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30311.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30311.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30311.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30311.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new abe20da3a30 Updating config from bot
abe20da3a30 is described below

commit abe20da3a303fc4d9c1548b3ce29b059e45ff9e5
Author: github-actions 
AuthorDate: Wed Feb 14 05:06:24 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30310.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30310.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30310.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30310.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.49.0 created (now 4ec98d49624)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.49.0
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 4ec98d49624 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks

No new revisions were added by this update.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1 deleted (was 6e43de65d99)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1
in repository https://gitbox.apache.org/repos/asf/beam.git


 was 6e43de65d99 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15 deleted (was 8109e85355d)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15
in repository https://gitbox.apache.org/repos/asf/beam.git


 was 8109e85355d Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in 
/sdks

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.16.0 created (now 03a76a0bd8d)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.16.0
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 03a76a0bd8d Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in 
/sdks

No new revisions were added by this update.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.26.6 deleted (was 875dffc7ab4)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.26.6
in repository https://gitbox.apache.org/repos/asf/beam.git


 was 875dffc7ab4 Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 
1.26.6 in /sdks

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.27.0 created (now 0b132cb55cf)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.27.0
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 0b132cb55cf Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 
1.27.0 in /sdks

No new revisions were added by this update.

(beam) branch dependabot/go_modules/sdks/github.com/docker/docker-25.0.3incompatible created (now 8842262fca6)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/docker/docker-25.0.3incompatible
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 8842262fca6 Bump github.com/docker/docker in /sdks

No new revisions were added by this update.

(beam) branch nightly-refs/heads/master updated (fa3249206d0 -> 9a4e1144532)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git


from fa3249206d0 [JdbcIO] - Adding option for max batch buffering duration 
(#30259)
 add 4410e68 [BEAM-12550] Implement Parallelizable Skew and Kurtosis 
(Skew implementation)
 add 7e29ef21aba merge with master
 add 60bd66736a1 Merge remote-tracking branch 'origin/master'
 add a2265c90863 Merge remote-tracking branch 'origin/master'
 add 50e22a51871 Merge remote-tracking branch 'origin/master'
 add d5da6cff1f4 Merge remote-tracking branch 'origin/master'
 add ef0a3468778 Merge remote-tracking branch 'origin/master'
 add 702138bf6b8 Remove WriteToFiles is experimental. (#30291)
 add b923a67369e fix bug in getProcessingTimesByStepCopy (#30270)
 add 514b03bebcc check for cachetoken representing a retry before 
activating and completing work (#29082)
 add 0ade62cd083 Only populate DataflowHistogramValue::OutlierStats when 
overflow/underflow values exist. (#30246)
 add 3b09283e45d Bump github.com/docker/go-connections from 0.4.0 to 0.5.0 
in /sdks (#30247)
 add ea68a4590d0 Register URN for SplittableParDo centrally, to avoid 
conflicts
 add 95d6ec049ae Remove avro dependency from runners/core-construction-java
 add b5cfd9523cd Merge pull request #30235: Remove avro dependency from 
runners-core-construction
 add baa673e0b8a Merge remote-tracking branch 'origin/master'
 add 46256e45450 update package
 add 23dc4afae44 Merge pull request #30298 from 
svetakvsundhar/healthcare_java_package
 add 40ed8b31e6b Delete update-reviewers (#30293)
 add cce70cdd95c Remove python 3.7 from programming guide (#30302)
 add 9452a29f489 fix: temporarily double idle timeout to workaround a bug 
in watchdog (#30299)
 add 491174ee2df Makr kafka dependency for 
runner-google-cloud-dataflow-java (#30300)
 add c004cc7fa14 Remove CoGBK in MLTransform's TFTProcessHandler (#30146)
 add c6f72c3be46 Fixes a bug related to upgrading Kafka read transforms 
using the Transform Service
 add 74a7bab41e3 Merge pull request #30264: Fixes a bug related to 
upgrading Kafka read transforms using the Transform Service
 add 9a4e1144532 Add instructions for Go custom container builds, link from 
2.55.0 release notes. (#30290)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/pr-bot-update-reviewers.yml  |  48 -
 CHANGES.md |   3 +
 .../org/apache/beam/gradle/BeamModulePlugin.groovy |   2 +-
 runners/core-construction-java/build.gradle|   2 -
 .../core/construction/CoderTranslation.java|  57 +++---
 .../beam/runners/core/construction/External.java   |   2 +-
 .../core/construction/PTransformTranslation.java   |   2 +
 .../core/construction/CoderTranslationTest.java|  10 +-
 .../core/construction/PipelineTranslationTest.java |   2 +-
 .../core/SplittableParDoViaKeyedWorkItems.java |  29 +++
 runners/direct-java/build.gradle   |   3 -
 .../runners/direct/TransformEvaluatorRegistry.java |   4 -
 .../flink/FlinkStreamingTransformTranslators.java  |   4 -
 runners/google-cloud-dataflow-java/build.gradle|   3 +-
 .../beam/runners/dataflow/DataflowRunner.java  |   8 +-
 .../dataflow/worker/DataflowExecutionContext.java  |  42 ++--
 .../MetricsToPerStepNamespaceMetricsConverter.java |  73 ---
 .../dataflow/worker/StreamingDataflowWorker.java   |  40 ++--
 .../dataflow/worker/streaming/ActiveWorkState.java | 161 +++
 .../worker/streaming/ComputationState.java |  14 +-
 .../runners/dataflow/worker/streaming/Work.java|  10 +
 .../runners/dataflow/worker/streaming/WorkId.java  |  35 ++--
 ...ricsToPerStepNamespaceMetricsConverterTest.java |   9 +-
 .../worker/StreamingDataflowWorkerTest.java|  94 +++--
 .../worker/StreamingStepMetricsContainerTest.java  |  10 +-
 .../worker/streaming/ActiveWorkStateTest.java  | 216 -
 sdks/go.mod|   2 +-
 sdks/go.sum|   4 +-
 sdks/java/extensions/avro/build.gradle |   5 +-
 .../avro}/AvroGenericCoderRegistrar.java   |   4 +-
 .../avro}/AvroGenericCoderTranslator.java  |   3 +-
 .../avro/{schemas => }/package-info.java   |   2 +-
 .../extensions/avro/AvroCoderTranslationTest.java  |  59 ++
 .../io/gcp/bigtable/BigtableConfigTranslator.java  |   7 +-
 .../sdk/io/kafka/upgrade/KafkaIOTranslation.java   |   4 +-
 .../io/kafka/upgrade/KafkaIOTranslationTest.java   |  25 +++
 sdks/python/apache_beam/io/fileio.py   |   2 -
 sdks/python/apache_beam/ml/transforms/handlers.py  | 163 ++--
 .../content/en/documentation/programming-guide.md  |   2 +-
 .../en/documentation/runtime/environments.md   |  40 +++-

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 239ecb4da76 Updating config from bot
239ecb4da76 is described below

commit 239ecb4da769250b68b14225c020adeb539569d8
Author: github-actions 
AuthorDate: Wed Feb 14 02:10:47 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
index 4494ded393c..421423720b1 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
@@ -3,7 +3,7 @@
   "dateOfLastReviewAssignment": {
 "Abacn": 1707614957467,
 "AnandInguva": 1707824067215,
-"chamikaramj": 1706660356104,
+"chamikaramj": 1707876645084,
 "damccorm": 1706746787532,
 "johnjcasey": 1706756770922,
 "jrmccluskey": 1707010152976,

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new a889bf6f2ac Updating config from bot
a889bf6f2ac is described below

commit a889bf6f2ac59888fe38d147b825a523d229a3de
Author: github-actions 
AuthorDate: Wed Feb 14 02:10:46 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30284.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30284.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30284.json
new file mode 100644
index 000..26e4be5dc8a
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30284.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"no-matching-label": "chamikaramj"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch asf-site updated: Publishing website 2024/02/13 23:38:37 at commit 9a4e114

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 86ec0386c65 Publishing website 2024/02/13 23:38:37 at commit 9a4e114
86ec0386c65 is described below

commit 86ec0386c65e79afb901c0cf4347e04ff17abab8
Author: runner 
AuthorDate: Tue Feb 13 23:38:37 2024 +

Publishing website 2024/02/13 23:38:37 at commit 9a4e114
---
 website/generated-content/documentation/index.xml  | 33 --
 .../documentation/programming-guide/index.html |  2 +-
 .../documentation/runtime/environments/index.html  | 29 +--
 website/generated-content/sitemap.xml  |  2 +-
 4 files changed, 59 insertions(+), 7 deletions(-)

diff --git a/website/generated-content/documentation/index.xml 
b/website/generated-content/documentation/index.xml
index 4fd1a9c260c..8cd51f72802 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -5387,7 +5387,7 @@ programming guide, take a look at the
 TypeScript SDK
 
 
-The Python SDK supports Python 3.7, 3.8, 3.9, 3.10, 
and 3.11.
+The Python SDK supports Python 3.8, 3.9, 3.10, and 
3.11.
 The Go 
SDK supports Go v1.20+.
 The Typescript SDK supports Node v16+ and is 
still experimental.
 1. Overview
@@ -17309,7 +17309,7 @@ limitations under the License.
 Launching third-party software in the worker environment
 Further customizing the execution environment
 
-This guide describes how to create and use customized containers for the 
Beam SDK.
+This guide describes how to create and use customized containers for the 
Beam SDKs.
 Prerequisites
 
 This guide requires building images using Docker. Install Docker locally. Some 
CI/CD platforms like Google
 Cloud Build also provide the ability to build images using 
Docker.
@@ -17456,7 +17456,34 @@ docker build -f Dockerfile -t 
"${IMAGE_NAME}:${TAG}" .
 If your runner is running remotely, retag the image and push 
the image to your repository.
 
 docker push 
"${IMAGE_NAME}:${TAG}"
-
Running pipelines with custom 
container images
+

Building a compatible container image from scratch (Go)

From the 2.55.0 release, the Beam Go SDK has moved to using distroless images as a base. +These images have a reduced security attack surface by not including common tools and utilities. +This may cause difficulties customizing the image with using one of the above approaches. +As a fallback, it’s possible to build a custom image from scratch, by building a matching boot loader, and setting +that as the container’s entry point.

For example, if it’s preferable to use alpine as the container OS your multi-stage docker file might +look like the following:

FROM golang:latest-alpine AS build_base
+# Set the Current Working Directory inside the container
+WORKDIR /tmp/beam
+# Build the Beam Go bootloader, to the local directory, matching your Beam 
version.
+# Similar go targets exist for other SDK languages.
+RUN GOBIN=`pwd` go install github.com/apache/beam/sdks/v2/go/container@v2.53.0
+# Set the real base image.
+FROM alpine:3.9
+RUN apk add ca-certificates
+# The following are required for the container to operate correctly.
+# Copy the boot loader `container` to the image.
+COPY --from=build_base /tmp/beam/container /opt/apache/beam/boot
+# Set the container to use the newly built boot loader.
+ENTRYPOINT ["/opt/apache/beam/boot"]
+

Build and push the new image as when modifying an existing base image above.

+
NOTE: Java and Python require additional dependencies, such as their runtimes, and SDK packages for +a valid container image. The bootloader isn’t sufficient for creating a custom container for these SDKs.
+

Running pipelines with custom container images

The common method for providing a container image requires using the PortableRunner flag --environment_config as suppor

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 18c7e602234 Updating config from bot
18c7e602234 is described below

commit 18c7e602234fb9fc1734d7ca139883b73c23d904
Author: github-actions 
AuthorDate: Tue Feb 13 23:34:42 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 56320f2f999..9f744e7451c 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
 {
   "label": "build",
   "dateOfLastReviewAssignment": {
-"damccorm": 1707269103643,
+"damccorm": 1707867277708,
 "Abacn": 1707503950521
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 4c79ff3cf96 Updating config from bot
4c79ff3cf96 is described below

commit 4c79ff3cf9675c9a96907f76245dd986a6662e61
Author: github-actions 
AuthorDate: Tue Feb 13 23:34:43 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 22bb0094182..8ebccc3c734 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -1,7 +1,7 @@
 {
   "label": "io",
   "dateOfLastReviewAssignment": {
-"chamikaramj": 1707425392751,
+"chamikaramj": 1707867277708,
 "johnjcasey": 1707761180436,
 "pabloem": 1691787951165,
 "Abacn": 1707503950521,

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 0a0979060a2 Updating config from bot
0a0979060a2 is described below

commit 0a0979060a265976f7149e977eafd1112b73d4d8
Author: github-actions 
AuthorDate: Tue Feb 13 23:34:40 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 4aff446b813..9019198fd80 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -5,7 +5,7 @@
 "yeandy": 1665802753763,
 "TheNeuralBit": 1667896849319,
 "ryanthompson591": 1670002443548,
-"tvalentyn": 1706897164657,
+"tvalentyn": 1707867277708,
 "pabloem": 1681281324703,
 "y1chi": 1667002607045,
 "damccorm": 1707761180436,

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new a51910a9508 Updating config from bot
a51910a9508 is described below

commit a51910a9508e31132254cef17231a27737d3f667
Author: github-actions 
AuthorDate: Tue Feb 13 23:34:39 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30307.json | 12 
 1 file changed, 12 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30307.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30307.json
new file mode 100644
index 000..abecf0d2d19
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30307.json
@@ -0,0 +1,12 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"python": "tvalentyn",
+"build": "damccorm",
+"io": "chamikaramj"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new bda29ee5279 Updating config from bot
bda29ee5279 is described below

commit bda29ee5279aff023b6757a618f5979edae2866e
Author: github-actions 
AuthorDate: Tue Feb 13 23:17:17 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30253.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30253.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30253.json
index 62b89857e47..e475087ee4f 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30253.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30253.json
@@ -5,7 +5,7 @@
 "io": "ahmedabu98"
   },
   "nextAction": "Reviewers",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 99f70342a4b Updating config from bot
99f70342a4b is described below

commit 99f70342a4b320bd6a8d95f7cd72fee8f5e94e2b
Author: github-actions 
AuthorDate: Tue Feb 13 22:34:18 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30306.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30306.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30306.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30306.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch master updated: Add instructions for Go custom container builds, link from 2.55.0 release notes. (#30290)

2024-02-13 Thread lostluck

This is an automated email from the ASF dual-hosted git repository.

lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 9a4e1144532 Add instructions for Go custom container builds, link from 
2.55.0 release notes. (#30290)
9a4e1144532 is described below

commit 9a4e1144532291244cfe6fde7f99655de1e9c88d
Author: Robert Burke 
AuthorDate: Tue Feb 13 13:23:26 2024 -0800

Add instructions for Go custom container builds, link from 2.55.0 release 
notes. (#30290)

* Stopgap message in release notes.

* Recommend 2.54.0 not 2.53.0.

* Add from scratch instructions.

* remove specific go version

* ws

* redundant line

* rm more ws

-

Co-authored-by: lostluck <13907733+lostl...@users.noreply.github.com>
---
 CHANGES.md |  3 ++
 .../en/documentation/runtime/environments.md   | 40 +-
 2 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/CHANGES.md b/CHANGES.md
index bda5937a93d..607f1a1185f 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -72,6 +72,9 @@
 
 * X behavior was changed ([#X](https://github.com/apache/beam/issues/X)).
 * Arrow version was bumped to 15.0.0 from 5.0.0 
([#30181](https://github.com/apache/beam/pull/30181)).
+* Go SDK users who build custom worker containers may run into issues with the 
move to distroless containers as a base (see Security Fixes).
+  * The issue stems from distroless containers lacking additional tools, which 
current custom container processes may rely on.
+  * See 
https://beam.apache.org/documentation/runtime/environments/#from-scratch-go for 
instructions on building and using a custom container.
 
 ## Deprecations
 
diff --git a/website/www/site/content/en/documentation/runtime/environments.md 
b/website/www/site/content/en/documentation/runtime/environments.md
index c860816e300..d9a42db29e2 100644
--- a/website/www/site/content/en/documentation/runtime/environments.md
+++ b/website/www/site/content/en/documentation/runtime/environments.md
@@ -29,7 +29,7 @@ You may want to customize container images for many reasons, 
including:
 * Launching third-party software in the worker environment
 * Further customizing the execution environment
 
- This guide describes how to create and use customized containers for the Beam 
SDK.
+ This guide describes how to create and use customized containers for the Beam 
SDKs.
 
 ### Prerequisites
 
@@ -218,6 +218,44 @@ Beam offers a way to provide your own custom container 
image. The easiest way to
   docker push "${IMAGE_NAME}:${TAG}"
   ```
 
+ Building a compatible container image from scratch (Go) {#from-scratch-go}
+
+From the 2.55.0 release, the Beam Go SDK has moved to using [distroless 
images](https://github.com/GoogleContainerTools/distroless) as a base.
+These images have a reduced security attack surface by not including common 
tools and utilities.
+This may cause difficulties customizing the image with using one of the above 
approaches.
+As a fallback, it's possible to build a custom image from scratch, by building 
a matching boot loader, and setting
+that as the container's entry point.
+
+For example, if it's preferable to use alpine as the container OS your 
multi-stage docker file might
+look like the following:
+
+```
+FROM golang:latest-alpine AS build_base
+
+# Set the Current Working Directory inside the container
+WORKDIR /tmp/beam
+
+# Build the Beam Go bootloader, to the local directory, matching your Beam 
version.
+# Similar go targets exist for other SDK languages.
+RUN GOBIN=`pwd` go install github.com/apache/beam/sdks/v2/go/container@v2.53.0
+
+# Set the real base image.
+FROM alpine:3.9
+RUN apk add ca-certificates
+
+# The following are required for the container to operate correctly.
+# Copy the boot loader `container` to the image.
+COPY --from=build_base /tmp/beam/container /opt/apache/beam/boot
+
+# Set the container to use the newly built boot loader.
+ENTRYPOINT ["/opt/apache/beam/boot"]
+```
+
+Build and push the new image as when [modifying an existing base 
image](#modify-existing-base-image) above.
+
+>**NOTE**: Java and Python require additional dependencies, such as their 
runtimes, and SDK packages for
+> a valid container image. The bootloader isn't sufficient for creating a 
custom container for these SDKs.
+
 ## Running pipelines with custom container images {#running-pipelines}
 
 The common method for providing a container image requires using the

(beam) branch master updated (c004cc7fa14 -> 74a7bab41e3)

2024-02-13 Thread chamikara

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from c004cc7fa14 Remove CoGBK in MLTransform's TFTProcessHandler (#30146)
 add c6f72c3be46 Fixes a bug related to upgrading Kafka read transforms 
using the Transform Service
 new 74a7bab41e3 Merge pull request #30264: Fixes a bug related to 
upgrading Kafka read transforms using the Transform Service

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../sdk/io/kafka/upgrade/KafkaIOTranslation.java   |  4 ++--
 .../io/kafka/upgrade/KafkaIOTranslationTest.java   | 25 ++
 2 files changed, 27 insertions(+), 2 deletions(-)

(beam) 01/01: Merge pull request #30264: Fixes a bug related to upgrading Kafka read transforms using the Transform Service

2024-02-13 Thread chamikara

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 74a7bab41e3b2764b1006b114f10500a2e000f1d
Merge: c004cc7fa14 c6f72c3be46
Author: Chamikara Jayalath 
AuthorDate: Tue Feb 13 12:37:52 2024 -0800

Merge pull request #30264: Fixes a bug related to upgrading Kafka read 
transforms using the Transform Service

 .../sdk/io/kafka/upgrade/KafkaIOTranslation.java   |  4 ++--
 .../io/kafka/upgrade/KafkaIOTranslationTest.java   | 25 ++
 2 files changed, 27 insertions(+), 2 deletions(-)

(beam) branch remove_cogbk deleted (was e0fc00b8a6d)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a change to branch remove_cogbk
in repository https://gitbox.apache.org/repos/asf/beam.git


 was e0fc00b8a6d Make _DataCoder internal

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch master updated: Remove CoGBK in MLTransform's TFTProcessHandler (#30146)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new c004cc7fa14 Remove CoGBK in MLTransform's TFTProcessHandler (#30146)
c004cc7fa14 is described below

commit c004cc7fa1425d14ef2b0c134784f80a42e555a7
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Tue Feb 13 19:19:34 2024 +

Remove CoGBK in MLTransform's TFTProcessHandler (#30146)

* Add _Encode and _DecodeDict

* Replace the CoGBK and utils with Encode and Decode utils

* Use PicklerCoder for encoding and decoding elements

* Remove comments

* update coder

* Address comments

* Remove return comment

* Update sdks/python/apache_beam/ml/transforms/handlers.py

Co-authored-by: tvalentyn 

* Make _DataCoder internal

-

Co-authored-by: tvalentyn 
---
 sdks/python/apache_beam/ml/transforms/handlers.py | 163 --
 1 file changed, 57 insertions(+), 106 deletions(-)

diff --git a/sdks/python/apache_beam/ml/transforms/handlers.py 
b/sdks/python/apache_beam/ml/transforms/handlers.py
index 3c37ddef1ed..5bcd0d16576 100644
--- a/sdks/python/apache_beam/ml/transforms/handlers.py
+++ b/sdks/python/apache_beam/ml/transforms/handlers.py
@@ -17,9 +17,10 @@
 # pytype: skip-file
 
 import collections
+import copy
 import os
 import typing
-import uuid
+from typing import Any
 from typing import Dict
 from typing import List
 from typing import Optional
@@ -31,6 +32,7 @@ import numpy as np
 import apache_beam as beam
 import tensorflow as tf
 import tensorflow_transform.beam as tft_beam
+from apache_beam import coders
 from apache_beam.io.filesystems import FileSystems
 from apache_beam.ml.transforms.base import ArtifactMode
 from apache_beam.ml.transforms.base import ProcessHandler
@@ -50,7 +52,7 @@ __all__ = [
 'TFTProcessHandler',
 ]
 
-_ID_COLUMN = 'tmp_uuid'  # Name for a temporary column.
+_TEMP_KEY = 'CODED_SAMPLE'  # key for the encoded sample
 
 RAW_DATA_METADATA_DIR = 'raw_data_metadata'
 SCHEMA_FILE = 'schema.pbtxt'
@@ -83,12 +85,41 @@ tft_process_handler_input_type = 
typing.Union[typing.NamedTuple,
 tft_process_handler_output_type = typing.Union[beam.Row, Dict[str, np.ndarray]]
 
 
+class _DataCoder:
+  def __init__(
+  self,
+  exclude_columns,
+  coder=coders.registry.get_coder(Any),
+  ):
+"""
+Encodes/decodes items of a dictionary into a single element.
+Args:
+  exclude_columns: list of columns to exclude from the encoding.
+"""
+self.coder = coder
+self.exclude_columns = exclude_columns
+
+  def encode(self, element):
+data_to_encode = element.copy()
+element_to_return = element.copy()
+for key in self.exclude_columns:
+  if key in data_to_encode:
+del data_to_encode[key]
+element_to_return[_TEMP_KEY] = self.coder.encode(data_to_encode)
+return element_to_return
+
+  def decode(self, element):
+clone = copy.copy(element)
+clone.update(self.coder.decode(clone[_TEMP_KEY].item()))
+del clone[_TEMP_KEY]
+return clone
+
+
 class _ConvertScalarValuesToListValues(beam.DoFn):
   def process(
   self,
   element,
   ):
-id, element = element
 new_dict = {}
 for key, value in element.items():
   if isinstance(value,
@@ -96,7 +127,7 @@ class _ConvertScalarValuesToListValues(beam.DoFn):
 new_dict[key] = [value]
   else:
 new_dict[key] = value
-yield (id, new_dict)
+yield new_dict
 
 
 class _ConvertNamedTupleToDict(
@@ -124,79 +155,6 @@ class _ConvertNamedTupleToDict(
   return pcoll | beam.Map(lambda x: x._asdict())
 
 
-class _ComputeAndAttachUniqueID(beam.DoFn):
-  """
-  Computes and attaches a unique id to each element in the PCollection.
-  """
-  def process(self, element):
-# UUID1 includes machine-specific bits and has a counter. As long as not 
too
-# many are generated at the same time, they should be unique.
-# UUID4 generation should be unique in practice as long as underlying 
random
-# number generation is not compromised.
-# A combintation of both should avoid the anecdotal pitfalls where
-# replacing one with the other has helped some users.
-# UUID collision will result in data loss, but we can detect that and fail.
-
-# TODO(https://github.com/apache/beam/issues/29593): Evaluate MLTransform
-# implementation without CoGBK.
-unique_key = uuid.uuid1().bytes + uuid.uuid4().bytes
-yield (unique_key, element)
-
-
-class _GetMissingColumns(beam.DoFn):
-  """
-  Returns data containing only the columns that are not
-  present in the schema. This is needed since TFT only outputs
-  columns that are transformed by any of the data processing transforms.
-  """
-  def __init__(self, existing_columns):
-self

(beam) branch master updated (9452a29f489 -> 491174ee2df)

2024-02-13 Thread yhu

This is an automated email from the ASF dual-hosted git repository.

yhu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 9452a29f489 fix: temporarily double idle timeout to workaround a bug 
in watchdog (#30299)
 add 491174ee2df Makr kafka dependency for 
runner-google-cloud-dataflow-java (#30300)

No new revisions were added by this update.

Summary of changes:
 runners/google-cloud-dataflow-java/build.gradle   | 3 ++-
 .../java/org/apache/beam/runners/dataflow/DataflowRunner.java | 8 +++-
 2 files changed, 9 insertions(+), 2 deletions(-)

(beam) branch master updated (cce70cdd95c -> 9452a29f489)

2024-02-13 Thread ahmedabualsaud

This is an automated email from the ASF dual-hosted git repository.

ahmedabualsaud pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from cce70cdd95c Remove python 3.7 from programming guide (#30302)
 add 9452a29f489 fix: temporarily double idle timeout to workaround a bug 
in watchdog (#30299)

No new revisions were added by this update.

Summary of changes:
 .../apache/beam/sdk/io/gcp/bigtable/BigtableConfigTranslator.java  | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new c91a200f837 Updating config from bot
c91a200f837 is described below

commit c91a200f837ed997bc81307d1be814d2e9260856
Author: github-actions 
AuthorDate: Tue Feb 13 18:06:50 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30299.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30299.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30299.json
index aa4bdd58ba7..7d1f5a2a8cd 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30299.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30299.json
@@ -8,5 +8,5 @@
   "nextAction": "Reviewers",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
-  "committerAssigned": false
+  "committerAssigned": true
 }
\ No newline at end of file

(beam) branch master updated: Remove python 3.7 from programming guide (#30302)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new cce70cdd95c Remove python 3.7 from programming guide (#30302)
cce70cdd95c is described below

commit cce70cdd95c6db0c49a833d14becb1c9aa474027
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Tue Feb 13 17:54:01 2024 +

Remove python 3.7 from programming guide (#30302)
---
 website/www/site/content/en/documentation/programming-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/www/site/content/en/documentation/programming-guide.md 
b/website/www/site/content/en/documentation/programming-guide.md
index dc1f9f6..7c2c8ddacff 100644
--- a/website/www/site/content/en/documentation/programming-guide.md
+++ b/website/www/site/content/en/documentation/programming-guide.md
@@ -35,7 +35,7 @@ programming guide, take a look at the
 {{< language-switcher java py go typescript >}}
 
 {{< paragraph class="language-py" >}}
-The Python SDK supports Python 3.7, 3.8, 3.9, 3.10, and 3.11.
+The Python SDK supports Python 3.8, 3.9, 3.10, and 3.11.
 {{< /paragraph >}}
 
 {{< paragraph class="language-go">}}

(beam) branch damccorm-patch-1 deleted (was d5a32fe1353)

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch damccorm-patch-1
in repository https://gitbox.apache.org/repos/asf/beam.git


 was d5a32fe1353 Delete update-reviewers

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch master updated: Delete update-reviewers (#30293)

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 40ed8b31e6b Delete update-reviewers (#30293)
40ed8b31e6b is described below

commit 40ed8b31e6b4426b1d9cff9b0eb2de3e47f4928d
Author: Danny McCormick 
AuthorDate: Tue Feb 13 12:47:43 2024 -0500

Delete update-reviewers (#30293)
---
 .github/workflows/pr-bot-update-reviewers.yml | 48 ---
 1 file changed, 48 deletions(-)

diff --git a/.github/workflows/pr-bot-update-reviewers.yml 
b/.github/workflows/pr-bot-update-reviewers.yml
deleted file mode 100644
index ac80d1e346e..000
--- a/.github/workflows/pr-bot-update-reviewers.yml
+++ /dev/null
@@ -1,48 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: pr-bot-update-reviewers
-
-# Run every week on Sunday at 16:05 UTC
-on:
-  schedule:
-  - cron: '5 16 * * 0'
-  workflow_dispatch:
-
-jobs:
-  update-reviewers:
-permissions:
-  contents: write
-  pull-requests: write
-  checks: read
-  issues: read
-  statuses: read
-# Don't run on forks
-if: github.repository == 'apache/beam'
-runs-on: [self-hosted, ubuntu-20.04]
-steps:
-  - uses: actions/checkout@v4
-  - name: Setup Node
-uses: actions/setup-node@v4
-with:
-  node-version: 16
-  - name: Install pr-bot npm dependencies
-run: npm ci
-working-directory: 'scripts/ci/pr-bot'
-  # Runs a set of commands using the runners shell
-  - run: npm run updateReviewers
-env:
-  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-working-directory: 'scripts/ci/pr-bot'

(beam) branch asf-site updated: Publishing website 2024/02/13 17:37:44 at commit 23dc4af

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 2e078e7cefb Publishing website 2024/02/13 17:37:44 at commit 23dc4af
2e078e7cefb is described below

commit 2e078e7cefb9f3e1e62e023a1c43bb57c2424941
Author: runner 
AuthorDate: Tue Feb 13 17:37:44 2024 +

Publishing website 2024/02/13 17:37:44 at commit 23dc4af
---
 website/generated-content/sitemap.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/generated-content/sitemap.xml 
b/website/generated-content/sitemap.xml
index f679a41b90a..d90be192562 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/apache-beam-flink-and-kubernetes-part3/2024-02-13T01:22:11-08:00/categories/blog/2024-02-13T01:22:11-08:00/blog/2024-02-13T01:22:11-08:00/categories/2024-02-13T01:22:11-08:00http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/apache-beam-flink-and-kubernetes-part3/2024-02-13T16:50:53+00:00/categories/blog/2024-02-13T16:50:53+00:00/blog/2024-02-13T16:50:53+00:00/categories/2024-02-13T16:50:53+00:00

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 9212c5ad814 Updating config from bot
9212c5ad814 is described below

commit 9212c5ad814c3b434a8ab87343342636b9792acc
Author: github-actions 
AuthorDate: Tue Feb 13 17:34:33 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 26bded8e7c0..d13e8cf1f21 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -9,6 +9,6 @@
 "Abacn": 1707425392750,
 "bvolpato": 1707503950521,
 "m-trieu": 1707343533957,
-"damondouglas": 1707308019524
+"damondouglas": 1707845670465
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 7567936127a Updating config from bot
7567936127a is described below

commit 7567936127ab1f697a2383acb042c13c7365e884
Author: github-actions 
AuthorDate: Tue Feb 13 17:34:34 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 8f96d2543b2..22bb0094182 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -5,7 +5,7 @@
 "johnjcasey": 1707761180436,
 "pabloem": 1691787951165,
 "Abacn": 1707503950521,
-"ahmedabu98": 1707343533957,
+"ahmedabu98": 1707845670465,
 "bvolpato": 1707764084416,
 "manavgarg": 1690826779210,
 "damondouglas": 1707826420096

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 3183dc4ff2c Updating config from bot
3183dc4ff2c is described below

commit 3183dc4ff2cedef1e81392803509a5c517938943
Author: github-actions 
AuthorDate: Tue Feb 13 17:34:36 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json
index 6bee694e16a..8bbb6aac56d 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-bigtable.json
@@ -1,7 +1,7 @@
 {
   "label": "bigtable",
   "dateOfLastReviewAssignment": {
-"igorbernstein2": 1706738733803,
+"igorbernstein2": 1707845670465,
 "mutianf": 1704742532925
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 27cb875698e Updating config from bot
27cb875698e is described below

commit 27cb875698e7f972c205c2c9159c02e8a0227b6c
Author: github-actions 
AuthorDate: Tue Feb 13 17:34:31 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30299.json | 12 
 1 file changed, 12 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30299.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30299.json
new file mode 100644
index 000..aa4bdd58ba7
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30299.json
@@ -0,0 +1,12 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"java": "damondouglas",
+"io": "ahmedabu98",
+"bigtable": "igorbernstein2"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch anandinguva-patch updated: Remove python 3.7 from programming guide

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a commit to branch anandinguva-patch
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/anandinguva-patch by this push:
 new e4b071c785e Remove python 3.7 from programming guide
e4b071c785e is described below

commit e4b071c785e534a97623d8f6c9419ed8e5d44714
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Tue Feb 13 12:15:06 2024 -0500

Remove python 3.7 from programming guide
---
 website/www/site/content/en/documentation/programming-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/www/site/content/en/documentation/programming-guide.md 
b/website/www/site/content/en/documentation/programming-guide.md
index dc1f9f6..7c2c8ddacff 100644
--- a/website/www/site/content/en/documentation/programming-guide.md
+++ b/website/www/site/content/en/documentation/programming-guide.md
@@ -35,7 +35,7 @@ programming guide, take a look at the
 {{< language-switcher java py go typescript >}}
 
 {{< paragraph class="language-py" >}}
-The Python SDK supports Python 3.7, 3.8, 3.9, 3.10, and 3.11.
+The Python SDK supports Python 3.8, 3.9, 3.10, and 3.11.
 {{< /paragraph >}}
 
 {{< paragraph class="language-go">}}

(beam) branch anandinguva-patch created (now 23dc4afae44)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a change to branch anandinguva-patch
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 23dc4afae44 Merge pull request #30298 from 
svetakvsundhar/healthcare_java_package

No new revisions were added by this update.

(beam) branch master updated (b5cfd9523cd -> 23dc4afae44)

2024-02-13 Thread svetak

This is an automated email from the ASF dual-hosted git repository.

svetak pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from b5cfd9523cd Merge pull request #30235: Remove avro dependency from 
runners-core-construction
 add 4410e68 [BEAM-12550] Implement Parallelizable Skew and Kurtosis 
(Skew implementation)
 add 7e29ef21aba merge with master
 add 60bd66736a1 Merge remote-tracking branch 'origin/master'
 add a2265c90863 Merge remote-tracking branch 'origin/master'
 add 50e22a51871 Merge remote-tracking branch 'origin/master'
 add d5da6cff1f4 Merge remote-tracking branch 'origin/master'
 add ef0a3468778 Merge remote-tracking branch 'origin/master'
 add baa673e0b8a Merge remote-tracking branch 'origin/master'
 add 46256e45450 update package
 new 23dc4afae44 Merge pull request #30298 from 
svetakvsundhar/healthcare_java_package

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam) 01/01: Merge pull request #30298 from svetakvsundhar/healthcare_java_package

2024-02-13 Thread svetak

This is an automated email from the ASF dual-hosted git repository.

svetak pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 23dc4afae44f1285c3cfe13ad4a15fb73a8dc55f
Merge: b5cfd9523cd 46256e45450
Author: Svetak Sundhar 
AuthorDate: Tue Feb 13 16:50:53 2024 +

Merge pull request #30298 from svetakvsundhar/healthcare_java_package

Healthcare java package update (Jan)

 buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new af5ec671b06 Updating config from bot
af5ec671b06 is described below

commit af5ec671b06e7fca566db7b42dbb2c3d4c48ae6e
Author: github-actions 
AuthorDate: Tue Feb 13 16:47:41 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30300.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30300.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30300.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30300.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 510163de6c5 Updating config from bot
510163de6c5 is described below

commit 510163de6c500cc1b569fc23193a00707fca8783
Author: github-actions 
AuthorDate: Tue Feb 13 16:21:15 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30086.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30086.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30086.json
index 84a14da827e..88d1920a8bb 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30086.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30086.json
@@ -3,7 +3,7 @@
   "reviewersAssignedForLabels": {
 "python": "damccorm"
   },
-  "nextAction": "Author",
+  "nextAction": "Reviewers",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
   "committerAssigned": false

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 132494d9b6b Updating config from bot
132494d9b6b is described below

commit 132494d9b6b9fc5330c52bdbb6730100e046075e
Author: github-actions 
AuthorDate: Tue Feb 13 16:15:04 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30298.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30298.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30298.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30298.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch test-suite updated (bc5bba31e2b -> ecb443c4650)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a change to branch test-suite
in repository https://gitbox.apache.org/repos/asf/beam.git


omit bc5bba31e2b Include MLTransform IT test suite as a part of PostCommits
 add ecb443c4650 Include MLTransform IT test suite as a part of PostCommits

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (bc5bba31e2b)
\
 N -- N -- N   refs/heads/test-suite (ecb443c4650)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 build.gradle.kts | 8 +++-
 1 file changed, 3 insertions(+), 5 deletions(-)

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 26a825e0549 Updating config from bot
26a825e0549 is described below

commit 26a825e0549e3b1cefe2988dbc146d0d7f486c00
Author: github-actions 
AuthorDate: Tue Feb 13 15:30:47 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30086.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30086.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30086.json
index 88d1920a8bb..84a14da827e 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30086.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30086.json
@@ -3,7 +3,7 @@
   "reviewersAssignedForLabels": {
 "python": "damccorm"
   },
-  "nextAction": "Reviewers",
+  "nextAction": "Author",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
   "committerAssigned": false

(beam) branch master updated (3b09283e45d -> b5cfd9523cd)

2024-02-13 Thread kenn

This is an automated email from the ASF dual-hosted git repository.

kenn pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 3b09283e45d Bump github.com/docker/go-connections from 0.4.0 to 0.5.0 
in /sdks (#30247)
 add ea68a4590d0 Register URN for SplittableParDo centrally, to avoid 
conflicts
 add 95d6ec049ae Remove avro dependency from runners/core-construction-java
 new b5cfd9523cd Merge pull request #30235: Remove avro dependency from 
runners-core-construction

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 runners/core-construction-java/build.gradle|  2 -
 .../core/construction/CoderTranslation.java| 57 +
 .../beam/runners/core/construction/External.java   |  2 +-
 .../core/construction/PTransformTranslation.java   |  2 +
 .../core/construction/CoderTranslationTest.java| 10 ++--
 .../core/construction/PipelineTranslationTest.java |  2 +-
 .../core/SplittableParDoViaKeyedWorkItems.java | 29 +++
 runners/direct-java/build.gradle   |  3 --
 .../runners/direct/TransformEvaluatorRegistry.java |  4 --
 .../flink/FlinkStreamingTransformTranslators.java  |  4 --
 sdks/java/extensions/avro/build.gradle |  5 +-
 .../avro}/AvroGenericCoderRegistrar.java   |  4 +-
 .../avro}/AvroGenericCoderTranslator.java  |  3 +-
 .../avro/{schemas => }/package-info.java   |  2 +-
 .../extensions/avro/AvroCoderTranslationTest.java  | 59 ++
 15 files changed, 139 insertions(+), 49 deletions(-)
 rename 
{runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction
 => 
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro}/AvroGenericCoderRegistrar.java
 (90%)
 rename 
{runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction
 => 
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro}/AvroGenericCoderTranslator.java
 (94%)
 copy 
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/{schemas
 => }/package-info.java (95%)
 create mode 100644 
sdks/java/extensions/avro/src/test/java/org/apache/beam/sdk/extensions/avro/AvroCoderTranslationTest.java

(beam) 01/01: Merge pull request #30235: Remove avro dependency from runners-core-construction

2024-02-13 Thread kenn

This is an automated email from the ASF dual-hosted git repository.

kenn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit b5cfd9523cde3fee8610956682541d4e90ee967b
Merge: 3b09283e45d 95d6ec049ae
Author: Kenneth Knowles 
AuthorDate: Tue Feb 13 10:28:42 2024 -0500

Merge pull request #30235: Remove avro dependency from 
runners-core-construction

 runners/core-construction-java/build.gradle|  2 -
 .../core/construction/CoderTranslation.java| 57 +
 .../beam/runners/core/construction/External.java   |  2 +-
 .../core/construction/PTransformTranslation.java   |  2 +
 .../core/construction/CoderTranslationTest.java| 10 ++--
 .../core/construction/PipelineTranslationTest.java |  2 +-
 .../core/SplittableParDoViaKeyedWorkItems.java | 29 +++
 runners/direct-java/build.gradle   |  3 --
 .../runners/direct/TransformEvaluatorRegistry.java |  4 --
 .../flink/FlinkStreamingTransformTranslators.java  |  4 --
 sdks/java/extensions/avro/build.gradle |  5 +-
 .../avro}/AvroGenericCoderRegistrar.java   |  4 +-
 .../avro}/AvroGenericCoderTranslator.java  |  3 +-
 .../beam/sdk/extensions/avro/package-info.java | 26 ++
 .../extensions/avro/AvroCoderTranslationTest.java  | 59 ++
 15 files changed, 164 insertions(+), 48 deletions(-)

(beam) branch test-suite created (now bc5bba31e2b)

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a change to branch test-suite
in repository https://gitbox.apache.org/repos/asf/beam.git


  at bc5bba31e2b Include MLTransform IT test suite as a part of PostCommits

This branch includes the following new commits:

 new bc5bba31e2b Include MLTransform IT test suite as a part of PostCommits

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15 updated (d8f0088e9d7 -> 8109e85355d)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.15.15
in repository https://gitbox.apache.org/repos/asf/beam.git


 discard d8f0088e9d7 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in 
/sdks
 add 89d1c06e1ea Fix Fhir IO Search IT (#30158)
 add a221f98a5f4 fix: fix error handling in segment reader and add an 
integration test (#30147)
 add ddb93272ab6 Fix git trusted directory errors in website build. (#30171)
 add 30917ff7b3e Don't use '-' as a delimiter in BigQuerySinkMetrics 
(#30118)
 add fb0762352c2 Periodically send perWorkerMetrics on the WorkerMessages 
API (#30135)
 add 4f09e62f520 Improve varint encoding throughput with unrolled loop 
(#29689)
 add 5188e37ed44 Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 
in /sdks (#30175)
 add 96cf695a101 Fix JdbcIO PreCommit mysql breaking change (#30179)
 add 824e06cd967 Allow stale_bq_datasets_cleaner.sh to run to completion, 
even when there are failures to grab a dataset (#30153)
 add 1d9f6044c76 Remove constraint introduced in #30059 (#30186)
 add bdbbef9b54d [YAML] - PubSubLite proto (#30129)
 add 1a5466b7df7 Support dict as the input of delete_batch (#30188)
 add 574e7bfbc1d Fix script typos for validating docker licenses. (#30176)
 add 7fb05cbb5e1 Move the code from sdks-java-fn-execution to sdks-java-core
 add e8df60381a3 Merge pull request #30170: Move the code from 
sdks-java-fn-execution to sdks-java-core
 add 124880bc8ce Fix typo affecting 
DataflowPipelineDebugOptions.setStreamingSideInputCacheExpirationMillis (#30195)
 add 1f69271044d [Dataflow Streaming] Add a option to use multiple commit 
threads (#30194)
 add 1183b5d5274 Knowledge lookup prompts: (#29747)
 add 801682930e2 Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in 
/sdks (#29913)
 add 30a778b9cbe Bump github.com/containerd/containerd from 1.7.7 to 1.7.11 
in /sdks (#29823)
 add f03b115830b Remove extraneous mocking that was causing NPEs in 
DataflowWorkUnitClientTest
 add 00f07864784 Remove sdks/java/fn-execution
 add 8ec51f181a0 Merge pull request #30193: Remove sdks/java/fn-execution
 add 9f8dda274dc Split standalone transform service application from 
launcher jar (#30190)
 add f2056e72dea Fix stale dataflow prebuilt image cleaner (#30168)
 add 34bbbda9f98 Try a different approach for cibuildwheel flake. (#30204)
 add 1a1ff756f86 Bump golang.org/x/sync from 0.5.0 to 0.6.0 in /sdks 
(#30209)
 add c1b3a27ee0c Bump github.com/nats-io/nats-server/v2 from 2.10.7 to 
2.10.10 in /sdks (#30210)
 add a47b1faa527 [Prism] Implement jobservices.Server Cancel (#30178)
 add 7f91baadcf2 Upgrade website node to LTS (#30213)
 add d5aa44c9ba9 Handle a user whose account was deleted. (#30192)
 add a341eb69abe Added default watermark generation interval (apache#30141)
 add cd5f2711c83 Merge pull request #30141: Adding a default watermark emit 
interval for FlinkUnboundedSourceReader
 add b0f2eebb024 When failing work items during commit, make sure to call 
completeWork… (#30215)
 add 23dcb7ec1d5 Revert "When failing work items during commit, make sure 
to call completeWork…" (#30228)
 add 5494f114382 [Dataflow Streaming] Invalidate caches and remove work on 
failure before commit (#30229)
 add 27f1c0774fd Bump github.com/tetratelabs/wazero from 1.5.0 to 1.6.0 in 
/sdks (#30222)
 add 78d08235c72 Add grpc-services, grpc-utils to vendored grpc (#30196)
 add 3412b4f7a5b fix markdown links (#30227)
 add c006c5e2d53 Add Default method for OutputReceiver.outputWindowedValue 
(#30220)
 add 08a717dc5c2 Creating a Fully Managed Beam Streaming System with Flink 
Runner on Kubernetes - Part 3 (#29860)
 add 7fa4c99535f Fix to take StreamingDataflowWorkerOptions from external 
options (#30232)
 add f08058ca75c Bump cloud.google.com/go/spanner from 1.53.1 to 1.56.0 in 
/sdks (#30224)
 add 7a46686577d Fix a naming conflict of the TransformService test suite 
(#30243)
 add 4645a76ee2a gcsio: reduce number of get requests in function calls 
(#30205)
 add f250a843230 Adjust transform-service module names and deps with 
sdks/java/core
 add 13a70844abb Merge pull request #30237: Adjust transform-service module 
names and deps with sdks/java/core
 add 4f3963f7878 Bump google.golang.org/grpc from 1.60.1 to 1.61.0 in /sdks 
(#30249)
 add 2c5d153bf29 Add InferenceAPIEmbeddings to huggingface (#30138)
 add c72a9f83fed Fix Jms drop record (#30218)
 add a15dd7e7f84 [Go SDK] Use distroless:debian12 (no-ssl) as base image. 
(#30011)
 add 9b4b509e52c Fail later when duplicate transform translators are on the 
classpath
 add 382c6dccfcc Merge pull request #30252: Fail later when duplicate 
transform translators are on the classpath
 add 24efe7619d9 Track windmill

(beam) branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1 updated (19d70b4bd03 -> 6e43de65d99)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/service/s3-1.48.1
in repository https://gitbox.apache.org/repos/asf/beam.git


 discard 19d70b4bd03 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks
 add 89d1c06e1ea Fix Fhir IO Search IT (#30158)
 add a221f98a5f4 fix: fix error handling in segment reader and add an 
integration test (#30147)
 add ddb93272ab6 Fix git trusted directory errors in website build. (#30171)
 add 30917ff7b3e Don't use '-' as a delimiter in BigQuerySinkMetrics 
(#30118)
 add fb0762352c2 Periodically send perWorkerMetrics on the WorkerMessages 
API (#30135)
 add 4f09e62f520 Improve varint encoding throughput with unrolled loop 
(#29689)
 add 5188e37ed44 Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 
in /sdks (#30175)
 add 96cf695a101 Fix JdbcIO PreCommit mysql breaking change (#30179)
 add 824e06cd967 Allow stale_bq_datasets_cleaner.sh to run to completion, 
even when there are failures to grab a dataset (#30153)
 add 1d9f6044c76 Remove constraint introduced in #30059 (#30186)
 add bdbbef9b54d [YAML] - PubSubLite proto (#30129)
 add 1a5466b7df7 Support dict as the input of delete_batch (#30188)
 add 574e7bfbc1d Fix script typos for validating docker licenses. (#30176)
 add 7fb05cbb5e1 Move the code from sdks-java-fn-execution to sdks-java-core
 add e8df60381a3 Merge pull request #30170: Move the code from 
sdks-java-fn-execution to sdks-java-core
 add 124880bc8ce Fix typo affecting 
DataflowPipelineDebugOptions.setStreamingSideInputCacheExpirationMillis (#30195)
 add 1f69271044d [Dataflow Streaming] Add a option to use multiple commit 
threads (#30194)
 add 1183b5d5274 Knowledge lookup prompts: (#29747)
 add 801682930e2 Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in 
/sdks (#29913)
 add 30a778b9cbe Bump github.com/containerd/containerd from 1.7.7 to 1.7.11 
in /sdks (#29823)
 add f03b115830b Remove extraneous mocking that was causing NPEs in 
DataflowWorkUnitClientTest
 add 00f07864784 Remove sdks/java/fn-execution
 add 8ec51f181a0 Merge pull request #30193: Remove sdks/java/fn-execution
 add 9f8dda274dc Split standalone transform service application from 
launcher jar (#30190)
 add f2056e72dea Fix stale dataflow prebuilt image cleaner (#30168)
 add 34bbbda9f98 Try a different approach for cibuildwheel flake. (#30204)
 add 1a1ff756f86 Bump golang.org/x/sync from 0.5.0 to 0.6.0 in /sdks 
(#30209)
 add c1b3a27ee0c Bump github.com/nats-io/nats-server/v2 from 2.10.7 to 
2.10.10 in /sdks (#30210)
 add a47b1faa527 [Prism] Implement jobservices.Server Cancel (#30178)
 add 7f91baadcf2 Upgrade website node to LTS (#30213)
 add d5aa44c9ba9 Handle a user whose account was deleted. (#30192)
 add a341eb69abe Added default watermark generation interval (apache#30141)
 add cd5f2711c83 Merge pull request #30141: Adding a default watermark emit 
interval for FlinkUnboundedSourceReader
 add b0f2eebb024 When failing work items during commit, make sure to call 
completeWork… (#30215)
 add 23dcb7ec1d5 Revert "When failing work items during commit, make sure 
to call completeWork…" (#30228)
 add 5494f114382 [Dataflow Streaming] Invalidate caches and remove work on 
failure before commit (#30229)
 add 27f1c0774fd Bump github.com/tetratelabs/wazero from 1.5.0 to 1.6.0 in 
/sdks (#30222)
 add 78d08235c72 Add grpc-services, grpc-utils to vendored grpc (#30196)
 add 3412b4f7a5b fix markdown links (#30227)
 add c006c5e2d53 Add Default method for OutputReceiver.outputWindowedValue 
(#30220)
 add 08a717dc5c2 Creating a Fully Managed Beam Streaming System with Flink 
Runner on Kubernetes - Part 3 (#29860)
 add 7fa4c99535f Fix to take StreamingDataflowWorkerOptions from external 
options (#30232)
 add f08058ca75c Bump cloud.google.com/go/spanner from 1.53.1 to 1.56.0 in 
/sdks (#30224)
 add 7a46686577d Fix a naming conflict of the TransformService test suite 
(#30243)
 add 4645a76ee2a gcsio: reduce number of get requests in function calls 
(#30205)
 add f250a843230 Adjust transform-service module names and deps with 
sdks/java/core
 add 13a70844abb Merge pull request #30237: Adjust transform-service module 
names and deps with sdks/java/core
 add 4f3963f7878 Bump google.golang.org/grpc from 1.60.1 to 1.61.0 in /sdks 
(#30249)
 add 2c5d153bf29 Add InferenceAPIEmbeddings to huggingface (#30138)
 add c72a9f83fed Fix Jms drop record (#30218)
 add a15dd7e7f84 [Go SDK] Use distroless:debian12 (no-ssl) as base image. 
(#30011)
 add 9b4b509e52c Fail later when duplicate transform translators are on the 
classpath
 add 382c6dccfcc Merge pull request #30252: Fail later when duplicate 
transform translators are on the classpath
 add 24efe7619d9 Track windmill current active wo

(beam) 01/01: Include MLTransform IT test suite as a part of PostCommits

This is an automated email from the ASF dual-hosted git repository.

anandinguva pushed a commit to branch test-suite
in repository https://gitbox.apache.org/repos/asf/beam.git

commit bc5bba31e2ba87b50f8eee9886143f55d7af18b4
Author: Anand Inguva 
AuthorDate: Tue Feb 13 10:25:52 2024 -0500

Include MLTransform IT test suite as a part of PostCommits
---
 .../examples/ml_transform/ml_transform_it_test.py  |  1 +
 sdks/python/test-suites/dataflow/common.gradle | 22 --
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git 
a/sdks/python/apache_beam/examples/ml_transform/ml_transform_it_test.py 
b/sdks/python/apache_beam/examples/ml_transform/ml_transform_it_test.py
index 96fb3f77567..b17a85d5fc4 100644
--- a/sdks/python/apache_beam/examples/ml_transform/ml_transform_it_test.py
+++ b/sdks/python/apache_beam/examples/ml_transform/ml_transform_it_test.py
@@ -57,6 +57,7 @@ def _publish_metrics(pipeline, metric_value, metrics_table, 
metric_name):
   )])
 
 
+@pytest.mark.it_postcommit
 @pytest.mark.uses_tft
 class LargeMovieReviewDatasetProcessTest(unittest.TestCase):
   def test_process_large_movie_review_dataset(self):
diff --git a/sdks/python/test-suites/dataflow/common.gradle 
b/sdks/python/test-suites/dataflow/common.gradle
index 5fc1751a968..beee9b073c9 100644
--- a/sdks/python/test-suites/dataflow/common.gradle
+++ b/sdks/python/test-suites/dataflow/common.gradle
@@ -475,7 +475,7 @@ task tftTests {
 def argMap = [
 "test_opts": testOpts,
 "suite": "TFTransformTests-df${pythonVersionSuffix}",
-"collect": "uses_tft",
+"collect": "uses_tft and not it_postcommit",
 "requirements_file": 
"apache_beam/testing/benchmarks/cloudml/requirements.txt",
 "pipeline_opts": opts,
 ]
@@ -487,14 +487,32 @@ task tftTests {
   }
 }
 
+task MLTransformITTest {
+  dependsOn "installTFTRequirements"
+  doLast {
+def opts = project.findProperty('opts')
+opts += " --sdk_location=${project.ext.sdkLocation}"
+def testOpts = basicPytestOpts + ["--numprocesses=8", "--dist=loadfile"]
+def argMap = [
+"test_opts": testOpts,
+"suite": "MLTransformITTest-df${pythonVersionSuffix}",
+// Add MLTransform IT tests pytest markers.
+"collect": "uses_tft and it_postcommit",
+"requirements_file": 
"apache_beam/testing/benchmarks/cloudml/requirements.txt",
+"pipeline_opts": opts,
+]
+  }
+}
+
 // add all RunInference E2E tests that run on DataflowRunner
 // As of now, this test suite is enable in py38 suite as the base NVIDIA image 
used for Tensor RT
 // contains Python 3.8.
 // TODO: https://github.com/apache/beam/issues/22651
-project.tasks.register("inferencePostCommitIT") {
+project.tasks.register("BeamMLPostCommitIT") {
   dependsOn = [
   'tensorRTtests',
   'vertexAIInferenceTest',
+  'MLTransformITTest'
   ]
 }

(beam) branch master updated (0ade62cd083 -> 3b09283e45d)

2024-02-13 Thread jrmccluskey

This is an automated email from the ASF dual-hosted git repository.

jrmccluskey pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 0ade62cd083 Only populate DataflowHistogramValue::OutlierStats when 
overflow/underflow values exist. (#30246)
 add 3b09283e45d Bump github.com/docker/go-connections from 0.4.0 to 0.5.0 
in /sdks (#30247)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

(beam) branch dependabot/go_modules/sdks/github.com/docker/go-connections-0.5.0 deleted (was 4a31bf4b4c0)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/go_modules/sdks/github.com/docker/go-connections-0.5.0
in repository https://gitbox.apache.org/repos/asf/beam.git


 was 4a31bf4b4c0 Bump github.com/docker/go-connections from 0.4.0 to 0.5.0 
in /sdks

The revisions that were on this branch are still contained in
other references; therefore, this change does not discard any commits
from the repository.

(beam) branch master updated: Only populate DataflowHistogramValue::OutlierStats when overflow/underflow values exist. (#30246)

2024-02-13 Thread yhu

This is an automated email from the ASF dual-hosted git repository.

yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 0ade62cd083 Only populate DataflowHistogramValue::OutlierStats when 
overflow/underflow values exist. (#30246)
0ade62cd083 is described below

commit 0ade62cd08361e8769eed1f0148234f57a7ba6f6
Author: JayajP 
AuthorDate: Tue Feb 13 07:12:07 2024 -0800

Only populate DataflowHistogramValue::OutlierStats when overflow/underflow 
values exist. (#30246)

* Only populate outlier stats when overflow/underflow values exist.

* Fix StreamingStepMetricsContainerTest
---
 .../MetricsToPerStepNamespaceMetricsConverter.java | 73 ++
 ...ricsToPerStepNamespaceMetricsConverterTest.java |  9 +--
 .../worker/StreamingStepMetricsContainerTest.java  | 10 +--
 3 files changed, 50 insertions(+), 42 deletions(-)

diff --git 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java
 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java
index 7b72d650778..8f9cbd350a2 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/MetricsToPerStepNamespaceMetricsConverter.java
@@ -62,6 +62,35 @@ public class MetricsToPerStepNamespaceMetricsConverter {
 .setValueInt64(value));
   }
 
+  /**
+   * Adds {@code outlierStats} to {@code outputHistogram} if {@code 
inputHistogram} has recorded
+   * overflow or underflow values.
+   *
+   * @param inputHistogram
+   * @param outputHistogram
+   */
+  private static void addOutlierStatsToHistogram(
+  HistogramData inputHistogram, DataflowHistogramValue outputHistogram) {
+long overflowCount = inputHistogram.getTopBucketCount();
+long underflowCount = inputHistogram.getBottomBucketCount();
+if (underflowCount == 0 && overflowCount == 0) {
+  return;
+}
+
+OutlierStats outlierStats = new OutlierStats();
+if (underflowCount > 0) {
+  outlierStats
+  .setUnderflowCount(underflowCount)
+  .setUnderflowMean(inputHistogram.getBottomBucketMean());
+}
+if (overflowCount > 0) {
+  outlierStats
+  .setOverflowCount(overflowCount)
+  .setOverflowMean(inputHistogram.getTopBucketMean());
+}
+outputHistogram.setOutlierStats(outlierStats);
+  }
+
   /**
* @param metricName The {@link MetricName} that represents this Histogram.
* @param value The histogram value. Currently we only support converting 
histograms that use
@@ -70,8 +99,8 @@ public class MetricsToPerStepNamespaceMetricsConverter {
* Otherwise returns an empty optional.
*/
   private static Optional convertHistogramToMetricValue(
-  MetricName metricName, HistogramData value) {
-if (value.getTotalCount() == 0L) {
+  MetricName metricName, HistogramData inputHistogram) {
+if (inputHistogram.getTotalCount() == 0L) {
   return Optional.empty();
 }
 
@@ -81,32 +110,33 @@ public class MetricsToPerStepNamespaceMetricsConverter {
   return Optional.empty();
 }
 
-DataflowHistogramValue histogramValue = new DataflowHistogramValue();
-int numberOfBuckets = value.getBucketType().getNumBuckets();
+DataflowHistogramValue outputHistogram = new DataflowHistogramValue();
+int numberOfBuckets = inputHistogram.getBucketType().getNumBuckets();
 
-if (value.getBucketType() instanceof HistogramData.LinearBuckets) {
-  HistogramData.LinearBuckets buckets = (HistogramData.LinearBuckets) 
value.getBucketType();
+if (inputHistogram.getBucketType() instanceof HistogramData.LinearBuckets) 
{
+  HistogramData.LinearBuckets buckets =
+  (HistogramData.LinearBuckets) inputHistogram.getBucketType();
   Linear linearOptions =
   new Linear()
   .setNumberOfBuckets(numberOfBuckets)
   .setWidth(buckets.getWidth())
   .setStart(buckets.getStart());
-  histogramValue.setBucketOptions(new 
BucketOptions().setLinear(linearOptions));
-} else if (value.getBucketType() instanceof 
HistogramData.ExponentialBuckets) {
+  outputHistogram.setBucketOptions(new 
BucketOptions().setLinear(linearOptions));
+} else if (inputHistogram.getBucketType() instanceof 
HistogramData.ExponentialBuckets) {
   HistogramData.ExponentialBuckets buckets =
-  (HistogramData.ExponentialBuckets) value.getBucketType();
+  (HistogramData.ExponentialBuckets) inputHistogram.getBucketType();
   Base2Exponent expoenntialOptions =
   new 
Base2Exponent().setNumberOfBuckets(nu

(beam) 01/01: Debug logging for prebuilt container workflow

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a commit to branch damccorm-patch-2
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 2994fc6a4231a7104cd4575c19edde2284834aae
Author: Danny McCormick 
AuthorDate: Tue Feb 13 09:48:29 2024 -0500

Debug logging for prebuilt container workflow
---
 .test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh 
b/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh
index 126249324fe..9cb8f24913b 100755
--- a/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh
+++ b/.test-infra/tools/stale_dataflow_prebuilt_image_cleaner.sh
@@ -17,7 +17,7 @@
 #
 #Delete stale python prebuilt SDK container images older than one day.
 #
-set -euo pipefail
+set -exuo pipefail
 
 # Clean up private registry (us.gcr.io)
 # Images more than 5 day old and not the latest (either has latest label or 
newest)
@@ -143,4 +143,4 @@ fi
 if [ -n "$FAILED_IMAGES" ]; then
   echo "Failed delete images $FAILED_IMAGES"
   exit 1
-fi
\ No newline at end of file
+fi

(beam) branch damccorm-patch-2 created (now 2994fc6a423)

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch damccorm-patch-2
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 2994fc6a423 Debug logging for prebuilt container workflow

This branch includes the following new commits:

 new 2994fc6a423 Debug logging for prebuilt container workflow

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 8010db8 Updating config from bot
8010db8 is described below

commit 8010db82aac113daeb189ac33c42b85e1de8
Author: github-actions 
AuthorDate: Tue Feb 13 14:47:53 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30293.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30293.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30293.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30293.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) 01/01: Delete update-reviewers

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a commit to branch damccorm-patch-1
in repository https://gitbox.apache.org/repos/asf/beam.git

commit d5a32fe1353734f4e57aa1f0b822b769927cc1d5
Author: Danny McCormick 
AuthorDate: Tue Feb 13 09:45:49 2024 -0500

Delete update-reviewers
---
 .github/workflows/pr-bot-update-reviewers.yml | 48 ---
 1 file changed, 48 deletions(-)

diff --git a/.github/workflows/pr-bot-update-reviewers.yml 
b/.github/workflows/pr-bot-update-reviewers.yml
deleted file mode 100644
index ac80d1e346e..000
--- a/.github/workflows/pr-bot-update-reviewers.yml
+++ /dev/null
@@ -1,48 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-name: pr-bot-update-reviewers
-
-# Run every week on Sunday at 16:05 UTC
-on:
-  schedule:
-  - cron: '5 16 * * 0'
-  workflow_dispatch:
-
-jobs:
-  update-reviewers:
-permissions:
-  contents: write
-  pull-requests: write
-  checks: read
-  issues: read
-  statuses: read
-# Don't run on forks
-if: github.repository == 'apache/beam'
-runs-on: [self-hosted, ubuntu-20.04]
-steps:
-  - uses: actions/checkout@v4
-  - name: Setup Node
-uses: actions/setup-node@v4
-with:
-  node-version: 16
-  - name: Install pr-bot npm dependencies
-run: npm ci
-working-directory: 'scripts/ci/pr-bot'
-  # Runs a set of commands using the runners shell
-  - run: npm run updateReviewers
-env:
-  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-working-directory: 'scripts/ci/pr-bot'

(beam) branch damccorm-patch-1 created (now d5a32fe1353)

This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch damccorm-patch-1
in repository https://gitbox.apache.org/repos/asf/beam.git


  at d5a32fe1353 Delete update-reviewers

This branch includes the following new commits:

 new d5a32fe1353 Delete update-reviewers

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.

(beam) branch master updated: check for cachetoken representing a retry before activating and completing work (#29082)

2024-02-13 Thread scwhittle

This is an automated email from the ASF dual-hosted git repository.

scwhittle pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 514b03bebcc check for cachetoken representing a retry before 
activating and completing work (#29082)
514b03bebcc is described below

commit 514b03bebcc1addf39d2250c09c6aa42ee68b3db
Author: martin trieu 
AuthorDate: Tue Feb 13 04:41:41 2024 -0800

check for cachetoken representing a retry before activating and completing 
work (#29082)
---
 .../dataflow/worker/StreamingDataflowWorker.java   |  40 ++--
 .../dataflow/worker/streaming/ActiveWorkState.java | 161 +++
 .../worker/streaming/ComputationState.java |  14 +-
 .../runners/dataflow/worker/streaming/Work.java|  10 +
 .../runners/dataflow/worker/streaming/WorkId.java  |  48 +
 .../worker/StreamingDataflowWorkerTest.java|  94 +++--
 .../worker/streaming/ActiveWorkStateTest.java  | 216 -
 7 files changed, 418 insertions(+), 165 deletions(-)

diff --git 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
index 2f9e18cde67..4d2ef6a03cf 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
@@ -87,7 +87,6 @@ import 
org.apache.beam.runners.dataflow.worker.status.DebugCapture.Capturable;
 import 
org.apache.beam.runners.dataflow.worker.status.LastExceptionDataProvider;
 import org.apache.beam.runners.dataflow.worker.status.StatusDataProvider;
 import org.apache.beam.runners.dataflow.worker.status.WorkerStatusPages;
-import 
org.apache.beam.runners.dataflow.worker.streaming.ActiveWorkState.FailedTokens;
 import org.apache.beam.runners.dataflow.worker.streaming.Commit;
 import org.apache.beam.runners.dataflow.worker.streaming.ComputationState;
 import org.apache.beam.runners.dataflow.worker.streaming.ExecutionState;
@@ -97,6 +96,7 @@ import 
org.apache.beam.runners.dataflow.worker.streaming.StageInfo;
 import org.apache.beam.runners.dataflow.worker.streaming.WeightedBoundedQueue;
 import org.apache.beam.runners.dataflow.worker.streaming.Work;
 import org.apache.beam.runners.dataflow.worker.streaming.Work.State;
+import org.apache.beam.runners.dataflow.worker.streaming.WorkId;
 import 
org.apache.beam.runners.dataflow.worker.streaming.sideinput.SideInputStateFetcher;
 import org.apache.beam.runners.dataflow.worker.util.BoundedQueueExecutor;
 import org.apache.beam.runners.dataflow.worker.util.MemoryMonitor;
@@ -104,6 +104,7 @@ import 
org.apache.beam.runners.dataflow.worker.util.common.worker.ElementCounter
 import 
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputObjectAndByteCounter;
 import 
org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation;
 import org.apache.beam.runners.dataflow.worker.windmill.Windmill;
+import 
org.apache.beam.runners.dataflow.worker.windmill.Windmill.ComputationHeartbeatResponse;
 import 
org.apache.beam.runners.dataflow.worker.windmill.Windmill.LatencyAttribution;
 import 
org.apache.beam.runners.dataflow.worker.windmill.Windmill.WorkItemCommitRequest;
 import org.apache.beam.runners.dataflow.worker.windmill.WindmillServerStub;
@@ -1311,7 +1312,7 @@ public class StreamingDataflowWorker {
 // Consider the item invalid. It will eventually be retried by 
Windmill if it still needs to
 // be processed.
 computationState.completeWorkAndScheduleNextWorkForKey(
-ShardedKey.create(key, workItem.getShardingKey()), 
workItem.getWorkToken());
+ShardedKey.create(key, workItem.getShardingKey()), work.id());
   }
 } finally {
   // Update total processing time counters. Updating in finally clause 
ensures that
@@ -1389,7 +1390,10 @@ public class StreamingDataflowWorker {
 for (Windmill.WorkItemCommitRequest workRequest : 
entry.getValue().getRequestsList()) {
   computationState.completeWorkAndScheduleNextWorkForKey(
   ShardedKey.create(workRequest.getKey(), 
workRequest.getShardingKey()),
-  workRequest.getWorkToken());
+  WorkId.builder()
+  .setCacheToken(workRequest.getCacheToken())
+  .setWorkToken(workRequest.getWorkToken())
+  .build());
 }
   }
 }
@@ -1409,7 +1413,11 @@ public class StreamingDataflowWorker {
   .forComputation(state.getComputationId())
   .invalidate(request.getKey(), request.getShardingKey());
   state.completeWorkAndScheduleNextWorkForKey(
-  ShardedKey.crea

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 8ae4aeb1be1 Updating config from bot
8ae4aeb1be1 is described below

commit 8ae4aeb1be1f77b19cee49211f7ae4f39bd03f6d
Author: github-actions 
AuthorDate: Tue Feb 13 12:13:42 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 81a56b4846c..26bded8e7c0 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -2,7 +2,7 @@
   "label": "java",
   "dateOfLastReviewAssignment": {
 "lukecwik": 1680701852535,
-"kennknowles": 170723738,
+"kennknowles": 1707826419673,
 "robertwb": 1707764084416,
 "kileys": 1674428678843,
 "apilloud": 1678822446183,

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 378761d60a5 Updating config from bot
378761d60a5 is described below

commit 378761d60a5d00f64a42aae5954cdfd5be73bb44
Author: github-actions 
AuthorDate: Tue Feb 13 12:13:43 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index f1c86dc5c00..8f96d2543b2 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -8,6 +8,6 @@
 "ahmedabu98": 1707343533957,
 "bvolpato": 1707764084416,
 "manavgarg": 1690826779210,
-"damondouglas": 170723738
+"damondouglas": 1707826420096
   }
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new e42e8df0f14 Updating config from bot
e42e8df0f14 is described below

commit e42e8df0f1460c14b6b2feaf87fdb9020de946fb
Author: github-actions 
AuthorDate: Tue Feb 13 12:13:41 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30172.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30172.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30172.json
index 049ee37801c..00e748fba87 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30172.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30172.json
@@ -1,8 +1,8 @@
 {
   "commentedAboutFailingChecks": false,
   "reviewersAssignedForLabels": {
-"java": "bvolpato",
-"io": "chamikaramj",
+"java": "kennknowles",
+"io": "damondouglas",
 "bigtable": "igorbernstein2"
   },
   "nextAction": "Reviewers",

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new c7150deb62a Updating config from bot
c7150deb62a is described below

commit c7150deb62a9dc4d6c39e2074b999ab04a3326ad
Author: github-actions 
AuthorDate: Tue Feb 13 11:56:13 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30287.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30287.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30287.json
index e6d8e3d0a02..0144737f932 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-30287.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30287.json
@@ -4,7 +4,7 @@
 "no-matching-label": "AnandInguva"
   },
   "nextAction": "Reviewers",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 02d2c561e0b Updating config from bot
02d2c561e0b is described below

commit 02d2c561e0b57966e594f20321d915b4886f9f0b
Author: github-actions 
AuthorDate: Tue Feb 13 11:34:28 2024 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-30287.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-30287.json 
b/scripts/ci/pr-bot/state/pr-state/pr-30287.json
new file mode 100644
index 000..e6d8e3d0a02
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-30287.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"no-matching-label": "AnandInguva"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

(beam) branch pr-bot-state updated: Updating config from bot