(beam) branch master updated: [Python] BigQuery handler for enrichment transform (#31295)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 44177d1a9f3 [Python] BigQuery handler for enrichment transform (#31295)
44177d1a9f3 is described below

commit 44177d1a9f32c3707f60a6a7a3b67334372a4291
Author: Ritesh Ghorse 
AuthorDate: Fri May 17 14:58:42 2024 -0400

[Python] BigQuery handler for enrichment transform (#31295)

* bigquery enrichment with it

* unit tests

* more integration tests, move BE to RRIO

* trigger postcommit

* fix doc, project name

* refactor, improve doc comments

* refactor

* remove query_fn check
---
 .github/trigger_files/beam_PostCommit_Python.json  |   1 +
 CHANGES.md |   1 +
 sdks/python/apache_beam/io/requestresponse.py  | 121 ++---
 sdks/python/apache_beam/transforms/enrichment.py   |  14 +-
 .../transforms/enrichment_handlers/bigquery.py | 256 ++
 .../enrichment_handlers/bigquery_it_test.py| 289 +
 .../enrichment_handlers/bigquery_test.py   |  70 +
 sdks/python/scripts/generate_pydoc.sh  |   1 +
 8 files changed, 705 insertions(+), 48 deletions(-)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
index c4edaa85a89..63bd5651def 100644
--- a/.github/trigger_files/beam_PostCommit_Python.json
+++ b/.github/trigger_files/beam_PostCommit_Python.json
@@ -1,3 +1,4 @@
 {
   "comment": "Modify this file in a trivial way to cause this test suite to 
run"
 }
+ 
diff --git a/CHANGES.md b/CHANGES.md
index bce9636237e..dd66e9192b6 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -71,6 +71,7 @@
 * Beam YAML now supports the jinja templating syntax.
   Template variables can be passed with the (json-formatted) 
`--jinja_variables` flag.
 * DataFrame API now supports pandas 2.1.x and adds 12 more string functions 
for Series.([#31185](https://github.com/apache/beam/pull/31185)).
+* Added BigQuery handler for enrichment transform (Python) 
([#31295](https://github.com/apache/beam/pull/31295))
 
 ## Breaking Changes
 
diff --git a/sdks/python/apache_beam/io/requestresponse.py 
b/sdks/python/apache_beam/io/requestresponse.py
index 4458aa59c18..d7011e5a8ff 100644
--- a/sdks/python/apache_beam/io/requestresponse.py
+++ b/sdks/python/apache_beam/io/requestresponse.py
@@ -28,6 +28,8 @@ from datetime import timedelta
 from typing import Any
 from typing import Dict
 from typing import Generic
+from typing import List
+from typing import Mapping
 from typing import Optional
 from typing import Tuple
 from typing import TypeVar
@@ -42,6 +44,7 @@ from apache_beam.coders import coders
 from apache_beam.io.components.adaptive_throttler import AdaptiveThrottler
 from apache_beam.metrics import Metrics
 from apache_beam.ml.inference.vertex_ai_inference import MSEC_TO_SEC
+from apache_beam.transforms.util import BatchElements
 from apache_beam.utils import retry
 
 RequestT = TypeVar('RequestT')
@@ -143,6 +146,10 @@ class Caller(contextlib.AbstractContextManager,
 """
 return ""
 
+  def batch_elements_kwargs(self) -> Mapping[str, Any]:
+"""Returns a kwargs suitable for `beam.BatchElements`."""
+return {}
+
 
 class ShouldBackOff(abc.ABC):
   """
@@ -476,53 +483,67 @@ class _RedisCaller(Caller):
   def __enter__(self):
 self.client = redis.Redis(self.host, self.port, **self.kwargs)
 
-  def __call__(self, element, *args, **kwargs):
-if self.mode == _RedisMode.READ:
-  cache_request = self.source_caller.get_cache_key(element)
-  # check if the caller is a enrichment handler. EnrichmentHandler
-  # provides the request format for cache.
-  if cache_request:
-encoded_request = self.request_coder.encode(cache_request)
-  else:
-encoded_request = self.request_coder.encode(element)
-
-  encoded_response = self.client.get(encoded_request)
-  if not encoded_response:
-# no cache entry present for this request.
+  def _read_cache(self, element):
+cache_request = self.source_caller.get_cache_key(element)
+# check if the caller is a enrichment handler. EnrichmentHandler
+# provides the request format for cache.
+if cache_request:
+  encoded_request = self.request_coder.encode(cache_request)
+else:
+  encoded_request = self.request_coder.encode(element)
+
+encoded_response = self.client.get(encoded_request)
+if not encoded_response:
+  # no cache entry present for this request.
+  return element, None
+
+if self.response_coder is None:
+  try:
+response_dict = json.loads(encoded_response.decode('utf-8'))
+response = beam.Row(**

(beam) branch master updated (8b5d0009a1d -> fbb3ae1ebef)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 8b5d0009a1d Fix Python 3.12 python hinting errors on MacOS (#31246)
 add fbb3ae1ebef add retries for flaky test (#31247)

No new revisions were added by this update.

Summary of changes:
 sdks/python/apache_beam/io/requestresponse_test.py | 58 ++
 1 file changed, 37 insertions(+), 21 deletions(-)

(beam) branch master updated: [Python] Add feast feature store handler for enrichment transform (#30957)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 3329edba79f [Python] Add feast feature store handler for enrichment 
transform (#30957)
3329edba79f is described below

commit 3329edba79f7b99af983b22cf44ab6e77a8ee987
Author: Ritesh Ghorse 
AuthorDate: Fri Apr 26 14:31:28 2024 -0400

[Python] Add feast feature store handler for enrichment transform (#30957)

* add feast feature store handler

* add changes, unit test

* remove duplicate test, add doc

* correct string formatting

* add lambda, use filesystems, start test

* update pydoc
---
 .github/trigger_files/beam_PostCommit_Python.json  |   1 -
 CHANGES.md |  34 
 .../enrichment_handlers/feast_feature_store.py | 200 +
 .../feast_feature_store_it_test.py | 121 +
 .../feast_feature_store_test.py|  67 +++
 .../feast_tests_requirements.txt   |  18 ++
 sdks/python/pytest.ini |   1 +
 sdks/python/scripts/generate_pydoc.sh  |   2 +-
 sdks/python/test-suites/direct/common.gradle   |  28 +++
 9 files changed, 470 insertions(+), 2 deletions(-)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
index 63bd5651def..c4edaa85a89 100644
--- a/.github/trigger_files/beam_PostCommit_Python.json
+++ b/.github/trigger_files/beam_PostCommit_Python.json
@@ -1,4 +1,3 @@
 {
   "comment": "Modify this file in a trivial way to cause this test suite to 
run"
 }
- 
diff --git a/CHANGES.md b/CHANGES.md
index 97894b08374..ec7fbe45668 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -53,6 +53,40 @@
 * ([#X](https://github.com/apache/beam/issues/X)).
 -->
 
+# [2.57.0] - Unreleased
+
+## Highlights
+
+* New highly anticipated feature X added to Python SDK 
([#X](https://github.com/apache/beam/issues/X)).
+* New highly anticipated feature Y added to Java SDK 
([#Y](https://github.com/apache/beam/issues/Y)).
+
+## I/Os
+
+* Support for X source added (Java/Python) 
([#X](https://github.com/apache/beam/issues/X)).
+
+## New Features / Improvements
+
+* Added Feast feature store handler for enrichment transform (Python) 
([#30957](https://github.com/apache/beam/issues/30964)).
+
+## Breaking Changes
+
+* X behavior was changed ([#X](https://github.com/apache/beam/issues/X)).
+
+## Deprecations
+
+* X behavior is deprecated and will be removed in X versions 
([#X](https://github.com/apache/beam/issues/X)).
+
+## Bugfixes
+
+* Fixed X (Java/Python) ([#X](https://github.com/apache/beam/issues/X)).
+
+## Security Fixes
+* Fixed (CVE--)[https://www.cve.org/CVERecord?id=CVE--] 
(Java/Python/Go) ([#X](https://github.com/apache/beam/issues/X)).
+
+## Known Issues
+
+* ([#X](https://github.com/apache/beam/issues/X)).
+
 # [2.56.0] - Unreleased
 
 ## Highlights
diff --git 
a/sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store.py 
b/sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store.py
new file mode 100644
index 000..dc2a71786f6
--- /dev/null
+++ 
b/sdks/python/apache_beam/transforms/enrichment_handlers/feast_feature_store.py
@@ -0,0 +1,200 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+import logging
+import tempfile
+from pathlib import Path
+from typing import Any
+from typing import Callable
+from typing import List
+from typing import Mapping
+from typing import Optional
+
+import apache_beam as beam
+from apache_beam.io.filesystems import FileSystems
+from apache_beam.transforms.enrichment import EnrichmentSourceHandler
+from apache_beam.transforms.enrichment_handlers.utils import ExceptionLevel
+from feast import FeatureStore
+
+__all__ = [
+'FeastFeatureStoreEnrichmentHandler',
+]
+
+EntityRowFn = Callable[[beam.Row], Mapping[str, Any]]
+
+_LOGGER = logging.getLogger(__name__)
+
+LOCAL_FEATURE_STORE_YAML_FILENAME = 'fs_yaml_file.yaml'
+
+

(beam) branch master updated: Bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdks (#31054)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 49da7a34d99 Bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdks 
(#31054)
49da7a34d99 is described below

commit 49da7a34d999e2565f0b819639a553f3bc6d8e73
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Apr 22 11:15:31 2024 -0400

Bump golang.org/x/net from 0.22.0 to 0.23.0 in /sdks (#31054)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.22.0 to 
0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 65d244b265c..5d3f6cfbd58 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -54,7 +54,7 @@ require (
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
go.mongodb.org/mongo-driver v1.13.1
-   golang.org/x/net v0.22.0
+   golang.org/x/net v0.23.0
golang.org/x/oauth2 v0.18.0
golang.org/x/sync v0.6.0
golang.org/x/sys v0.18.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 4a51032d86b..d824a1a1113 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -556,8 +556,8 @@ golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod 
h1:sp8m0HH+o8qH0wwXwY
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod 
h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-2022202133-69e39bad7dc2/go.mod 
h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod 
h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
-golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
+golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
+golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod 
h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=

(beam) branch master updated: [Python] Clean doc related to write data in bigquery.py (#30887)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 2eb1a756258 [Python] Clean doc related to write data in bigquery.py 
(#30887)
2eb1a756258 is described below

commit 2eb1a756258b24b4c3b92c14e0959d64b0008bbf
Author: Kevin ZHOU 
AuthorDate: Thu Apr 18 15:57:47 2024 +0200

[Python] Clean doc related to write data in bigquery.py (#30887)

* add missing closing parenthesis
* add unique names to PTransform operations
---
 sdks/python/apache_beam/io/gcp/bigquery.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/python/apache_beam/io/gcp/bigquery.py 
b/sdks/python/apache_beam/io/gcp/bigquery.py
index 08698b273b1..43bd1702218 100644
--- a/sdks/python/apache_beam/io/gcp/bigquery.py
+++ b/sdks/python/apache_beam/io/gcp/bigquery.py
@@ -140,15 +140,15 @@ events of different types to different tables, and the 
table names are
 computed at pipeline runtime, one may do something like the following::
 
 with Pipeline() as p:
-  elements = (p | beam.Create([
+  elements = (p | 'Create elements' >> beam.Create([
 {'type': 'error', 'timestamp': '12:34:56', 'message': 'bad'},
 {'type': 'user_log', 'timestamp': '12:34:59', 'query': 'flu symptom'},
   ]))
 
-  table_names = (p | beam.Create([
+  table_names = (p | 'Create table_names' >> beam.Create([
 ('error', 'my_project:dataset1.error_table_for_today'),
 ('user_log', 'my_project:dataset1.query_table_for_today'),
-  ])
+  ]))
 
   table_names_dict = beam.pvalue.AsDict(table_names)
 

(beam) branch master updated (db585b7b462 -> 1a26eada0b0)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from db585b7b462 replace clock.milliseconds with stopwatch (#30678)
 add 1a26eada0b0 [Python] Allow lambda function in bigtable handler to 
build a custom row key (#30974)

No new revisions were added by this update.

Summary of changes:
 .github/trigger_files/beam_PostCommit_Python.json  |  1 +
 CHANGES.md |  1 +
 .../transforms/enrichment_handlers/bigtable.py | 26 
 .../enrichment_handlers/bigtable_it_test.py| 28 ++
 .../bigtable_test.py}  | 28 +++---
 5 files changed, 66 insertions(+), 18 deletions(-)
 copy sdks/python/apache_beam/transforms/{enrichment_test.py => 
enrichment_handlers/bigtable_test.py} (53%)

(beam) branch master updated: Bump tar (#30928)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new cd368cbd402 Bump tar (#30928)
cd368cbd402 is described below

commit cd368cbd402d38169a374236682f84a49777f5b9
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Apr 11 23:08:21 2024 -0400

Bump tar (#30928)

Bumps [tar](https://github.com/isaacs/node-tar) from 6.1.11 to 6.2.1.
- [Release notes](https://github.com/isaacs/node-tar/releases)
- [Changelog](https://github.com/isaacs/node-tar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/isaacs/node-tar/compare/v6.1.11...v6.2.1)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 .../extensions/apache-beam-jupyterlab-sidepanel/yarn.lock   | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git 
a/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/yarn.lock
 
b/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/yarn.lock
index f72e23ebd40..78e979b37f9 100644
--- 
a/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/yarn.lock
+++ 
b/sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel/yarn.lock
@@ -6700,6 +6700,11 @@ minipass@^3.0.0, minipass@^3.1.1:
   dependencies:
 yallist "^4.0.0"
 
+minipass@^5.0.0:
+  version "5.0.0"
+  resolved 
"https://registry.yarnpkg.com/minipass/-/minipass-5.0.0.tgz#3e9788ffb90b694a5d0ec94479a45b5d8738133d;
+  integrity 
sha512-3FnjYuehv9k6ovOEbyOswadCDPX1piCfhV8ncmYtHOjuPwylVWsghTLo7rabjC3Rx5xD4HDx8Wm1xnMF7S5qFQ==
+
 minizlib@^2.1.1:
   version "2.1.2"
   resolved 
"https://registry.yarnpkg.com/minizlib/-/minizlib-2.1.2.tgz#e90d3466ba209b932451508a11ce3d3632145931;
@@ -8697,13 +8702,13 @@ tapable@^2.1.1, tapable@^2.2.0:
   integrity 
sha512-GNzQvQTOIP6RyTfE2Qxb8ZVlNmw0n88vp1szwWRimP02mnTsx3Wtn5qRdqY9w2XduFNUgvOwhNnQsjwCp+kqaQ==
 
 tar@^6.0.2:
-  version "6.1.11"
-  resolved 
"https://registry.yarnpkg.com/tar/-/tar-6.1.11.tgz#6760a38f003afa1b2ffd0ffe9e9abbd0eab3d621;
-  integrity 
sha512-an/KZQzQUkZCkuoAA64hM92X0Urb6VpRhAFllDzz44U2mcD5scmT3zBc4VgVpkugF580+DQn8eAFSyoQt0tznA==
+  version "6.2.1"
+  resolved 
"https://registry.yarnpkg.com/tar/-/tar-6.2.1.tgz#717549c541bc3c2af15751bea94b1dd068d4b03a;
+  integrity 
sha512-DZ4yORTwrbTj/7MZYq2w+/ZFdI6OZ/f9SFHR+71gIVUZhOQPHzVCLpvRnPgyaMpfWxxk/4ONva3GQSyNIKRv6A==
   dependencies:
 chownr "^2.0.0"
 fs-minipass "^2.0.0"
-minipass "^3.0.0"
+minipass "^5.0.0"
 minizlib "^2.1.1"
 mkdirp "^1.0.3"
 yallist "^4.0.0"

(beam) branch master updated (1651e54786d -> ffc96bc2edf)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 1651e54786d Docker - Remove Deprecated Types (#30826)
 add ffc96bc2edf Bump cloud.google.com/go/bigquery from 1.59.1 to 1.60.0 in 
/sdks (#30837)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 10 +-
 sdks/go.sum | 22 +++---
 2 files changed, 16 insertions(+), 16 deletions(-)

(beam) branch master updated (4eefc6dd544 -> 1651e54786d)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 4eefc6dd544 Apply changes (#30857)
 add 1651e54786d Docker - Remove Deprecated Types (#30826)

No new revisions were added by this update.

Summary of changes:
 sdks/go/pkg/beam/runners/prism/internal/environments.go | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

(beam) branch master updated: [Python] Update dev SDK container tag for Python SDK (#30803)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new fc5df6f261b [Python] Update dev SDK container tag for Python SDK 
(#30803)
fc5df6f261b is described below

commit fc5df6f261bbe6ea910ffc1de8e6c093c9751c60
Author: Ritesh Ghorse 
AuthorDate: Fri Mar 29 17:38:25 2024 -0400

[Python] Update dev SDK container tag for Python SDK (#30803)

* update container tags

* add postcommit check
---
 .github/trigger_files/beam_PostCommit_Python.json  | 0
 sdks/python/apache_beam/runners/dataflow/internal/names.py | 2 +-
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
deleted file mode 100644
index e69de29bb2d..000
diff --git a/sdks/python/apache_beam/runners/dataflow/internal/names.py 
b/sdks/python/apache_beam/runners/dataflow/internal/names.py
index fb588136b72..cd2924bf6e4 100644
--- a/sdks/python/apache_beam/runners/dataflow/internal/names.py
+++ b/sdks/python/apache_beam/runners/dataflow/internal/names.py
@@ -34,6 +34,6 @@ SERIALIZED_SOURCE_KEY = 'serialized_source'
 # Unreleased sdks use container image tag specified below.
 # Update this tag whenever there is a change that
 # requires changes to SDK harness container or SDK harness launcher.
-BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20240110'
+BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20240329'
 
 DATAFLOW_CONTAINER_IMAGE_REPOSITORY = 'gcr.io/cloud-dataflow/v1beta3'

(beam) branch master updated: [Python] Add redis client to python dependencies (#30677)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 25805db9278 [Python] Add redis client to python dependencies (#30677)
25805db9278 is described below

commit 25805db92786478d959ee3e2a1943e584e75c304
Author: Ritesh Ghorse 
AuthorDate: Thu Mar 28 20:16:16 2024 -0400

[Python] Add redis client to python dependencies (#30677)

* add redis client to python  dependencies

* fix lint

* update dependencies

* add upper bound

* remove redis requirements file and uses_redis tag

* new task for testcontainers

* fix expected value after syncing

* remove requirement file
---
 .github/trigger_files/beam_PostCommit_Python.json  |  0
 .../snippets/transforms/elementwise/enrichment_test.py |  6 +++---
 sdks/python/apache_beam/io/requestresponse.py  |  2 +-
 sdks/python/apache_beam/io/requestresponse_it_test.py  |  2 +-
 .../io/requestresponse_tests_requirements.txt  | 18 --
 .../transforms/enrichment_handlers/bigtable_it_test.py |  2 +-
 .../vertex_ai_feature_store_it_test.py |  2 +-
 .../vertex_ai_feature_store_tests_requirement.txt  | 18 --
 .../python/container/py310/base_image_requirements.txt |  4 +++-
 .../python/container/py311/base_image_requirements.txt |  3 ++-
 sdks/python/container/py38/base_image_requirements.txt |  4 +++-
 sdks/python/container/py39/base_image_requirements.txt |  4 +++-
 sdks/python/pytest.ini |  2 +-
 sdks/python/setup.py   |  1 +
 sdks/python/test-suites/dataflow/common.gradle |  9 +
 sdks/python/test-suites/direct/common.gradle   | 16 +---
 16 files changed, 26 insertions(+), 67 deletions(-)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
new file mode 100644
index 000..e69de29bb2d
diff --git 
a/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment_test.py
 
b/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment_test.py
index 3fd759bbc32..8a7cdfbe926 100644
--- 
a/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment_test.py
+++ 
b/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment_test.py
@@ -44,9 +44,9 @@ Row(sale_id=5, customer_id=5, product_id=4, quantity=2, 
product={'product_id': '
 
 def validate_enrichment_with_vertex_ai():
   expected = '''[START enrichment_with_vertex_ai]
-Row(user_id='2963', product_id=14235, sale_price=15.0, age=29.0, gender='1', 
state='97', country='2')
-Row(user_id='21422', product_id=11203, sale_price=12.0, age=36.0, state='184', 
gender='1', country='5')
-Row(user_id='20592', product_id=8579, sale_price=9.0, age=30.0, state='86', 
gender='1', country='4')
+Row(user_id='2963', product_id=14235, sale_price=15.0, age=12.0, state='1', 
gender='1', country='1')
+Row(user_id='21422', product_id=11203, sale_price=12.0, age=12.0, state='0', 
gender='0', country='0')
+Row(user_id='20592', product_id=8579, sale_price=9.0, age=12.0, state='2', 
gender='1', country='2')
   [END enrichment_with_vertex_ai]'''.splitlines()[1:-1]
   return expected
 
diff --git a/sdks/python/apache_beam/io/requestresponse.py 
b/sdks/python/apache_beam/io/requestresponse.py
index 706bce95f5e..4458aa59c18 100644
--- a/sdks/python/apache_beam/io/requestresponse.py
+++ b/sdks/python/apache_beam/io/requestresponse.py
@@ -33,10 +33,10 @@ from typing import Tuple
 from typing import TypeVar
 from typing import Union
 
+import redis
 from google.api_core.exceptions import TooManyRequests
 
 import apache_beam as beam
-import redis
 from apache_beam import pvalue
 from apache_beam.coders import coders
 from apache_beam.io.components.adaptive_throttler import AdaptiveThrottler
diff --git a/sdks/python/apache_beam/io/requestresponse_it_test.py 
b/sdks/python/apache_beam/io/requestresponse_it_test.py
index bd8c63dea58..712ccc7881d 100644
--- a/sdks/python/apache_beam/io/requestresponse_it_test.py
+++ b/sdks/python/apache_beam/io/requestresponse_it_test.py
@@ -203,7 +203,7 @@ class FakeCallerForCache(Caller[str, str]):
 pass
 
 
-@pytest.mark.uses_redis
+@pytest.mark.uses_testcontainer
 class TestRedisCache(unittest.TestCase):
   def setUp(self) -> None:
 self.retries = 3
diff --git a/sdks/python/apache_beam/io/requestresponse_tests_requirements.txt 
b/sdks/python/apache_beam/io/requestresponse_tests_requirements.txt
deleted file mode 100644
index 1d886970509..000
--- a/sdks/python/apache_beam/io/requestresponse_tests_requirements.txt
+++ /dev/null
@@ -1,18 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  

(beam) branch master updated (e894d8ca7fe -> d4df535bf60)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from e894d8ca7fe Add support to record multiple values at once in Histogram 
metric (#30717)
 add d4df535bf60 [Docs] Update Bigtable enrichment notebook (#30783)

No new revisions were added by this update.

Summary of changes:
 .../beam-ml/bigtable_enrichment_transform.ipynb| 119 +++--
 1 file changed, 89 insertions(+), 30 deletions(-)

(beam) branch master updated: [Docs] Beam website doc for vertex ai enrichment handler (#30692)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4af19ffa8cc [Docs] Beam website doc for vertex ai enrichment handler 
(#30692)
4af19ffa8cc is described below

commit 4af19ffa8cc9039e2c2110686ff8ad043a07f5cc
Author: Ritesh Ghorse 
AuthorDate: Tue Mar 26 16:04:48 2024 -0400

[Docs] Beam website doc for vertex ai enrichment handler (#30692)

* add vetex ai page

* update beam side docs for vertex ai enrichment

* add links to release doc

* update links, correct table formatting

* Update 
website/www/site/content/en/documentation/transforms/python/elementwise/enrichment-bigtable.md

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update 
website/www/site/content/en/documentation/transforms/python/elementwise/enrichment-vertexai.md

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

-

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
---
 .../snippets/transforms/elementwise/enrichment.py  | 67 
 .../transforms/elementwise/enrichment_test.py  | 37 -
 .../python/elementwise/enrichment-bigtable.md  | 62 +++
 .../python/elementwise/enrichment-vertexai.md  | 89 ++
 .../transforms/python/elementwise/enrichment.md| 37 +++--
 .../transforms/python/elementwise/runinference.md  |  2 +-
 .../partials/section-menu/en/documentation.html|  9 ++-
 7 files changed, 275 insertions(+), 28 deletions(-)

diff --git 
a/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment.py
 
b/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment.py
index 59af3858441..acee633b6f6 100644
--- 
a/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment.py
+++ 
b/sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment.py
@@ -49,3 +49,70 @@ def enrichment_with_bigtable():
 | "Enrich W/ BigTable" >> Enrichment(bigtable_handler)
 | "Print" >> beam.Map(print))
   # [END enrichment_with_bigtable]
+
+
+def enrichment_with_vertex_ai():
+  # [START enrichment_with_vertex_ai]
+  import apache_beam as beam
+  from apache_beam.transforms.enrichment import Enrichment
+  from apache_beam.transforms.enrichment_handlers.vertex_ai_feature_store \
+import VertexAIFeatureStoreEnrichmentHandler
+
+  project_id = 'apache-beam-testing'
+  location = 'us-central1'
+  api_endpoint = f"{location}-aiplatform.googleapis.com"
+  data = [
+  beam.Row(user_id='2963', product_id=14235, sale_price=15.0),
+  beam.Row(user_id='21422', product_id=11203, sale_price=12.0),
+  beam.Row(user_id='20592', product_id=8579, sale_price=9.0),
+  ]
+
+  vertex_ai_handler = VertexAIFeatureStoreEnrichmentHandler(
+  project=project_id,
+  location=location,
+  api_endpoint=api_endpoint,
+  feature_store_name="vertexai_enrichment_example",
+  feature_view_name="users",
+  row_key="user_id",
+  )
+  with beam.Pipeline() as p:
+_ = (
+p
+| "Create" >> beam.Create(data)
+| "Enrich W/ Vertex AI" >> Enrichment(vertex_ai_handler)
+| "Print" >> beam.Map(print))
+  # [END enrichment_with_vertex_ai]
+
+
+def enrichment_with_vertex_ai_legacy():
+  # [START enrichment_with_vertex_ai_legacy]
+  import apache_beam as beam
+  from apache_beam.transforms.enrichment import Enrichment
+  from apache_beam.transforms.enrichment_handlers.vertex_ai_feature_store \
+import VertexAIFeatureStoreLegacyEnrichmentHandler
+
+  project_id = 'apache-beam-testing'
+  location = 'us-central1'
+  api_endpoint = f"{location}-aiplatform.googleapis.com"
+  data = [
+  beam.Row(entity_id="movie_01", title='The Shawshank Redemption'),
+  beam.Row(entity_id="movie_02", title="The Shining"),
+  beam.Row(entity_id="movie_04", title='The Dark Knight'),
+  ]
+
+  vertex_ai_handler = VertexAIFeatureStoreLegacyEnrichmentHandler(
+  project=project_id,
+  location=location,
+  api_endpoint=api_endpoint,
+  entity_type_id='movies',
+  feature_store_id="movie_prediction_unique",
+  feature_ids=["title", "genres"],
+  row_key="entity_id",
+  )
+  with beam.Pipeline() as p:
+_ = (
+p
+| "Create" >> beam.Create(data)
+| "Enrich W/ Vertex AI" >> Enrichment(ve

(beam) branch master updated: Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.0 in /sdks (#30745)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4d350cae2a9 Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.0 in 
/sdks (#30745)
4d350cae2a9 is described below

commit 4d350cae2a92b40787304d3c80e37b201bd99cc0
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Mar 26 09:42:51 2024 -0400

Bump github.com/go-sql-driver/mysql from 1.7.1 to 1.8.0 in /sdks (#30745)

Bumps 
[github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql) from 
1.7.1 to 1.8.0.
- [Release notes](https://github.com/go-sql-driver/mysql/releases)
- 
[Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.7.1...v1.8.0)

---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 3 ++-
 sdks/go.sum | 6 --
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 60e998d82d2..4432cb45d01 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -38,7 +38,7 @@ require (
github.com/aws/smithy-go v1.20.1
github.com/docker/go-connections v0.5.0
github.com/dustin/go-humanize v1.0.1
-   github.com/go-sql-driver/mysql v1.7.1
+   github.com/go-sql-driver/mysql v1.8.0
github.com/golang/protobuf v1.5.4 // TODO(danoliveira): Fully replace 
this with google.golang.org/protobuf
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
@@ -75,6 +75,7 @@ require (
 
 require (
dario.cat/mergo v1.0.0 // indirect
+   filippo.io/edwards25519 v1.1.0 // indirect
github.com/Microsoft/hcsshim v0.11.4 // indirect
github.com/apache/arrow/go/v14 v14.0.2 // indirect
github.com/containerd/log v0.1.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index f951e22762e..6b9fbed1d54 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -48,6 +48,8 @@ cloud.google.com/go/storage v1.38.0/go.mod 
h1:tlUADB0mAb9BgYls9lq+8MGkfzOXuLrnHX
 dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
 dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod 
h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
+filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
+filippo.io/edwards25519 v1.1.0/go.mod 
h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
 github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 
h1:bvDV9vkmnHYOMsOr4WLk+Vo07yKIzd94sVoIqshQ4bU=
 github.com/Azure/azure-pipeline-go v0.2.3/go.mod 
h1:x841ezTBIMG6O3lAcl8ATHnsOPVl2bqk7S3ta6S6u4k=
 github.com/Azure/azure-storage-blob-go v0.14.0/go.mod 
h1:SMqIBi+SuiQH32bvyjngEewEeXoPfKMgWlBDaYf6fck=
@@ -196,8 +198,8 @@ github.com/go-logr/stdr v1.2.2/go.mod 
h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre
 github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
 github.com/go-ole/go-ole v1.2.6/go.mod 
h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
 github.com/go-sql-driver/mysql v1.5.0/go.mod 
h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
-github.com/go-sql-driver/mysql v1.7.1 
h1:lUIinVbN1DY0xBg0eMOzmmtGoHwWBbvnWubQUrtU8EI=
-github.com/go-sql-driver/mysql v1.7.1/go.mod 
h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
+github.com/go-sql-driver/mysql v1.8.0 
h1:UtktXaU2Nb64z/pLiGIxY4431SJ4/dR5cjMmlVHgnT4=
+github.com/go-sql-driver/mysql v1.8.0/go.mod 
h1:wEBSXgmK//2ZFJyE+qWnIsVGmvmEKlqwuVSjsCm7DZg=
 github.com/goccy/go-json v0.10.2 
h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
 github.com/goccy/go-json v0.10.2/go.mod 
h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
 github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=

(beam) branch master updated (c8ad120bde8 -> cd4e6cf14ae)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from c8ad120bde8 Fix all tour examples from part 1 (#30478)
 add cd4e6cf14ae Update documentation link in bigtable_enrichment notebook 
(#30740)

No new revisions were added by this update.

Summary of changes:
 .../beam-ml/bigtable_enrichment_transform.ipynb| 487 ++---
 1 file changed, 228 insertions(+), 259 deletions(-)

(beam) branch master updated: [Docs] Add Vertex AI Feature Store enrichment notebook (#30689)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 65344d46700 [Docs] Add Vertex AI Feature Store enrichment notebook 
(#30689)
65344d46700 is described below

commit 65344d467000c960ab406e70b3c059cc315e4c4f
Author: Ritesh Ghorse 
AuthorDate: Mon Mar 25 16:05:00 2024 -0400

[Docs] Add Vertex AI Feature Store enrichment notebook (#30689)

* add vertex ai enrichment notebook

* address review comments, add to readme

* Update examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* update content from review

-

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
---
 examples/notebooks/beam-ml/README.md   |5 +
 .../vertex_ai_feature_store_enrichment.ipynb   | 2502 
 2 files changed, 2507 insertions(+)

diff --git a/examples/notebooks/beam-ml/README.md 
b/examples/notebooks/beam-ml/README.md
index f1c19747fc7..8f723c2c914 100644
--- a/examples/notebooks/beam-ml/README.md
+++ b/examples/notebooks/beam-ml/README.md
@@ -56,6 +56,11 @@ This section contains the following example notebooks.
 * [Use MLTransform to scale 
data](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/data_preprocessing/scale_data.ipynb)
 * [Preprocessing with the Apache Beam DataFrames 
API](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/dataframe_api_preprocessing.ipynb)
 
+### Data enrichment
+
+* [Use Bigtable to enrich 
data](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb)
+* [Use Vertex AI Feature Store for feature 
enrichment](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb)
+
 ### Prediction and inference with pretrained models
 
 * [Apache Beam RunInference for 
PyTorch](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch.ipynb)
diff --git 
a/examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb 
b/examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb
new file mode 100644
index 000..c8ae558a1ba
--- /dev/null
+++ b/examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb
@@ -0,0 +1,2502 @@
+{
+  "cells": [
+{
+  "cell_type": "code",
+  "execution_count": null,
+  "metadata": {
+"cellView": "form",
+"id": "fFjof1NgAJwu"
+  },
+  "outputs": [],
+  "source": [
+"# @title ## Licensed to the Apache Software Foundation (ASF), 
Version 2.0 (the \"License\")\n",
+"\n",
+"# Licensed to the Apache Software Foundation (ASF) under one\n",
+"# or more contributor license agreements. See the NOTICE file\n",
+"# distributed with this work for additional information\n",
+"# regarding copyright ownership. The ASF licenses this file\n",
+"# to you under the Apache License, Version 2.0 (the\n",
+"# \"License\"); you may not use this file except in compliance\n",
+"# with the License. You may obtain a copy of the License at\n",
+"#\n",
+"#   http://www.apache.org/licenses/LICENSE-2.0\n;,
+"#\n",
+"# Unless required by applicable law or agreed to in writing,\n",
+"# software distributed under the License is distributed on an\n",
+"# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+"# KIND, either express or implied. See the License for the\n",
+"# specific language governing permissions and limitations\n",
+"# under the License"
+  ]
+},
+{
+  "cell_type": "markdown",
+  "metadata": {
+"id": "A8xNRyZMW1yK"
+  },
+  "source": [
+"# Use Apache Beam and Vertex AI Feature Store to enrich data\n",
+"\n",
+"\n",
+"  \n",
+"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/vertex_ai_feature_store_enrichment.ipynb\;>https://raw.githubusercontent.com/google/or-tools/mai

(beam) branch master updated: Bump github.com/golang/protobuf from 1.5.3 to 1.5.4 in /sdks (#30698)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new e8e272b89e9 Bump github.com/golang/protobuf from 1.5.3 to 1.5.4 in 
/sdks (#30698)
e8e272b89e9 is described below

commit e8e272b89e9dc30a83aecc61322f75dbe1613fe4
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Mar 21 09:50:28 2024 -0400

Bump github.com/golang/protobuf from 1.5.3 to 1.5.4 in /sdks (#30698)

Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 
1.5.3 to 1.5.4.
- [Release notes](https://github.com/golang/protobuf/releases)
- [Commits](https://github.com/golang/protobuf/compare/v1.5.3...v1.5.4)

---
updated-dependencies:
- dependency-name: github.com/golang/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 26e9bba48cd..2d22bbe77df 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -39,7 +39,7 @@ require (
github.com/docker/go-connections v0.5.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
-   github.com/golang/protobuf v1.5.3 // TODO(danoliveira): Fully replace 
this with google.golang.org/protobuf
+   github.com/golang/protobuf v1.5.4 // TODO(danoliveira): Fully replace 
this with google.golang.org/protobuf
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
github.com/johannesboyne/gofakes3 v0.0.0-20221110173912-32fb85c5aed6
diff --git a/sdks/go.sum b/sdks/go.sum
index d59d0ba2173..92a13cc8429 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -228,8 +228,8 @@ github.com/golang/protobuf v1.4.1/go.mod 
h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD
 github.com/golang/protobuf v1.4.3/go.mod 
h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
 github.com/golang/protobuf v1.5.0/go.mod 
h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
 github.com/golang/protobuf v1.5.2/go.mod 
h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
-github.com/golang/protobuf v1.5.3 
h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg=
-github.com/golang/protobuf v1.5.3/go.mod 
h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
+github.com/golang/protobuf v1.5.4 
h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
+github.com/golang/protobuf v1.5.4/go.mod 
h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
 github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod 
h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
 github.com/golang/snappy v0.0.1/go.mod 
h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
 github.com/golang/snappy v0.0.3/go.mod 
h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=

(beam) branch master updated: Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 in /sdks (#30685)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 81cb194f8d1 Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 in /sdks 
(#30685)
81cb194f8d1 is described below

commit 81cb194f8d12e5aa1d39ec9a10c1f1404c4073f7
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Mar 20 10:07:48 2024 -0400

Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 in /sdks (#30685)

Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.17.0 
to 0.18.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 4 ++--
 sdks/go.sum | 8 
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index a44a90adba7..c5887395a14 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -54,8 +54,8 @@ require (
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
go.mongodb.org/mongo-driver v1.13.1
-   golang.org/x/net v0.21.0
-   golang.org/x/oauth2 v0.17.0
+   golang.org/x/net v0.22.0
+   golang.org/x/oauth2 v0.18.0
golang.org/x/sync v0.6.0
golang.org/x/sys v0.18.0
golang.org/x/text v0.14.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 72b1037f254..eaa713f7d06 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -554,15 +554,15 @@ golang.org/x/net 
v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwY
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod 
h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-2022202133-69e39bad7dc2/go.mod 
h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod 
h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.21.0 h1:AQyQV4dYCvJ7vGmJyKki9+PBdyvhkSd8EIx/qb0AYv4=
-golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
+golang.org/x/net v0.22.0 h1:9sGLhx7iRIHEiX0oAJ3MRZMUCElJgy7Br1nO+AMN3Tc=
+golang.org/x/net v0.22.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod 
h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
 golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.17.0 h1:6m3ZPmLEFdVxKKWnKq4VqZ60gutO35zm+zrAHVmHyDQ=
-golang.org/x/oauth2 v0.17.0/go.mod 
h1:OzPDGQiuQMguemayvdylqddI7qcD9lnSDb+1FiwQ5HA=
+golang.org/x/oauth2 v0.18.0 h1:09qnuIAgzdx1XplqJvW6CQqMCtGZykZWcXzPMPUusvI=
+golang.org/x/oauth2 v0.18.0/go.mod 
h1:Wf7knwG0MPoWIMMBgFlEaSUDaKskp0dCfrlJRJXbBi8=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

(beam) branch master updated: [Python] Check feature store existence at pipeline construction time (#30668)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 50f33cd786d [Python] Check feature store existence at pipeline 
construction time (#30668)
50f33cd786d is described below

commit 50f33cd786dc63463688315a1c73b1cf4ef18807
Author: Ritesh Ghorse 
AuthorDate: Tue Mar 19 10:01:35 2024 -0400

[Python] Check feature store existence at pipeline construction time 
(#30668)

* check feature store existence at construction time

* postcommit
---
 .github/trigger_files/beam_PostCommit_Python.json  |  0
 .../enrichment_handlers/vertex_ai_feature_store.py | 49 --
 .../vertex_ai_feature_store_it_test.py | 45 ++--
 3 files changed, 70 insertions(+), 24 deletions(-)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
deleted file mode 100644
index e69de29bb2d..000
diff --git 
a/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store.py
 
b/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store.py
index b135739ef59..753b04e1793 100644
--- 
a/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store.py
+++ 
b/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store.py
@@ -110,6 +110,32 @@ class 
VertexAIFeatureStoreEnrichmentHandler(EnrichmentSourceHandler[beam.Row,
 else:
   self.kwargs['client_options'] = {"api_endpoint": self.api_endpoint}
 
+# check if the feature store exists
+try:
+  admin_client = aiplatform.gapic.FeatureOnlineStoreAdminServiceClient(
+  **self.kwargs)
+except Exception:
+  _LOGGER.warning(
+  'Due to insufficient admin permission, could not verify '
+  'the existence of feature store. If the `exception_level` '
+  'is set to WARN then make sure the feature store exists '
+  'otherwise the data enrichment will not happen without '
+  'throwing an error.')
+else:
+  location_path = admin_client.common_location_path(
+  project=self.project, location=self.location)
+  feature_store_path = admin_client.feature_online_store_path(
+  project=self.project,
+  location=self.location,
+  feature_online_store=self.feature_store_name)
+  feature_store = admin_client.get_feature_online_store(
+  name=feature_store_path)
+
+  if not feature_store:
+raise NotFound(
+'Vertex AI Feature Store %s does not exists in %s' %
+(self.feature_store_name, location_path))
+
   def __enter__(self):
 """Connect with the Vertex AI Feature Store."""
 self.client = aiplatform.gapic.FeatureOnlineStoreServiceClient(
@@ -228,26 +254,25 @@ class 
VertexAIFeatureStoreLegacyEnrichmentHandler(EnrichmentSourceHandler):
 else:
   self.kwargs['client_options'] = {"api_endpoint": self.api_endpoint}
 
-  def __enter__(self):
-"""Connect with the Vertex AI Feature Store (Legacy)."""
+# checks if feature store exists
 try:
-  # checks if feature store exists
   _ = aiplatform.Featurestore(
   featurestore_name=self.feature_store_id,
   project=self.project,
   location=self.location,
   credentials=self.kwargs.get('credentials'),
   )
-  self.client = aiplatform.gapic.FeaturestoreOnlineServingServiceClient(
-  **self.kwargs)
-  self.entity_type_path = self.client.entity_type_path(
-  self.project,
-  self.location,
-  self.feature_store_id,
-  self.entity_type_id)
 except NotFound:
-  raise ValueError(
-  'Vertex AI Feature Store %s does not exist' % self.feature_store_id)
+  raise NotFound(
+  'Vertex AI Feature Store (Legacy) %s does not exist' %
+  self.feature_store_id)
+
+  def __enter__(self):
+"""Connect with the Vertex AI Feature Store (Legacy)."""
+self.client = aiplatform.gapic.FeaturestoreOnlineServingServiceClient(
+**self.kwargs)
+self.entity_type_path = self.client.entity_type_path(
+self.project, self.location, self.feature_store_id, 
self.entity_type_id)
 
   def __call__(self, request: beam.Row, *args, **kwargs):
 """Fetches feature value for an entity-id from
diff --git 
a/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store_it_test.py
 
b/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store_it_test.py
index d4224be060e..066553decf2 100644
--- 
a/sdks/python/apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store_it_test.py
+++ 
b/sdks/python/apache_

(beam-starter-go) 01/01: Merge pull request #31 from apache/dependabot/go_modules/google.golang.org/protobuf-1.33.0

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git

commit 1a22558511d86b22e49ce30c4f6eebf21686438c
Merge: 45f8ea3 0bc79e8
Author: Ritesh Ghorse 
AuthorDate: Wed Mar 13 21:41:42 2024 -0400

Merge pull request #31 from 
apache/dependabot/go_modules/google.golang.org/protobuf-1.33.0

Bump google.golang.org/protobuf from 1.31.0 to 1.33.0

 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

(beam-starter-go) branch main updated (45f8ea3 -> 1a22558)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git


from 45f8ea3  Update README.md to point to beam site quickstart guide.
 add 0bc79e8  Bump google.golang.org/protobuf from 1.31.0 to 1.33.0
 new 1a22558  Merge pull request #31 from 
apache/dependabot/go_modules/google.golang.org/protobuf-1.33.0

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

(beam) branch master updated (289bbc45c56 -> 4ad189db265)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 289bbc45c56 [Python] Pin keras version for tensorflow inference test 
(#30614)
 add 4ad189db265 Bump github.com/nats-io/nats-server/v2 from 2.10.10 to 
2.10.12 in /sdks (#30620)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 12 ++--
 sdks/go.sum | 24 
 2 files changed, 18 insertions(+), 18 deletions(-)

(beam) branch master updated: [Python] Pin keras version for tensorflow inference test (#30614)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 289bbc45c56 [Python] Pin keras version for tensorflow inference test 
(#30614)
289bbc45c56 is described below

commit 289bbc45c56a0738e6e0f3c9ba17873d0d1a234c
Author: Ritesh Ghorse 
AuthorDate: Tue Mar 12 21:58:19 2024 -0400

[Python] Pin keras version for tensorflow inference test (#30614)
---
 .github/trigger_files/beam_PostCommit_Python.json  | 0
 sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt | 1 +
 2 files changed, 1 insertion(+)

diff --git a/.github/trigger_files/beam_PostCommit_Python.json 
b/.github/trigger_files/beam_PostCommit_Python.json
new file mode 100644
index 000..e69de29bb2d
diff --git 
a/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt 
b/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt
index 8195059a6c1..aebddf61226 100644
--- a/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt
+++ b/sdks/python/apache_beam/ml/inference/tensorflow_tests_requirements.txt
@@ -15,6 +15,7 @@
 # limitations under the License.
 #
 
+keras<3.0.0
 tensorflow>=2.12.0
 tensorflow_hub>=0.10.0
 Pillow>=9.0.0

(beam) branch master updated: [Python] Log dependencies installed in submission environment (#28564)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 7497495bcf7 [Python] Log dependencies installed in submission 
environment (#28564)
7497495bcf7 is described below

commit 7497495bcf75a66fd17b639c3d0180c8f3ba1f08
Author: Ritesh Ghorse 
AuthorDate: Thu Mar 7 15:14:40 2024 -0500

[Python] Log dependencies installed in submission environment (#28564)

* log runtime dependencies

* log submission env dependencies

* rm spare line

* update tests for staged files

* update equals test to ignore artifacts

* handle unit tests, refactor

* unit test sub env staging, convert to string

* change Log to Printf

* change log level to warning

* try env

* add artifact_service method

* correct urn

* fix artifact comparison, file reader

* add mock for python sdk dependencies and update artifact service method

* fix lint

* use magic mock instead of mocking entire function

* update dataflow runner test

* Update sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py

* use debug option to disable

* remove tmp directory mock
---
 .../runners/dataflow/dataflow_runner_test.py   | 23 --
 .../runners/portability/expansion_service.py   |  7 ++
 .../runners/portability/portable_runner_test.py| 41 --
 .../apache_beam/runners/portability/stager.py  | 54 -
 .../apache_beam/runners/portability/stager_test.py | 91 +-
 .../apache_beam/transforms/environments_test.py| 11 +++
 sdks/python/container/boot.go  | 20 -
 7 files changed, 209 insertions(+), 38 deletions(-)

diff --git a/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py 
b/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
index b20b5f293c0..b5568305ce6 100644
--- a/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
+++ b/sdks/python/apache_beam/runners/dataflow/dataflow_runner_test.py
@@ -16,7 +16,6 @@
 #
 
 """Unit tests for the DataflowRunner class."""
-
 # pytype: skip-file
 
 import unittest
@@ -206,12 +205,15 @@ class DataflowRunnerTest(unittest.TestCase, 
ExtraAssertionsMixin):
 self.default_properties.append('--experiments=beam_fn_api')
 self.default_properties.append('--worker_harness_container_image=LEGACY')
 remote_runner = DataflowRunner()
-with Pipeline(remote_runner,
-  options=PipelineOptions(self.default_properties)) as p:
+options = PipelineOptions(self.default_properties)
+options.view_as(DebugOptions).add_experiment(
+'disable_logging_submission_environment')
+with Pipeline(remote_runner, options=options) as p:
   (  # pylint: disable=expression-not-assigned
   p | ptransform.Create([1, 2, 3])
   | 'Do' >> ptransform.FlatMap(lambda x: [(x, x)])
   | ptransform.GroupByKey())
+
 self.assertEqual(
 list(remote_runner.proto_pipeline.components.environments.values()),
 [
@@ -219,19 +221,24 @@ class DataflowRunnerTest(unittest.TestCase, 
ExtraAssertionsMixin):
 urn=common_urns.environments.DOCKER.urn,
 payload=beam_runner_api_pb2.DockerPayload(
 container_image='LEGACY').SerializeToString(),
-capabilities=environments.python_sdk_docker_capabilities())
+capabilities=environments.python_sdk_docker_capabilities(),
+dependencies=environments.python_sdk_dependencies(
+options=options))
 ])
 
   def test_environment_override_translation_sdk_container_image(self):
 self.default_properties.append('--experiments=beam_fn_api')
 self.default_properties.append('--sdk_container_image=FOO')
 remote_runner = DataflowRunner()
-with Pipeline(remote_runner,
-  options=PipelineOptions(self.default_properties)) as p:
+options = PipelineOptions(self.default_properties)
+options.view_as(DebugOptions).add_experiment(
+'disable_logging_submission_environment')
+with Pipeline(remote_runner, options=options) as p:
   (  # pylint: disable=expression-not-assigned
   p | ptransform.Create([1, 2, 3])
   | 'Do' >> ptransform.FlatMap(lambda x: [(x, x)])
   | ptransform.GroupByKey())
+
 self.assertEqual(
 list(remote_runner.proto_pipeline.components.environments.values()),
 [
@@ -239,7 +246,9 @@ class DataflowRunnerTest(unittest.TestCase, 
ExtraAssertionsMixin):
 urn=common_urns.environments.DOCKER.urn,
 payload=beam_runner_api_pb2.DockerPayload(
   

(beam) branch master updated: [Python] Vertex AI Feature Store enrichment handler (#30388)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 314bd722eea [Python] Vertex AI Feature Store enrichment handler 
(#30388)
314bd722eea is described below

commit 314bd722eeacd27126070bafb4de2fb5fa8836e6
Author: Ritesh Ghorse 
AuthorDate: Wed Mar 6 13:23:05 2024 -0500

[Python] Vertex AI Feature Store enrichment handler (#30388)

* vertex ai v1

* vertex ai base

* bigtable and legacy support

* correct pydoc, project name

* clear exception message, refactor, unittests

* add main

* drop postcommit file
---
 CHANGES.md |   1 +
 sdks/python/apache_beam/transforms/enrichment.py   |   6 +-
 .../transforms/enrichment_handlers/bigtable.py |  20 +-
 .../transforms/enrichment_handlers/utils.py|  38 +++
 .../enrichment_handlers/vertex_ai_feature_store.py | 306 +
 .../vertex_ai_feature_store_it_test.py | 287 +++
 .../vertex_ai_feature_store_test.py|  58 
 .../vertex_ai_feature_store_tests_requirement.txt  |  18 ++
 sdks/python/test-suites/direct/common.gradle   |   5 +-
 9 files changed, 717 insertions(+), 22 deletions(-)

diff --git a/CHANGES.md b/CHANGES.md
index 57d729ac5f5..1b0e6ec6f0d 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -77,6 +77,7 @@
 * Redis cache support added to RequestResponseIO and Enrichment transform 
(Python) ([#30307](https://github.com/apache/beam/pull/30307))
 * Merged sdks/java/fn-execution and runners/core-construction-java into the 
main SDK. These artifacts were never meant for users, but noting
   that they no longer exist. These are steps to bring portability into the 
core SDK alongside all other core functionality.
+* Added Vertex AI Feature Store handler for Enrichment transform (Python) 
([#30388](https://github.com/apache/beam/pull/30388))
 
 ## Breaking Changes
 
diff --git a/sdks/python/apache_beam/transforms/enrichment.py 
b/sdks/python/apache_beam/transforms/enrichment.py
index 93344835e93..ddfbba5337f 100644
--- a/sdks/python/apache_beam/transforms/enrichment.py
+++ b/sdks/python/apache_beam/transforms/enrichment.py
@@ -161,8 +161,10 @@ class Enrichment(beam.PTransform[beam.PCollection[InputT],
 throttler=self._throttler)
 
 # EnrichmentSourceHandler returns a tuple of (request,response).
-return fetched_data | beam.Map(
-lambda x: self._join_fn(x[0]._asdict(), x[1]._asdict()))
+return (
+fetched_data
+| "enrichment_join" >>
+beam.Map(lambda x: self._join_fn(x[0]._asdict(), x[1]._asdict(
 
   def with_redis_cache(
   self,
diff --git a/sdks/python/apache_beam/transforms/enrichment_handlers/bigtable.py 
b/sdks/python/apache_beam/transforms/enrichment_handlers/bigtable.py
index 943000a9f6b..af35f91a42f 100644
--- a/sdks/python/apache_beam/transforms/enrichment_handlers/bigtable.py
+++ b/sdks/python/apache_beam/transforms/enrichment_handlers/bigtable.py
@@ -15,7 +15,6 @@
 # limitations under the License.
 #
 import logging
-from enum import Enum
 from typing import Any
 from typing import Dict
 from typing import Optional
@@ -28,30 +27,15 @@ from google.cloud.bigtable.row_filters import RowFilter
 
 import apache_beam as beam
 from apache_beam.transforms.enrichment import EnrichmentSourceHandler
+from apache_beam.transforms.enrichment_handlers.utils import ExceptionLevel
 
 __all__ = [
 'BigTableEnrichmentHandler',
-'ExceptionLevel',
 ]
 
 _LOGGER = logging.getLogger(__name__)
 
 
-class ExceptionLevel(Enum):
-  """ExceptionLevel defines the exception level options to either
-  log a warning, or raise an exception, or do nothing when a BigTable query
-  returns an empty row.
-
-  Members:
-- RAISE: Raise the exception.
-- WARN: Log a warning for exception without raising it.
-- QUIET: Neither log nor raise the exception.
-  """
-  RAISE = 0
-  WARN = 1
-  QUIET = 2
-
-
 class BigTableEnrichmentHandler(EnrichmentSourceHandler[beam.Row, beam.Row]):
   """A handler for :class:`apache_beam.transforms.enrichment.Enrichment`
   transform to interact with GCP BigTable.
@@ -70,7 +54,7 @@ class 
BigTableEnrichmentHandler(EnrichmentSourceHandler[beam.Row, beam.Row]):
 encoding (str): encoding type to convert the string to bytes and vice-versa
   from BigTable. Default is `utf-8`.
 exception_level: a `enum.Enum` value from
-  ``apache_beam.transforms.enrichment_handlers.bigtable.ExceptionLevel``
+  ``apache_beam.transforms.enrichment_handlers.utils.ExceptionLevel``
   to set the level when an empty row is returned from the BigTable query.
   Defaults to ``ExceptionLevel.WARN``.
 include_timestamp (bool): If enabled, the times

(beam) branch master updated: Fix website precommit (#30540)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 1c7fb4bc8a0 Fix website precommit (#30540)
1c7fb4bc8a0 is described below

commit 1c7fb4bc8a01e460c55164aa9f4ddb8ddacda52a
Author: Danny McCormick 
AuthorDate: Wed Mar 6 11:34:55 2024 -0500

Fix website precommit (#30540)
---
 website/www/site/content/en/documentation/sdks/yaml.md | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/website/www/site/content/en/documentation/sdks/yaml.md 
b/website/www/site/content/en/documentation/sdks/yaml.md
index d9c3b8db51b..8d4c6fb94b5 100644
--- a/website/www/site/content/en/documentation/sdks/yaml.md
+++ b/website/www/site/content/en/documentation/sdks/yaml.md
@@ -615,7 +615,12 @@ pipeline:
 topic: myPubSubTopic
 format: ...
 schema: ...
-...
+  windowing:
+type: fixed
+size: 60s
+- type: Sql
+  config:
+query: "select col1, count(*) as c from PCOLLECTION"
 - type: WriteToPubSub
   config:
 topic: anotherPubSubTopic

(beam) branch master updated: Bump cloud.google.com/go/pubsub from 1.34.0 to 1.36.2 in /sdks (#30465)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 6f5b2197751 Bump cloud.google.com/go/pubsub from 1.34.0 to 1.36.2 in 
/sdks (#30465)
6f5b2197751 is described below

commit 6f5b219775131a0922019eb11ce68f4366509fc2
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Mar 6 10:33:16 2024 -0500

Bump cloud.google.com/go/pubsub from 1.34.0 to 1.36.2 in /sdks (#30465)

Bumps 
[cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) 
from 1.34.0 to 1.36.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- 
[Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- 
[Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.34.0...pubsub/v1.36.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/pubsub
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod |  63 ++--
 sdks/go.sum | 136 
 2 files changed, 94 insertions(+), 105 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 67bb0beec63..38f3b0bbfd9 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -23,13 +23,13 @@ module github.com/apache/beam/sdks/v2
 go 1.20
 
 require (
-   cloud.google.com/go/bigquery v1.58.0
+   cloud.google.com/go/bigquery v1.59.1
cloud.google.com/go/bigtable v1.21.0
cloud.google.com/go/datastore v1.15.0
cloud.google.com/go/profiler v0.4.0
-   cloud.google.com/go/pubsub v1.34.0
+   cloud.google.com/go/pubsub v1.36.2
cloud.google.com/go/spanner v1.56.0
-   cloud.google.com/go/storage v1.36.0
+   cloud.google.com/go/storage v1.37.0
github.com/aws/aws-sdk-go-v2 v1.25.2
github.com/aws/aws-sdk-go-v2/config v1.27.4
github.com/aws/aws-sdk-go-v2/credentials v1.17.4
@@ -54,13 +54,13 @@ require (
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
go.mongodb.org/mongo-driver v1.13.1
-   golang.org/x/net v0.20.0
-   golang.org/x/oauth2 v0.16.0
+   golang.org/x/net v0.21.0
+   golang.org/x/oauth2 v0.17.0
golang.org/x/sync v0.6.0
-   golang.org/x/sys v0.16.0
+   golang.org/x/sys v0.17.0
golang.org/x/text v0.14.0
-   google.golang.org/api v0.157.0
-   google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80
+   google.golang.org/api v0.166.0
+   google.golang.org/genproto v0.0.0-20240213162025-012b6fc9bca9
google.golang.org/grpc v1.62.0
google.golang.org/protobuf v1.32.0
gopkg.in/retry.v1 v1.0.3
@@ -70,17 +70,18 @@ require (
 
 require (
github.com/fsouza/fake-gcs-server v1.47.7
-   golang.org/x/exp v0.0.0-20230807204917-050eac23e9de
+   golang.org/x/exp v0.0.0-20231006140011-7918f672742d
 )
 
 require (
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/hcsshim v0.11.4 // indirect
+   github.com/apache/arrow/go/v14 v14.0.2 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.5.0 // indirect
github.com/frankban/quicktest v1.14.0 // indirect
-   github.com/go-logr/logr v1.3.0 // indirect
+   github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/json-iterator/go v1.1.12 // indirect
@@ -101,27 +102,26 @@ require (
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/yusufpapurcu/wmi v1.2.3 // indirect
-   
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 
v0.46.1 // indirect
-   go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 
// indirect
-   go.opentelemetry.io/otel v1.21.0 // indirect
-   go.opentelemetry.io/otel/metric v1.21.0 // indirect
+   go.einride.tech/aip v0.66.0 // indirect
+   
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc 
v0.48.0 // indirect
+   go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.48.0 
// indirect
+   go.opentelemetry.io/otel v1.23.0 // indirect
+   go.opentelemetry.io/otel/metric v1.23.0 // indirect
go.opentelemetry.io/otel/sdk v1.21.0 // indirect
-   go.opentelemetry.io/otel/trace v1.21.0 // indirect
+   go.opentelemetry.i

(beam) branch master updated: [Dataflow] ActiveWorkRefresh (#30390)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new aebb34a94c3 [Dataflow] ActiveWorkRefresh (#30390)
aebb34a94c3 is described below

commit aebb34a94c382a208481d9317da6230122790713
Author: martin trieu 
AuthorDate: Thu Feb 29 12:47:11 2024 -0800

[Dataflow] ActiveWorkRefresh (#30390)

* factor out async work management

* add tests

* remove sleep from test

* add sleep in tests
---
 .../dataflow/worker/StreamingDataflowWorker.java   |  71 ++
 .../windmill/work/refresh/ActiveWorkRefresher.java | 107 +
 .../work/refresh/ActiveWorkRefreshers.java |  50 
 .../refresh/DispatchedActiveWorkRefresher.java |  68 ++
 .../refresh/DispatchedActiveWorkRefresherTest.java | 256 +
 5 files changed, 501 insertions(+), 51 deletions(-)

diff --git 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
index 639100b238f..d726e2a1486 100644
--- 
a/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
+++ 
b/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java
@@ -118,6 +118,8 @@ import 
org.apache.beam.runners.dataflow.worker.windmill.client.grpc.GrpcWindmill
 import 
org.apache.beam.runners.dataflow.worker.windmill.client.grpc.GrpcWindmillStreamFactory;
 import 
org.apache.beam.runners.dataflow.worker.windmill.state.WindmillStateCache;
 import 
org.apache.beam.runners.dataflow.worker.windmill.state.WindmillStateReader;
+import 
org.apache.beam.runners.dataflow.worker.windmill.work.refresh.ActiveWorkRefresher;
+import 
org.apache.beam.runners.dataflow.worker.windmill.work.refresh.ActiveWorkRefreshers;
 import org.apache.beam.sdk.coders.Coder;
 import org.apache.beam.sdk.coders.KvCoder;
 import org.apache.beam.sdk.extensions.gcp.util.Transport;
@@ -293,6 +295,7 @@ public class StreamingDataflowWorker {
   private int maxWorkItemCommitBytes = Integer.MAX_VALUE;
 
   private final DataflowExecutionStateSampler sampler = 
DataflowExecutionStateSampler.instance();
+  private final ActiveWorkRefresher activeWorkRefresher;
 
   private StreamingDataflowWorker(
   WindmillServerStub windmillServer,
@@ -441,6 +444,20 @@ public class StreamingDataflowWorker {
 
 this.mapTaskToNetwork = mapTaskToBaseNetwork;
 
+int stuckCommitDurationMillis =
+windmillServiceEnabled && options.getStuckCommitDurationMillis() > 0
+? options.getStuckCommitDurationMillis()
+: 0;
+this.activeWorkRefresher =
+ActiveWorkRefreshers.createDispatchedActiveWorkRefresher(
+clock,
+options.getActiveWorkRefreshPeriodMillis(),
+stuckCommitDurationMillis,
+() -> Collections.unmodifiableCollection(computationMap.values()),
+sampler,
+metricTrackingWindmillServer::refreshActiveWork,
+executorSupplier.apply("RefreshWork"));
+
 LOG.debug("windmillServiceEnabled: {}", windmillServiceEnabled);
 LOG.debug("WindmillServiceEndpoint: {}", 
options.getWindmillServiceEndpoint());
 LOG.debug("WindmillServicePort: {}", options.getWindmillServicePort());
@@ -722,30 +739,7 @@ public class StreamingDataflowWorker {
   scheduledExecutors.add(workerMessageTimer);
 }
 
-ScheduledExecutorService refreshWorkTimer = 
executorSupplier.apply("RefreshWork");
-if (options.getActiveWorkRefreshPeriodMillis() > 0) {
-  refreshWorkTimer.scheduleWithFixedDelay(
-  new Runnable() {
-@Override
-public void run() {
-  try {
-refreshActiveWork();
-  } catch (RuntimeException e) {
-LOG.warn("Failed to refresh active work: ", e);
-  }
-}
-  },
-  options.getActiveWorkRefreshPeriodMillis(),
-  options.getActiveWorkRefreshPeriodMillis(),
-  TimeUnit.MILLISECONDS);
-  scheduledExecutors.add(refreshWorkTimer);
-}
-if (windmillServiceEnabled && options.getStuckCommitDurationMillis() > 0) {
-  int periodMillis = Math.max(options.getStuckCommitDurationMillis() / 10, 
100);
-  refreshWorkTimer.scheduleWithFixedDelay(
-  this::invalidateStuckCommits, periodMillis, periodMillis, 
TimeUnit.MILLISECONDS);
-  scheduledExecutors.add(refreshWorkTimer);
-}
+activeWorkRefresher.start();
 
 if (options.getPeriodicStatusPageOutputDirectory() != null) {
  

(beam) branch master updated: Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py39 (#30373)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4ea3898e589 Bump cryptography from 42.0.3 to 42.0.4 in 
/sdks/python/container/py39 (#30373)
4ea3898e589 is described below

commit 4ea3898e589ebc3f8960b1d5bb60888f144d23a5
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Feb 29 10:57:11 2024 -0500

Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py39 
(#30373)

Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.3 to 
42.0.4.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/42.0.3...42.0.4)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/python/container/py39/base_image_requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/python/container/py39/base_image_requirements.txt 
b/sdks/python/container/py39/base_image_requirements.txt
index bfe9a74e145..2647cb6 100644
--- a/sdks/python/container/py39/base_image_requirements.txt
+++ b/sdks/python/container/py39/base_image_requirements.txt
@@ -32,7 +32,7 @@ charset-normalizer==3.3.2
 click==8.1.7
 cloudpickle==2.2.1
 crcmod==1.7
-cryptography==42.0.3
+cryptography==42.0.4
 Cython==0.29.37
 Deprecated==1.2.14
 deprecation==2.1.0

(beam) branch master updated: [Python] Add doc comment for WriteToText.skip_if_empty (#30409)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 2a84a20236b [Python] Add doc comment for WriteToText.skip_if_empty 
(#30409)
2a84a20236b is described below

commit 2a84a20236b6b6145cbd6e961ed0e1848c360100
Author: Ritesh Ghorse 
AuthorDate: Tue Feb 27 09:27:27 2024 -0500

[Python] Add doc comment for WriteToText.skip_if_empty (#30409)

* enable skip_if_empty

* add doc comment
---
 sdks/python/apache_beam/io/textio.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sdks/python/apache_beam/io/textio.py 
b/sdks/python/apache_beam/io/textio.py
index 0e7ea8ff243..454fe4d69de 100644
--- a/sdks/python/apache_beam/io/textio.py
+++ b/sdks/python/apache_beam/io/textio.py
@@ -871,6 +871,8 @@ class WriteToText(PTransform):
 to exceed this value.  This also tracks the uncompressed,
 not compressed, size of the shard.
   skip_if_empty: Don't write any shards if the PCollection is empty.
+In case of an empty PCollection, this will still delete existing
+files having same file path and not create new ones.
 """
 
 self._sink = _TextSink(

(beam) branch master updated (6406cfe3c9b -> 2ae4a28e6fc)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 6406cfe3c9b Remove some uses of ClassLoadingStrategy.Default.INJECTION 
(#30367)
 add 2ae4a28e6fc Force downgrade mpmath to avoid test breakages (#30418)

No new revisions were added by this update.

Summary of changes:
 sdks/python/tox.ini | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

(beam) branch master updated: Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.4 in /sdks (#30415)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new d5a2e3b3c9b Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 
1.27.4 in /sdks (#30415)
d5a2e3b3c9b is described below

commit d5a2e3b3c9bc12a6b0ae431bdc499925d37400dc
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Feb 26 09:59:52 2024 -0500

Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.4 in /sdks 
(#30415)

Bumps 
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) 
from 1.26.2 to 1.27.4.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.26.2...config/v1.27.4)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 26 +-
 sdks/go.sum | 52 ++--
 2 files changed, 39 insertions(+), 39 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index df02e9a9061..a58f77dc4eb 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,12 +30,12 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.56.0
cloud.google.com/go/storage v1.36.0
-   github.com/aws/aws-sdk-go-v2 v1.24.1
-   github.com/aws/aws-sdk-go-v2/config v1.26.2
-   github.com/aws/aws-sdk-go-v2/credentials v1.16.16
+   github.com/aws/aws-sdk-go-v2 v1.25.2
+   github.com/aws/aws-sdk-go-v2/config v1.27.4
+   github.com/aws/aws-sdk-go-v2/credentials v1.17.4
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.8
github.com/aws/aws-sdk-go-v2/service/s3 v1.42.2
-   github.com/aws/smithy-go v1.19.0
+   github.com/aws/smithy-go v1.20.1
github.com/docker/go-connections v0.5.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
@@ -124,18 +124,18 @@ require (
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.1 // indirect
-   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 // indirect
+   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.15.2 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.2 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.2 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.3 // indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 
// indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.1 
// indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.2 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.18.7 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.26.7 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.20.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.23.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.28.1 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 50b6d2039fc..553dab3234a 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -81,39 +81,39 @@ github.com/aws/aws-sdk-go v1.30.19/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZve
 github.com/aws/aws-sdk-go v1.34.0 
h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
 github.com/aws/aws-sdk-go v1.34.0/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
 github.com/aws/aws-sdk-go-v2 v1.7.1/go.mod 
h1:L5LuPC1ZgDr2xQS7AmIec/Jlc7O/Y1u2KxJyNVab250=
-github.com/aws/aws-sdk-go-v2 v1.24.1 
h1:xAojnj+ktS95YZlDf0zxWBkbFtymPeDP+rvUQIH3uAU=
-github.com/aws/aws-sdk-go-v2 v1.24.

(beam) branch master updated: [Python] Redis cache support for enrichment transform (#30307)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 801252ab613 [Python] Redis cache support for enrichment transform 
(#30307)
801252ab613 is described below

commit 801252ab613dd6601f23918cc46a28cded4b0d98
Author: Ritesh Ghorse 
AuthorDate: Tue Feb 20 12:22:16 2024 -0500

[Python] Redis cache support for enrichment transform (#30307)

* redis cache support for enrichment transform

* update rrio test

* convert rrio tests to execute pipeline, tagged outputs, cache_request_key

* fix rrio exception test, add to changes

* get_cache_key, doc comments, changed __all__

* fix pydoc

* use pool manager

* revert test

* use property() for setter
---
 CHANGES.md |   2 +-
 sdks/python/apache_beam/io/requestresponse.py  | 595 +
 .../apache_beam/io/requestresponse_it_test.py  | 192 +--
 sdks/python/apache_beam/io/requestresponse_test.py |   3 +-
 .../io/requestresponse_tests_requirements.txt  |  18 +
 sdks/python/apache_beam/transforms/enrichment.py   |  85 ++-
 .../transforms/enrichment_handlers/bigtable.py |  10 +-
 .../enrichment_handlers/bigtable_it_test.py|  94 +++-
 .../apache_beam/transforms/enrichment_it_test.py   |   2 +-
 sdks/python/pytest.ini |   2 +
 sdks/python/scripts/generate_pydoc.sh  |   6 +-
 sdks/python/test-suites/dataflow/common.gradle |  30 ++
 sdks/python/test-suites/direct/common.gradle   |  28 +
 13 files changed, 909 insertions(+), 158 deletions(-)

diff --git a/CHANGES.md b/CHANGES.md
index 7a460962a33..ea59c3e964a 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -65,8 +65,8 @@
 
 ## New Features / Improvements
 
-* [Enrichment Transform](https://s.apache.org/enrichment-transform) along with 
GCP BigTable handler added to Python SDK 
([#30001](https://github.com/apache/beam/pull/30001)).
 * Allow writing clustered and not time partitioned BigQuery tables (Java) 
([#30094](https://github.com/apache/beam/pull/30094)).
+* Redis cache support added to RequestResponseIO and Enrichment transform 
(Python) ([#30307](https://github.com/apache/beam/pull/30307))
 * Merged sdks/java/fn-execution and runners/core-construction-java into the 
main SDK. These artifacts were never meant for users, but noting
   that they no longer exist. These are steps to bring portability into the 
core SDK alongside all other core functionality.
 
diff --git a/sdks/python/apache_beam/io/requestresponse.py 
b/sdks/python/apache_beam/io/requestresponse.py
index 63ec7061d3e..706bce95f5e 100644
--- a/sdks/python/apache_beam/io/requestresponse.py
+++ b/sdks/python/apache_beam/io/requestresponse.py
@@ -19,16 +19,26 @@
 import abc
 import concurrent.futures
 import contextlib
+import enum
+import json
 import logging
 import sys
 import time
+from datetime import timedelta
+from typing import Any
+from typing import Dict
 from typing import Generic
 from typing import Optional
+from typing import Tuple
 from typing import TypeVar
+from typing import Union
 
 from google.api_core.exceptions import TooManyRequests
 
 import apache_beam as beam
+import redis
+from apache_beam import pvalue
+from apache_beam.coders import coders
 from apache_beam.io.components.adaptive_throttler import AdaptiveThrottler
 from apache_beam.metrics import Metrics
 from apache_beam.ml.inference.vertex_ai_inference import MSEC_TO_SEC
@@ -37,10 +47,24 @@ from apache_beam.utils import retry
 RequestT = TypeVar('RequestT')
 ResponseT = TypeVar('ResponseT')
 
-DEFAULT_TIMEOUT_SECS = 30  # seconds
+# DEFAULT_TIMEOUT_SECS represents the time interval for completing the request
+# with external source.
+DEFAULT_TIMEOUT_SECS = 30
+
+# DEFAULT_CACHE_ENTRY_TTL_SEC represents the total time-to-live
+# for cache record.
+DEFAULT_CACHE_ENTRY_TTL_SEC = 24 * 60 * 60
 
 _LOGGER = logging.getLogger(__name__)
 
+__all__ = [
+'RequestResponseIO',
+'ExponentialBackOffRepeater',
+'DefaultThrottler',
+'NoOpsRepeater',
+'RedisCache',
+]
+
 
 class UserCodeExecutionException(Exception):
   """Base class for errors related to calling Web APIs."""
@@ -90,6 +114,7 @@ class Caller(contextlib.AbstractContextManager,
  abc.ABC,
  Generic[RequestT, ResponseT]):
   """Interface for user custom code intended for API calls.
+
   For setup and teardown of clients when applicable, implement the
   ``__enter__`` and ``__exit__`` methods respectively."""
   @abc.abstractmethod
@@ -107,16 +132,27 @@ class Caller(contextlib.AbstractContextManager,
   def __exit__(self, exc_type, exc_val, exc_tb):
 return None
 
+  def get_cache_key(self, request: RequestT) -> str:
+"

(beam) branch master updated: [Docs] Add example notebook for enrichment transform with bigtable (#30315)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 6377cbe20cc [Docs] Add example notebook for enrichment transform with 
bigtable (#30315)
6377cbe20cc is described below

commit 6377cbe20cc387d7986709014efc4cedfb2099f3
Author: Ritesh Ghorse 
AuthorDate: Wed Feb 14 18:50:47 2024 -0500

[Docs] Add example notebook for enrichment transform with bigtable (#30315)

* bigtable enrichment notebook

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* Update examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

* changes from the review

* Apply suggestions from code review

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>

-

Co-authored-by: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
---
 .../beam-ml/bigtable_enrichment_transform.ipynb| 854 +
 1 file changed, 854 insertions(+)

diff --git a/examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb 
b/examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb
new file mode 100644
index 000..455b5cccd5d
--- /dev/null
+++ b/examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb
@@ -0,0 +1,854 @@
+{
+  "cells": [
+{
+  "cell_type": "code",
+  "execution_count": null,
+  "metadata": {
+"id": "fFjof1NgAJwu",
+"cellView": "form"
+  },
+  "outputs": [],
+  "source": [
+"# @title ## Licensed to the Apache Software Foundation (ASF), 
Version 2.0 (the \"License\")\n",
+"\n",
+"# Licensed to the Apache Software Foundation (ASF) under one\n",
+"# or more contributor license agreements. See the NOTICE file\n",
+"# distributed with this work for additional information\n",
+"# regarding copyright ownership. The ASF licenses this file\n",
+"# to you under the Apache License, Version 2.0 (the\n",
+"# \"License\"); you may not use this file except in compliance\n",
+"# with the License. You may obtain a copy of the License at\n",
+"#\n",
+"#   http://www.apache.org/licenses/LICENSE-2.0\n;,
+"#\n",
+"# Unless required by applicable law or agreed to in writing,\n",
+"# software distributed under the License is distributed on an\n",
+"# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+"# KIND, either express or implied. See the License for the\n",
+"# specific language governing permissions and limitations\n",
+"# under the License"
+  ]
+},
+{
+  "cell_type": "markdown",
+  "metadata": {
+"id": "A8xNRyZMW1yK"
+  },
+  "source": [
+"# Use Apache Beam and Bigtable to enrich data\n",
+"\n",
+"\n",
+"  \n",
+"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/bigtable_enrichment_transform.ipynb\;>https://raw.gith

(beam) branch master updated (bfe6168505c -> 914cf145cba)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from bfe6168505c fixed the display issue for impersonateServiceAccount with 
Java (#30283)
 add 914cf145cba [Docs] Add transform catalog page for enrichment transform 
(#30187)

No new revisions were added by this update.

Summary of changes:
 .../snippets/transforms/elementwise/enrichment.py  | 51 +
 .../transforms/elementwise/enrichment_test.py  | 53 +
 .../site/content/en/documentation/ml/overview.md   |  3 +-
 .../transforms/python/elementwise/enrichment.md| 66 ++
 .../python/elementwise/runinference-pytorch.md |  2 +-
 .../en/documentation/transforms/python/overview.md |  1 +
 .../partials/section-menu/en/documentation.html|  1 +
 7 files changed, 175 insertions(+), 2 deletions(-)
 create mode 100644 
sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment.py
 create mode 100644 
sdks/python/apache_beam/examples/snippets/transforms/elementwise/enrichment_test.py
 create mode 100644 
website/www/site/content/en/documentation/transforms/python/elementwise/enrichment.md

(beam) branch master updated: Bump google.golang.org/grpc from 1.60.1 to 1.61.0 in /sdks (#30249)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4f3963f7878 Bump google.golang.org/grpc from 1.60.1 to 1.61.0 in /sdks 
(#30249)
4f3963f7878 is described below

commit 4f3963f7878534172b2ea094f069745f92f0b307
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Feb 7 13:07:28 2024 -0500

Bump google.golang.org/grpc from 1.60.1 to 1.61.0 in /sdks (#30249)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.60.1 
to 1.61.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.60.1...v1.61.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 4 ++--
 sdks/go.sum | 8 
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 71b3f5a3c99..8683c181a99 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -61,7 +61,7 @@ require (
golang.org/x/text v0.14.0
google.golang.org/api v0.157.0
google.golang.org/genproto v0.0.0-20240116215550-a9fa1716bcac
-   google.golang.org/grpc v1.60.1
+   google.golang.org/grpc v1.61.0
google.golang.org/protobuf v1.32.0
gopkg.in/retry.v1 v1.0.3
gopkg.in/yaml.v2 v2.4.0
@@ -138,7 +138,7 @@ require (
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe // indirect
-   github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 // indirect
+   github.com/cncf/xds/go v0.0.0-20231109132714-523115ebc101 // indirect
github.com/containerd/containerd v1.7.11 // indirect
github.com/cpuguy83/dockercfg v0.3.1 // indirect
github.com/docker/distribution v2.8.2+incompatible // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index eefb376c6dd..acb53d86716 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -146,8 +146,8 @@ github.com/cncf/udpa/go 
v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGX
 github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe 
h1:QQ3GSy+MqSHxm/d8nCtnAiZdYFd45cYZPs8vOOIYKfk=
 github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe/go.mod 
h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI=
 github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod 
h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
-github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 
h1:/inchEIKaYC1Akx+H+gqO04wryn5h75LSazbRlnya1k=
-github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod 
h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
+github.com/cncf/xds/go v0.0.0-20231109132714-523115ebc101 
h1:7To3pQ+pZo0i3dsWEbinPNFs5gPSBOsJtx3wTT94VBY=
+github.com/cncf/xds/go v0.0.0-20231109132714-523115ebc101/go.mod 
h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
 github.com/colinmarc/hdfs/v2 v2.1.1/go.mod 
h1:M3x+k8UKKmxtFu++uAZ0OtDU8jR3jnaZIAc6yK4Ue0c=
 github.com/containerd/containerd v1.7.11 
h1:lfGKw3eU35sjV0aG2eYZTiwFEY1pCzxdzicHP3SZILw=
 github.com/containerd/containerd v1.7.11/go.mod 
h1:5UluHxHTX2rdvYuZ5OJTC5m/KJNs0Zs9wVoJm9zf5ZE=
@@ -725,8 +725,8 @@ google.golang.org/grpc v1.26.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8
 google.golang.org/grpc v1.27.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.27.1/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.33.2/go.mod 
h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.60.1 h1:26+wFr+cNqSGFcOXcabYC0lUVJVRa2Sb2ortSK7VrEU=
-google.golang.org/grpc v1.60.1/go.mod 
h1:OlCHIeLYqSSsLi6i49B5QGdzaMZK9+M7LXN2FKz4eGM=
+google.golang.org/grpc v1.61.0 h1:TOvOcuXn30kRao+gfcvsebNEa5iZIiLkisYEkf7R7o0=
+google.golang.org/grpc v1.61.0/go.mod 
h1:VUbo7IFqmF1QtCAstipjG0GIoq49KvMe9+h1jFLBNJs=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod 
h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod 
h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod 
h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=

(beam) branch master updated: Bump golang.org/x/sync from 0.5.0 to 0.6.0 in /sdks (#30209)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 1a1ff756f86 Bump golang.org/x/sync from 0.5.0 to 0.6.0 in /sdks 
(#30209)
1a1ff756f86 is described below

commit 1a1ff756f868416508be1de1f3afb2afb9b7fc73
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Feb 5 09:09:22 2024 -0500

Bump golang.org/x/sync from 0.5.0 to 0.6.0 in /sdks (#30209)

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.5.0 to 
0.6.0.
- [Commits](https://github.com/golang/sync/compare/v0.5.0...v0.6.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index ebcea04a37c..493e9f7e898 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -56,7 +56,7 @@ require (
go.mongodb.org/mongo-driver v1.13.1
golang.org/x/net v0.19.0
golang.org/x/oauth2 v0.15.0
-   golang.org/x/sync v0.5.0
+   golang.org/x/sync v0.6.0
golang.org/x/sys v0.15.0
golang.org/x/text v0.14.0
google.golang.org/api v0.154.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 7ef59c4f469..7578e41acec 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -567,8 +567,8 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod 
h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE=
-golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
+golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
+golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod 
h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190130150945-aca44879d564/go.mod 
h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod 
h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

(beam) branch master updated: Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks (#30175)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 5188e37ed44 Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 
in /sdks (#30175)
5188e37ed44 is described below

commit 5188e37ed445bbe01ce7e71ee8177c2fde8c91ad
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Jan 31 22:14:09 2024 -0500

Bump github.com/opencontainers/runc from 1.1.5 to 1.1.12 in /sdks (#30175)

Bumps 
[github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 
1.1.5 to 1.1.12.
- [Release notes](https://github.com/opencontainers/runc/releases)
- 
[Changelog](https://github.com/opencontainers/runc/blob/v1.1.12/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.12)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod |  4 ++--
 sdks/go.sum | 40 ++--
 2 files changed, 8 insertions(+), 36 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index de7b3693045..80a017c0917 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -77,6 +77,7 @@ require (
dario.cat/mergo v1.0.0 // indirect
github.com/Microsoft/hcsshim v0.11.1 // indirect
github.com/containerd/log v0.1.0 // indirect
+   github.com/frankban/quicktest v1.14.0 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
@@ -155,7 +156,6 @@ require (
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/klauspost/compress v1.17.4 // indirect
github.com/klauspost/cpuid/v2 v2.2.6 // indirect
-   github.com/kr/text v0.2.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // 
indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
@@ -166,7 +166,7 @@ require (
github.com/morikuni/aec v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc5 // indirect
-   github.com/opencontainers/runc v1.1.5 // indirect
+   github.com/opencontainers/runc v1.1.12 // indirect
github.com/pierrec/lz4/v4 v4.1.15 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/xattr v0.4.9 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 60b711ab059..06b251d29a8 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -138,11 +138,9 @@ github.com/census-instrumentation/opencensus-proto v0.4.1 
h1:iKLQ0xPNFxR/2hzXZMr
 github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod 
h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw=
 github.com/cespare/xxhash/v2 v2.2.0 
h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
 github.com/cespare/xxhash/v2 v2.2.0/go.mod 
h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
-github.com/checkpoint-restore/go-criu/v5 v5.3.0/go.mod 
h1:E/eQpaFtUKGOOSEBZgmKAcn+zUUwWxqcaKZlF54wK8E=
 github.com/chzyer/logex v1.1.10/go.mod 
h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
 github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod 
h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
 github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod 
h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
-github.com/cilium/ebpf v0.7.0/go.mod 
h1:/oI2+1shJiTGAMgl6/RgJr36Eo1jzrRcAWbcXO2usCA=
 github.com/client9/misspell v0.3.4/go.mod 
h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
 github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod 
h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
 github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe 
h1:QQ3GSy+MqSHxm/d8nCtnAiZdYFd45cYZPs8vOOIYKfk=
@@ -151,19 +149,15 @@ github.com/cncf/xds/go 
v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWH
 github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4 
h1:/inchEIKaYC1Akx+H+gqO04wryn5h75LSazbRlnya1k=
 github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4/go.mod 
h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
 github.com/colinmarc/hdfs/v2 v2.1.1/go.mod 
h1:M3x+k8UKKmxtFu++uAZ0OtDU8jR3jnaZIAc6yK4Ue0c=
-github.com/containerd/console v1.0.3/go.mod 
h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
 github.com/containerd/containerd v1.7.7 
h1:QOC2K4A42RQpcrZyptP6z9EJZnlHfHJUfZrAAHe15q4=
 github.com/containerd/containerd v1.7.7/go.mod 
h1:3c4XZv6VeT9qgf9GMTxNTMFxGJrGpI2vz1yk4ye+YY8=
 github.com/containerd/log v0.1.0 
h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
 github.com/containerd/log v0.1.

(beam) branch master updated (27214149d34 -> 674fe77df47)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 27214149d34 Support dynamic destinations with Python Storage API 
(#30045)
 add 674fe77df47 [Python] Add an option to retain timestamp returned from 
the BigTable row (#30088)

No new revisions were added by this update.

Summary of changes:
 .../transforms/enrichment_handlers/bigtable.py | 24 ++---
 .../enrichment_handlers/bigtable_it_test.py| 63 ++
 2 files changed, 70 insertions(+), 17 deletions(-)

(beam) branch master updated: Bump pillow (#30076)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new e8e0ccfdd7f Bump pillow (#30076)
e8e0ccfdd7f is described below

commit e8e0ccfdd7f52d466f05ae43dfde2588a6bdc621
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Jan 23 13:59:37 2024 -0500

Bump pillow (#30076)

Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.2.0 to 
10.2.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.2.0...10.2.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 .../examples/ml-orchestration/kfp/components/train/requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/requirements.txt
 
b/sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/requirements.txt
index 57beb184499..2e65f0fba24 100644
--- 
a/sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/requirements.txt
+++ 
b/sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/requirements.txt
@@ -15,4 +15,4 @@
 
 torch==1.13.1
 numpy==1.22.4
-Pillow==9.2.0
\ No newline at end of file
+Pillow==10.2.0
\ No newline at end of file

(beam) branch master updated: [Python] Enrichment Transform with BigTable handler (#30001)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 1de8454ddd2 [Python] Enrichment Transform with BigTable handler 
(#30001)
1de8454ddd2 is described below

commit 1de8454ddd215b9659fd049bded6a4b4c484b12d
Author: Ritesh Ghorse 
AuthorDate: Thu Jan 18 21:22:11 2024 -0500

[Python] Enrichment Transform with BigTable handler (#30001)

* enrichment v1

* add documentation

* add doc comment

* rerun

* update docs, lint

* update docs, lint

* add generic type

* add generic type

* adjust doc path

* create test row

* use request type

* use request type

* change module name

* more tests

* remove non-functional params

* lint, doc

* change types for general use

* callable type

* dict type

* update signatures

* fix unit test

* bigtable with column family, ids, rrio-throttler

* update tests for row filter

* convert handler types from dict to Row

* update tests for bigtable

* ran pydocs

* ran pydocs

* mark postcommit

* remove _test file, fix import

* enable postcommit

* add more tests

* skip tests when dependencies are not installed

* add deleted imports from last commit

* add skip test condition

* fix import order, add TooManyRequests to try-catch

* make throttler, repeater non-optional

* add exception level and tests

* correct pydoc statement

* add throttle tests

* add bigtable improvements

* default app_profile_id

* add documentation, ignore None assignment

* add to changes.md

* change test structure that throws exception, skip http test for now

* drop postcommit trigger file
---
 CHANGES.md |   2 +-
 sdks/python/apache_beam/io/requestresponse.py  | 413 +
 ...nseio_it_test.py => requestresponse_it_test.py} |  37 +-
 sdks/python/apache_beam/io/requestresponse_test.py | 156 
 sdks/python/apache_beam/io/requestresponseio.py| 218 ---
 .../apache_beam/io/requestresponseio_test.py   |  88 -
 sdks/python/apache_beam/transforms/enrichment.py   | 137 +++
 .../transforms/enrichment_handlers/__init__.py |  16 +
 .../transforms/enrichment_handlers/bigtable.py | 151 
 .../enrichment_handlers/bigtable_it_test.py| 300 +++
 .../apache_beam/transforms/enrichment_it_test.py   | 162 
 .../apache_beam/transforms/enrichment_test.py  |  41 ++
 12 files changed, 1398 insertions(+), 323 deletions(-)

diff --git a/CHANGES.md b/CHANGES.md
index 81a519b07d7..dbad15f3dba 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -69,7 +69,7 @@
 
 ## New Features / Improvements
 
-* X feature added (Java/Python) 
([#X](https://github.com/apache/beam/issues/X)).
+* [Enrichment Transform](https://s.apache.org/enrichment-transform) along with 
GCP BigTable handler added to Python SDK 
([#30001](https://github.com/apache/beam/pull/30001)).
 
 ## Breaking Changes
 
diff --git a/sdks/python/apache_beam/io/requestresponse.py 
b/sdks/python/apache_beam/io/requestresponse.py
new file mode 100644
index 000..63ec7061d3e
--- /dev/null
+++ b/sdks/python/apache_beam/io/requestresponse.py
@@ -0,0 +1,413 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""``PTransform`` for reading from and writing to Web APIs."""
+import abc
+import concurrent.futures
+import contextlib
+import logging
+import sys
+import time
+from typing import Generic
+from typing import Optional
+from typing import TypeVar
+
+from google.api_core.exceptions import TooManyRequests
+
+import apache_beam as beam
+from apache_beam.io.components.adaptive_throttler import AdaptiveThrottler
+from apache_beam.metrics import Metrics
+from apache_beam.ml.i

(beam) branch master updated: [Typescript] stop service when done (#29929)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 8bd4a1e525b [Typescript] stop service when done (#29929)
8bd4a1e525b is described below

commit 8bd4a1e525ba5f3256124ae331acf0cc48ad65f9
Author: Cheskel Twersky 
AuthorDate: Fri Jan 5 16:15:32 2024 +0200

[Typescript] stop service when done (#29929)
---
 sdks/typescript/src/apache_beam/runners/dataflow.ts | 17 -
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/sdks/typescript/src/apache_beam/runners/dataflow.ts 
b/sdks/typescript/src/apache_beam/runners/dataflow.ts
index e7da1f7ada5..0a88b17f695 100644
--- a/sdks/typescript/src/apache_beam/runners/dataflow.ts
+++ b/sdks/typescript/src/apache_beam/runners/dataflow.ts
@@ -36,13 +36,20 @@ export function dataflowRunner(runnerOptions: {
   augmentedOptions.experiments.push("use_runner_v2");
   augmentedOptions.experiments.push("use_portable_job_submission");
   augmentedOptions.experiments.push("use_sibling_sdk_workers");
-  return new PortableRunner(
+  const service = PythonService.forModule(
+"apache_beam.runners.dataflow.dataflow_job_service",
+["--port", "{{PORT}}"]
+  );
+  const result = new PortableRunner(
 runnerOptions as any,
-PythonService.forModule(
-  "apache_beam.runners.dataflow.dataflow_job_service",
-  ["--port", "{{PORT}}"]
-)
+service
   ).runPipeline(pipeline, augmentedOptions);
+  result.then((res) => {
+res.waitUntilFinish().then((_state) => {
+  service.stop();
+});
+  });
+  return result;
 }
   })();
 }

(beam-starter-python) 01/01: Merge pull request #28 from apache/dependabot/pip/apache-beam-2.53.0

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-python.git

commit cc2c153590cde0ecd13a7a93b1b9a459dabb3957
Merge: da8d298 83ca91a
Author: Ritesh Ghorse 
AuthorDate: Fri Jan 5 09:02:51 2024 -0500

Merge pull request #28 from apache/dependabot/pip/apache-beam-2.53.0

Bump apache-beam from 2.52.0 to 2.53.0

 requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam-starter-python) branch main updated (da8d298 -> cc2c153)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-python.git


from da8d298  Merge pull request #27 from 
apache/dependabot/github_actions/actions/setup-python-5
 add 83ca91a  Bump apache-beam from 2.52.0 to 2.53.0
 new cc2c153  Merge pull request #28 from 
apache/dependabot/pip/apache-beam-2.53.0

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam) branch master updated (80450399472 -> 6d534fdd802)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 80450399472 Add `svetakvsundhar` to Healthcare reviewer pool (#29899)
 add 6d534fdd802 Bump github.com/aws/aws-sdk-go-v2/config from 1.26.1 to 
1.26.2 in /sdks (#29850)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod |  6 +++---
 sdks/go.sum | 12 ++--
 2 files changed, 9 insertions(+), 9 deletions(-)

(beam) branch master updated: fix contructor names (#29855)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 6a12e8b88df fix contructor names (#29855)
6a12e8b88df is described below

commit 6a12e8b88dfe0e0f635bd9b601bb809ef2919e99
Author: Ritesh Ghorse 
AuthorDate: Thu Dec 21 12:48:41 2023 -0500

fix contructor names (#29855)
---
 .../core-transforms/map/co-group-by-key/description.md  | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git 
a/learning/tour-of-beam/learning-content/core-transforms/map/co-group-by-key/description.md
 
b/learning/tour-of-beam/learning-content/core-transforms/map/co-group-by-key/description.md
index 543b216..a15321cd8ea 100644
--- 
a/learning/tour-of-beam/learning-content/core-transforms/map/co-group-by-key/description.md
+++ 
b/learning/tour-of-beam/learning-content/core-transforms/map/co-group-by-key/description.md
@@ -235,7 +235,7 @@ static class ProductWeight {
 private String fruit;
 private Integer productWeight;
 
-public WordsAlphabet(String country, String fruit, Integer 
productWeight) {
+public ProductWeight(String country, String fruit, Integer 
productWeight) {
 this.country = country;
 this.fruit = fruit;
 this.productWeight = productWeight;
@@ -275,7 +275,7 @@ static PCollection 
applyTransform(PCollection fruits, PCollectio
 String fruit = coGbkResult.getOnly(fruitsTag);
 String country = coGbkResult.getOnly(countriesTag);
 
-out.output(new WordsAlphabet(alphabet, fruit, 
country).toString());
+out.output(new ProductWeight(alphabet, fruit, 
country).toString());
 }
 
 }));
@@ -302,7 +302,7 @@ The union takes place through the keys:
 def apply_transforms(fruits, weights):
 def cogbk_result_to_product_weight(cgbk_result):
 (country, values) = cgbk_result
-return WordsAlphabet(values['weights'][0], values['fruits'][0], 
country)
+return ProductWeight(values['weights'][0], values['fruits'][0], 
country)
 
 return ({'fruits': fruits, 'weights': weights}
 | beam.CoGroupByKey()

(beam) branch master updated: Bump transformers (#29844)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 05068ead3ea Bump transformers (#29844)
05068ead3ea is described below

commit 05068ead3eaca28d7eb50fdc3345fc4e4e478664
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 21 09:45:05 2023 -0500

Bump transformers (#29844)

Bumps [transformers](https://github.com/huggingface/transformers) from 
4.30.0 to 4.36.0.
- [Release notes](https://github.com/huggingface/transformers/releases)
- 
[Commits](https://github.com/huggingface/transformers/compare/v4.30.0...v4.36.0)

---
updated-dependencies:
- dependency-name: transformers
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 .../inference/anomaly_detection/anomaly_detection_pipeline/setup.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/sdks/python/apache_beam/examples/inference/anomaly_detection/anomaly_detection_pipeline/setup.py
 
b/sdks/python/apache_beam/examples/inference/anomaly_detection/anomaly_detection_pipeline/setup.py
index 15a68228c66..72a65ab7281 100644
--- 
a/sdks/python/apache_beam/examples/inference/anomaly_detection/anomaly_detection_pipeline/setup.py
+++ 
b/sdks/python/apache_beam/examples/inference/anomaly_detection/anomaly_detection_pipeline/setup.py
@@ -32,7 +32,7 @@ REQUIREMENTS = [
 "apache-beam[gcp]==2.41.0",
 "hdbscan==0.8.28",
 "scikit-learn==1.0.2",
-"transformers==4.30.0",
+"transformers==4.36.0",
 "torch==1.13.1",
 "pandas==1.3.5",
 "yagmail==0.15.283",

(beam) branch master updated (0bbf2c3351b -> 1986517d38a)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 0bbf2c3351b Merge pull request #29826 Revert to non-schema-based Sql 
transform.
 add 1986517d38a Enable User State tests for Prism (#29822)

No new revisions were added by this update.

Summary of changes:
 sdks/go/test/integration/integration.go | 11 ---
 1 file changed, 11 deletions(-)

(beam) branch master updated: Bump cloud.google.com/go/storage from 1.35.1 to 1.36.0 in /sdks (#29767)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 75746c6fe4a Bump cloud.google.com/go/storage from 1.35.1 to 1.36.0 in 
/sdks (#29767)
75746c6fe4a is described below

commit 75746c6fe4a5489b6ec584569a7dc8eba14f537b
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 14 09:24:41 2023 -0500

Bump cloud.google.com/go/storage from 1.35.1 to 1.36.0 in /sdks (#29767)

Bumps 
[cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) 
from 1.35.1 to 1.36.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- 
[Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- 
[Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.35.1...spanner/v1.36.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index b186a42930e..075df0812ca 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -29,7 +29,7 @@ require (
cloud.google.com/go/profiler v0.4.0
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.53.1
-   cloud.google.com/go/storage v1.35.1
+   cloud.google.com/go/storage v1.36.0
github.com/aws/aws-sdk-go-v2 v1.24.0
github.com/aws/aws-sdk-go-v2/config v1.26.1
github.com/aws/aws-sdk-go-v2/credentials v1.16.12
diff --git a/sdks/go.sum b/sdks/go.sum
index 763f2708075..284879e23c6 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -43,8 +43,8 @@ cloud.google.com/go/spanner v1.53.1/go.mod 
h1:liG4iCeLqm5L3fFLU5whFITqP0e0orsAW1
 cloud.google.com/go/storage v1.0.0/go.mod 
h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
 cloud.google.com/go/storage v1.5.0/go.mod 
h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
 cloud.google.com/go/storage v1.6.0/go.mod 
h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
-cloud.google.com/go/storage v1.35.1 
h1:B59ahL//eDfx2IIKFBeT5Atm9wnNmj3+8xG/W4WB//w=
-cloud.google.com/go/storage v1.35.1/go.mod 
h1:M6M/3V/D3KpzMTJyPOR/HU6n2Si5QdaXYEsng2xgOs8=
+cloud.google.com/go/storage v1.36.0 
h1:P0mOkAcaJxhCTvAkMhxMfrTKiNcub4YmmPBtlhAyTr8=
+cloud.google.com/go/storage v1.36.0/go.mod 
h1:M6M/3V/D3KpzMTJyPOR/HU6n2Si5QdaXYEsng2xgOs8=
 dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
 dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
 dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod 
h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=

(beam) branch master updated: test revert _beam_schema_proto (#29754)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new c49e9ec7dc7 test revert _beam_schema_proto (#29754)
c49e9ec7dc7 is described below

commit c49e9ec7dc7b028945a72ab1e72ff72ed7f28d4d
Author: Ritesh Ghorse 
AuthorDate: Wed Dec 13 15:54:42 2023 -0500

test revert _beam_schema_proto (#29754)
---
 sdks/python/apache_beam/typehints/schemas.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/sdks/python/apache_beam/typehints/schemas.py 
b/sdks/python/apache_beam/typehints/schemas.py
index 6a6544aaa07..b8176dccb8e 100644
--- a/sdks/python/apache_beam/typehints/schemas.py
+++ b/sdks/python/apache_beam/typehints/schemas.py
@@ -561,7 +561,6 @@ class SchemaTranslation(object):
 '__reduce__',
 _named_tuple_reduce_method(schema.SerializeToString()))
 setattr(user_type, row_type._BEAM_SCHEMA_ID, schema.id)
-user_type._beam_schema_proto = _Ephemeral(schema)
 
 self.schema_registry.add(user_type, schema)
 coders.registry.register_coder(user_type, coders.RowCoder)

(beam) branch master updated: Fix syntax (#29719)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 475a0c78690 Fix syntax  (#29719)
475a0c78690 is described below

commit 475a0c7869051331a03f2bcb65af45113fd984a3
Author: Ritesh Ghorse 
AuthorDate: Tue Dec 12 14:08:45 2023 -0500

Fix syntax  (#29719)

* correct syntax

* remove unused inference_args
---
 .../apache_beam/ml/inference/huggingface_inference.py | 19 +--
 1 file changed, 1 insertion(+), 18 deletions(-)

diff --git a/sdks/python/apache_beam/ml/inference/huggingface_inference.py 
b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
index 1bc92c462c9..25367d22eaa 100644
--- a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
+++ b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
@@ -221,7 +221,6 @@ class 
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
   *,
   inference_fn: Optional[Callable[..., Iterable[PredictionResult]]] = None,
   load_model_args: Optional[Dict[str, Any]] = None,
-  inference_args: Optional[Dict[str, Any]] = None,
   min_batch_size: Optional[int] = None,
   max_batch_size: Optional[int] = None,
   max_batch_duration_secs: Optional[int] = None,
@@ -250,10 +249,6 @@ class 
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
   load_model_args (Dict[str, Any]): (Optional) Keyword arguments to provide
 load options while loading models from Hugging Face Hub.
 Defaults to None.
-  inference_args (Dict[str, Any]): (Optional) Non-batchable arguments
-required as inputs to the model's inference function. Unlike Tensors
-in `batch`, these parameters will not be dynamically batched.
-Defaults to None.
   min_batch_size: the minimum batch size to use when batching inputs.
   max_batch_size: the maximum batch size to use when batching inputs.
   max_batch_duration_secs: the maximum amount of time to buffer a batch
@@ -273,7 +268,6 @@ class 
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
 self._device = device
 self._inference_fn = inference_fn
 self._model_config_args = load_model_args if load_model_args else {}
-self._inference_args = inference_args if inference_args else {}
 self._batching_kwargs = {}
 self._env_vars = kwargs.get("env_vars", {})
 if min_batch_size is not None:
@@ -293,7 +287,7 @@ class 
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
 model = self._model_class.from_pretrained(
 self._model_uri, **self._model_config_args)
 if self._framework == 'pt':
-  if self._device == "GPU" and is_gpu_available_torch:
+  if self._device == "GPU" and is_gpu_available_torch():
 model.to(torch.device("cuda"))
   if callable(getattr(model, 'requires_grad_', None)):
 model.requires_grad_(False)
@@ -407,7 +401,6 @@ class 
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
   *,
   inference_fn: Optional[Callable[..., Iterable[PredictionResult]]] = None,
   load_model_args: Optional[Dict[str, Any]] = None,
-  inference_args: Optional[Dict[str, Any]] = None,
   min_batch_size: Optional[int] = None,
   max_batch_size: Optional[int] = None,
   max_batch_duration_secs: Optional[int] = None,
@@ -436,10 +429,6 @@ class 
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
   load_model_args (Dict[str, Any]): (Optional) keyword arguments to provide
 load options while loading models from Hugging Face Hub.
 Defaults to None.
-  inference_args (Dict[str, Any]): (Optional) Non-batchable arguments
-required as inputs to the model's inference function. Unlike Tensors
-in `batch`, these parameters will not be dynamically batched.
-Defaults to None.
   min_batch_size: the minimum batch size to use when batching inputs.
   max_batch_size: the maximum batch size to use when batching inputs.
   max_batch_duration_secs: the maximum amount of time to buffer a batch
@@ -459,7 +448,6 @@ class 
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
 self._device = device
 self._inference_fn = inference_fn
 self._model_config_args = load_model_args if load_model_args else {}
-self._inference_args = inference_args if inference_args else {}
 self._batching_kwargs = {}
 self._env_vars = kwargs.get("env_vars", {})
 if min_batch_size is not None:
@@ -586,7 +574,6 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
   device: Optional[str] = None,
   inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
   load_pipeline_args: Optional[Dict[str, Any]] = None,
-  inference_args: Opti

(beam) branch master updated: [Python] Log dependencies in runtime environment (#29705)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new b91a5174d14 [Python] Log dependencies in runtime environment (#29705)
b91a5174d14 is described below

commit b91a5174d149145ceab4477bcfcaff857beeac2f
Author: Ritesh Ghorse 
AuthorDate: Mon Dec 11 14:27:51 2023 -0500

[Python] Log dependencies in runtime environment (#29705)

* log runtime dependencies

* log py version

* Update boot.go

* move print block

* fix indent

* fix indent

* fix indent
---
 sdks/python/container/boot.go | 28 
 1 file changed, 28 insertions(+)

diff --git a/sdks/python/container/boot.go b/sdks/python/container/boot.go
index ded10a44204..cbf05bbd04a 100644
--- a/sdks/python/container/boot.go
+++ b/sdks/python/container/boot.go
@@ -409,6 +409,9 @@ func installSetupPackages(ctx context.Context, logger 
*tools.Logger, files []str
if err := pipInstallPackage(ctx, logger, files, workDir, workflowFile, 
false, true, nil); err != nil {
return fmt.Errorf("failed to install workflow: %v", err)
}
+   if err := logRuntimeDependencies(ctx, logger); err != nil {
+   logger.Warnf(ctx, "couldn't fetch the runtime python 
dependencies: %v", err)
+   }
 
return nil
 }
@@ -453,3 +456,28 @@ func processArtifactsInSetupOnlyMode() {
log.Fatalf("Failed to install required packages: %v", setupErr)
}
 }
+
+// logRuntimeDependencies logs the python dependencies
+// installed in the runtime environment.
+func logRuntimeDependencies(ctx context.Context, logger *tools.Logger) error {
+   logger.Printf(ctx, "Logging runtime dependencies:")
+   pythonVersion, err := expansionx.GetPythonVersion()
+   if err != nil {
+   return err
+   }
+   logger.Printf(ctx, "Using Python version:")
+   args := []string{"--version"}
+   bufLogger := tools.NewBufferedLogger(logger)
+   if err := execx.ExecuteEnvWithIO(nil, os.Stdin, bufLogger, bufLogger, 
pythonVersion, args...); err != nil {
+   bufLogger.FlushAtError(ctx)
+   } else {
+   bufLogger.FlushAtDebug(ctx)
+   }
+   args = []string{"-m", "pip", "freeze"}
+   if err := execx.ExecuteEnvWithIO(nil, os.Stdin, bufLogger, bufLogger, 
pythonVersion, args...); err != nil {
+   bufLogger.FlushAtError(ctx)
+   } else {
+   bufLogger.FlushAtDebug(ctx)
+   }
+   return nil
+}

(beam) branch master updated (a3d011549d7 -> 5c7ec723a44)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from a3d011549d7 Merge pull request #29563: [runners-flink] #29558 emit 
watermark before source shutdown
 add 5c7ec723a44 Bump github.com/aws/aws-sdk-go-v2/config from 1.26.0 to 
1.26.1 in /sdks (#29702)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 12 ++--
 sdks/go.sum | 24 
 2 files changed, 18 insertions(+), 18 deletions(-)

(beam) branch master updated (209b0959ce8 -> a7976f54908)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 209b0959ce8 Bump github.com/aws/aws-sdk-go-v2/config from 1.25.8 to 
1.26.0 in /sdks (#29678)
 add a7976f54908 [Python][RRIO] Call PTransform with setup teardown (#29585)

No new revisions were added by this update.

Summary of changes:
 sdks/python/apache_beam/io/requestresponseio.py| 181 +++--
 .../apache_beam/io/requestresponseio_it_test.py|  18 +-
 .../apache_beam/io/requestresponseio_test.py   |  88 ++
 3 files changed, 270 insertions(+), 17 deletions(-)
 create mode 100644 sdks/python/apache_beam/io/requestresponseio_test.py

(beam) branch master updated: Bump github.com/aws/aws-sdk-go-v2/config from 1.25.8 to 1.26.0 in /sdks (#29678)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 209b0959ce8 Bump github.com/aws/aws-sdk-go-v2/config from 1.25.8 to 
1.26.0 in /sdks (#29678)
209b0959ce8 is described below

commit 209b0959ce83358825dd2042d7fb276f7205194a
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Dec 8 10:59:25 2023 -0500

Bump github.com/aws/aws-sdk-go-v2/config from 1.25.8 to 1.26.0 in /sdks 
(#29678)

Bumps 
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) 
from 1.25.8 to 1.26.0.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.25.8...config/v1.26.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 24 
 sdks/go.sum | 48 
 2 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index c46c7e28a58..6beb93088f0 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,12 +30,12 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.53.1
cloud.google.com/go/storage v1.35.1
-   github.com/aws/aws-sdk-go-v2 v1.23.5
-   github.com/aws/aws-sdk-go-v2/config v1.25.8
-   github.com/aws/aws-sdk-go-v2/credentials v1.16.9
+   github.com/aws/aws-sdk-go-v2 v1.24.0
+   github.com/aws/aws-sdk-go-v2/config v1.26.0
+   github.com/aws/aws-sdk-go-v2/credentials v1.16.11
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.8
github.com/aws/aws-sdk-go-v2/service/s3 v1.42.2
-   github.com/aws/smithy-go v1.18.1
+   github.com/aws/smithy-go v1.19.0
github.com/docker/go-connections v0.4.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
@@ -106,18 +106,18 @@ require (
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.1 // indirect
-   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.9 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.8 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.8 // indirect
+   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.10 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.9 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.9 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.7.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.3 // indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.3 
// indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 
// indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.8 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.9 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.18.2 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.2 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.26.2 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.18.4 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.4 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.26.4 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 5df50d7fd02..e4bf5964459 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -81,39 +81,39 @@ github.com/aws/aws-sdk-go v1.30.19/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZve
 github.com/aws/aws-sdk-go v1.34.0 
h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
 github.com/aws/aws-sdk-go v1.34.0/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
 github.com/aws/aws-sdk-go-v2 v1.7.1/go.mod 
h1:L5LuPC1ZgDr2xQS7AmIec/Jlc7O/Y1u2KxJyNVab250=
-github.com/aws/aws-sdk-go-v2 v1.23.5 
h1:xK6C4udTyDMd82RFvNkDQxtAd00xlzFUtX4fF2nMZyg=
-github.com/aws/aws-sdk-go-v2 v1.23.5/go.mod 
h1:t3szzKfP0NeRU27uBFczDivYJjsmSnqI8kIvKyWb9ds=
+github.com/aws/aws-sdk-go-v2

(beam-starter-go) branch main updated (ad212fa -> 291d696)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git


from ad212fa  Bump github.com/apache/beam/sdks/v2 from 2.51.0 to 2.52.0 
(#26)
 add c55a548  Bump actions/setup-go from 4 to 5
 new 291d696  Merge pull request #27 from 
apache/dependabot/github_actions/actions/setup-go-5

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .github/workflows/test.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam-starter-go) 01/01: Merge pull request #27 from apache/dependabot/github_actions/actions/setup-go-5

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git

commit 291d6961358359eb8bd48182b01dd559dce4e05f
Merge: ad212fa c55a548
Author: Ritesh Ghorse 
AuthorDate: Thu Dec 7 09:40:50 2023 -0500

Merge pull request #27 from 
apache/dependabot/github_actions/actions/setup-go-5

Bump actions/setup-go from 4 to 5

 .github/workflows/test.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(beam) branch master updated: Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 70926f5da78 Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in 
/sdks (#29657)
70926f5da78 is described below

commit 70926f5da784e6f0ca1eebc64352e942a6e556c7
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Dec 7 09:40:04 2023 -0500

Bump go.mongodb.org/mongo-driver from 1.13.0 to 1.13.1 in /sdks (#29657)

Bumps 
[go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 
1.13.0 to 1.13.1.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- 
[Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.13.0...v1.13.1)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index fb53e17b4a3..2060c3c0a50 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -53,7 +53,7 @@ require (
github.com/tetratelabs/wazero v1.5.0
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
-   go.mongodb.org/mongo-driver v1.13.0
+   go.mongodb.org/mongo-driver v1.13.1
golang.org/x/net v0.19.0
golang.org/x/oauth2 v0.14.0
golang.org/x/sync v0.5.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 2cf0742086d..3c99d658727 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -477,8 +477,8 @@ github.com/zeebo/assert v1.3.0 
h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
 github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
 github.com/zeebo/xxh3 v1.0.2/go.mod 
h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
 go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
-go.mongodb.org/mongo-driver v1.13.0 
h1:67DgFFjYOCMWdtTEmKFpV3ffWlFnh+CYZ8ZS/tXWUfY=
-go.mongodb.org/mongo-driver v1.13.0/go.mod 
h1:/rGBTebI3XYboVmgz+Wv3Bcbl3aD0QF9zl6kDDw18rQ=
+go.mongodb.org/mongo-driver v1.13.1 
h1:YIc7HTYsKndGK4RFzJ3covLz1byri52x0IoMB0Pt/vk=
+go.mongodb.org/mongo-driver v1.13.1/go.mod 
h1:wcDf1JBCXy2mOW0bWHwO/IOYqdca1MPCwDtFu/Z9+eo=
 go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
 go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
 go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=

(beam) branch master updated: Bump github.com/nats-io/nats-server/v2 from 2.10.5 to 2.10.6 in /sdks (#29629)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 50e0751d11c Bump github.com/nats-io/nats-server/v2 from 2.10.5 to 
2.10.6 in /sdks (#29629)
50e0751d11c is described below

commit 50e0751d11c8a3fc8607e0d2109b2e911be72925
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Dec 6 11:50:18 2023 -0500

Bump github.com/nats-io/nats-server/v2 from 2.10.5 to 2.10.6 in /sdks 
(#29629)

Bumps 
[github.com/nats-io/nats-server/v2](https://github.com/nats-io/nats-server) 
from 2.10.5 to 2.10.6.
- [Release notes](https://github.com/nats-io/nats-server/releases)
- 
[Changelog](https://github.com/nats-io/nats-server/blob/main/.goreleaser.yml)
- 
[Commits](https://github.com/nats-io/nats-server/compare/v2.10.5...v2.10.6)

---
updated-dependencies:
- dependency-name: github.com/nats-io/nats-server/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 4 ++--
 sdks/go.sum | 8 
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 33e3e7507cb..40899ff53e7 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -45,7 +45,7 @@ require (
github.com/johannesboyne/gofakes3 v0.0.0-20221110173912-32fb85c5aed6
github.com/lib/pq v1.10.9
github.com/linkedin/goavro/v2 v2.12.0
-   github.com/nats-io/nats-server/v2 v2.10.5
+   github.com/nats-io/nats-server/v2 v2.10.6
github.com/nats-io/nats.go v1.31.0
github.com/proullon/ramsql v0.1.3
github.com/spf13/cobra v1.8.0
@@ -146,7 +146,7 @@ require (
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
-   github.com/klauspost/compress v1.17.2 // indirect
+   github.com/klauspost/compress v1.17.3 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 0f61b39482a..6fb7c9bdc09 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -321,8 +321,8 @@ github.com/klauspost/asmfmt v1.3.2/go.mod 
h1:AG8TuvYojzulgDAMCnYn50l/5QV3Bs/tp6j
 github.com/klauspost/compress v1.9.7/go.mod 
h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
 github.com/klauspost/compress v1.13.1/go.mod 
h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
 github.com/klauspost/compress v1.13.6/go.mod 
h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
-github.com/klauspost/compress v1.17.2 
h1:RlWWUY/Dr4fL8qk9YG7DTZ7PDgME2V4csBXA8L/ixi4=
-github.com/klauspost/compress v1.17.2/go.mod 
h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE=
+github.com/klauspost/compress v1.17.3 
h1:qkRjuerhUU1EmXLYGkSH6EZL+vPSxIrYjLNAK4slzwA=
+github.com/klauspost/compress v1.17.3/go.mod 
h1:/dCuZOvVtNoHsyb+cuJD3itjs3NbnF6KH9zAO4BDxPM=
 github.com/klauspost/cpuid/v2 v2.2.5 
h1:0E5MSMDEoAulmXNFquVs//DdoomxaoTY1kUhbc/qbZg=
 github.com/klauspost/cpuid/v2 v2.2.5/go.mod 
h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
 github.com/kr/pretty v0.1.0/go.mod 
h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
@@ -366,8 +366,8 @@ github.com/morikuni/aec v1.0.0/go.mod 
h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7P
 github.com/mrunalp/fileutils v0.5.0/go.mod 
h1:M1WthSahJixYnrXQl/DFQuteStB1weuxD2QJNHXfbSQ=
 github.com/nats-io/jwt/v2 v2.5.3 
h1:/9SWvzc6hTfamcgXJ3uYRpgj+QuY2aLNqRiqrKcrpEo=
 github.com/nats-io/jwt/v2 v2.5.3/go.mod 
h1:iysuPemFcc7p4IoYots3IuELSI4EDe9Y0bQMe+I3Bf4=
-github.com/nats-io/nats-server/v2 v2.10.5 
h1:hhWt6m9ja/mNnm6ixc85jCthDaiUFPaeJI79K/MD980=
-github.com/nats-io/nats-server/v2 v2.10.5/go.mod 
h1:xUMTU4kS//SDkJCSvFwN9SyJ9nUuLhSkzB/Qz0dvjjg=
+github.com/nats-io/nats-server/v2 v2.10.6 
h1:40U3ngyAKyC1tNT4Kw7PjuvivY74NTYD3qyIHxZUHKQ=
+github.com/nats-io/nats-server/v2 v2.10.6/go.mod 
h1:IrTXS8o4Roa3G2kW8L5mEtSdmSrFjKhYb/m2g0gQ/vc=
 github.com/nats-io/nats.go v1.31.0 
h1:/WFBHEc/dOKBF6qf1TZhrdEfTmOZ5JzdJ+Y3m6Y/p7E=
 github.com/nats-io/nats.go v1.31.0/go.mod 
h1:di3Bm5MLsoB4Bx61CBTsxuarI36WbhAwOm8QrW39+i8=
 github.com/nats-io/nkeys v0.4.6 h1:IzVe95ru2CT6ta874rt9saQRkWfe2nFj1NtvYSLqMzY=

(beam) branch master updated: Bump go.mongodb.org/mongo-driver from 1.12.1 to 1.13.0 in /sdks (#29343)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 68bd67fb084 Bump go.mongodb.org/mongo-driver from 1.12.1 to 1.13.0 in 
/sdks (#29343)
68bd67fb084 is described below

commit 68bd67fb084e7e3064a466277fb1362fab67cb3d
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Dec 5 14:38:02 2023 -0500

Bump go.mongodb.org/mongo-driver from 1.12.1 to 1.13.0 in /sdks (#29343)

Bumps 
[go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 
1.12.1 to 1.13.0.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- 
[Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.12.1...v1.13.0)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index b55c0bb474b..33e3e7507cb 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -53,7 +53,7 @@ require (
github.com/tetratelabs/wazero v1.5.0
github.com/xitongsys/parquet-go v1.6.2
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
-   go.mongodb.org/mongo-driver v1.12.1
+   go.mongodb.org/mongo-driver v1.13.0
golang.org/x/net v0.19.0
golang.org/x/oauth2 v0.14.0
golang.org/x/sync v0.5.0
diff --git a/sdks/go.sum b/sdks/go.sum
index ed4921bfacc..0f61b39482a 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -477,8 +477,8 @@ github.com/zeebo/assert v1.3.0 
h1:g7C04CbJuIDKNPFHmsk4hwZDO5O+kntRxzaUoNXj+IQ=
 github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
 github.com/zeebo/xxh3 v1.0.2/go.mod 
h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
 go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
-go.mongodb.org/mongo-driver v1.12.1 
h1:nLkghSU8fQNaK7oUmDhQFsnrtcoNy7Z6LVFKsEecqgE=
-go.mongodb.org/mongo-driver v1.12.1/go.mod 
h1:/rGBTebI3XYboVmgz+Wv3Bcbl3aD0QF9zl6kDDw18rQ=
+go.mongodb.org/mongo-driver v1.13.0 
h1:67DgFFjYOCMWdtTEmKFpV3ffWlFnh+CYZ8ZS/tXWUfY=
+go.mongodb.org/mongo-driver v1.13.0/go.mod 
h1:/rGBTebI3XYboVmgz+Wv3Bcbl3aD0QF9zl6kDDw18rQ=
 go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
 go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
 go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=

(beam) branch master updated: Bump cryptography from 41.0.4 to 41.0.6 in /sdks/python/container/py38 (#29554)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 5f6afcef55f Bump cryptography from 41.0.4 to 41.0.6 in 
/sdks/python/container/py38 (#29554)
5f6afcef55f is described below

commit 5f6afcef55f8a117c0518e3aebacd5c94ca05b52
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Dec 1 11:01:08 2023 -0500

Bump cryptography from 41.0.4 to 41.0.6 in /sdks/python/container/py38 
(#29554)

Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.4 to 
41.0.6.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.4...41.0.6)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/python/container/py38/base_image_requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/python/container/py38/base_image_requirements.txt 
b/sdks/python/container/py38/base_image_requirements.txt
index 5d5f80d..83580f65f2d 100644
--- a/sdks/python/container/py38/base_image_requirements.txt
+++ b/sdks/python/container/py38/base_image_requirements.txt
@@ -33,7 +33,7 @@ charset-normalizer==3.3.0
 click==8.1.7
 cloudpickle==2.2.1
 crcmod==1.7
-cryptography==41.0.4
+cryptography==41.0.6
 Cython==0.29.36
 deprecation==2.1.0
 dill==0.3.1.1

(beam) branch master updated: Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#29575)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 7c99c330350 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks 
(#29575)
7c99c330350 is described below

commit 7c99c33035038a91d270f1f89f13af6ea7b62e6b
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Dec 1 11:00:28 2023 -0500

Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#29575)

Bumps 
[github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2)
 from 1.16.6 to 1.16.8.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.6...v1.16.8)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 22 +++---
 sdks/go.sum | 44 ++--
 2 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index d64ee503338..3223814d86b 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,12 +30,12 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.51.0
cloud.google.com/go/storage v1.35.1
-   github.com/aws/aws-sdk-go-v2 v1.23.2
+   github.com/aws/aws-sdk-go-v2 v1.23.4
github.com/aws/aws-sdk-go-v2/config v1.25.8
-   github.com/aws/aws-sdk-go-v2/credentials v1.16.6
+   github.com/aws/aws-sdk-go-v2/credentials v1.16.8
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.8
github.com/aws/aws-sdk-go-v2/service/s3 v1.42.2
-   github.com/aws/smithy-go v1.17.0
+   github.com/aws/smithy-go v1.18.1
github.com/docker/go-connections v0.4.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
@@ -106,18 +106,18 @@ require (
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.1 // indirect
-   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.6 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.5 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.5 // indirect
+   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.8 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.7 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.7 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.7.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.3 // indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.1 
// indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.3 
// indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.5 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.7 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.17.5 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.20.3 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.25.6 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.18.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.26.1 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 45f8971db12..b9914ab1b93 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -81,39 +81,39 @@ github.com/aws/aws-sdk-go v1.30.19/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZve
 github.com/aws/aws-sdk-go v1.34.0 
h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
 github.com/aws/aws-sdk-go v1.34.0/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
 github.com/aws/aws-sdk-go-v2 v1.7.1/go.mod 
h1:L5LuPC1ZgDr2xQS7AmIec/Jlc7O/Y1u2KxJyNVab250=
-github.com/aws/aws-sdk-go-v2 v1.23.2 
h1:UoTll1Y5b88x8h53OlsJGgOHwpggdMr7UVnLjMb3XYg=
-github.com/aws/aws-sdk-go-v2 v1.23.2/go.mod 
h1:i1XDttT4rnf6vxc9AuskLc6s7XBee8rlLilKlc03uAA=
+github.com/aws/aws-sdk-go-v2 v1.23.4 
h1:2P20ZjH0ouSAu/6yZep8oCmTReathLuEu6dwoqEgjts=
+github.com/aws/aws-sdk-go-v2 v1.23.

(beam) branch master updated: Bump github.com/aws/aws-sdk-go-v2/config from 1.25.1 to 1.25.8 in /sdks (#29556)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new f6842049064 Bump github.com/aws/aws-sdk-go-v2/config from 1.25.1 to 
1.25.8 in /sdks (#29556)
f6842049064 is described below

commit f68420490644139cc018439a7ef8455069f8b3de
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Nov 29 11:56:22 2023 -0500

Bump github.com/aws/aws-sdk-go-v2/config from 1.25.1 to 1.25.8 in /sdks 
(#29556)

Bumps 
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) 
from 1.25.1 to 1.25.8.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.25.1...config/v1.25.8)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 22 +++---
 sdks/go.sum | 44 ++--
 2 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index c346e3cdd8c..8d9fed84365 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,9 +30,9 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.51.0
cloud.google.com/go/storage v1.34.1
-   github.com/aws/aws-sdk-go-v2 v1.23.0
-   github.com/aws/aws-sdk-go-v2/config v1.25.1
-   github.com/aws/aws-sdk-go-v2/credentials v1.16.1
+   github.com/aws/aws-sdk-go-v2 v1.23.2
+   github.com/aws/aws-sdk-go-v2/config v1.25.8
+   github.com/aws/aws-sdk-go-v2/credentials v1.16.6
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.8
github.com/aws/aws-sdk-go-v2/service/s3 v1.42.2
github.com/aws/smithy-go v1.17.0
@@ -106,18 +106,18 @@ require (
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.1 // indirect
-   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.4 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.3 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.3 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/ini v1.7.0 // indirect
+   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.6 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.5 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.5 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/ini v1.7.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.1 
// indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.3 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.5 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.3 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.17.2 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.19.2 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.25.2 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.17.5 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.20.3 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.25.6 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index d63f059d0ad..2c72f467f78 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -81,29 +81,29 @@ github.com/aws/aws-sdk-go v1.30.19/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZve
 github.com/aws/aws-sdk-go v1.34.0 
h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
 github.com/aws/aws-sdk-go v1.34.0/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
 github.com/aws/aws-sdk-go-v2 v1.7.1/go.mod 
h1:L5LuPC1ZgDr2xQS7AmIec/Jlc7O/Y1u2KxJyNVab250=
-github.com/aws/aws-sdk-go-v2 v1.23.0 
h1:PiHAzmiQQr6JULBUdvR8fKlA+UPKLT/8KbiqpFBWiAo=
-github.com/aws/aws-sdk-go-v2 v1.23.0/go.mod 
h1:i1XDttT4rnf6vxc9AuskLc6s7XBee8rlLilKlc03uAA=
+github.com/aws/aws-sdk-go-v2 v1.23.2 
h1:UoTll1Y5b88x8h53OlsJGgOHwpggdMr7UVnLjMb3XYg=
+github.com/aws/aws-sdk-go-v2 v1.23.2/go.mod 
h1:i1XDttT4rnf6vxc9AuskLc6s7XBee8rlLilKlc03uAA=
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstrea

(beam) branch master updated (86f711b1bf7 -> 29cd0aa8fe5)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 86f711b1bf7 Don't pin an older commit in the documentation. (#29540)
 add 29cd0aa8fe5 Add warning for unknown pipeline options (#29474)

No new revisions were added by this update.

Summary of changes:
 sdks/python/apache_beam/options/pipeline_options.py | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

(beam) branch master updated: [RRIO][Python] Replace call with __call__ to make caller callable (#29497)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new c8a6d4593e7 [RRIO][Python] Replace call with __call__ to make caller 
callable (#29497)
c8a6d4593e7 is described below

commit c8a6d4593e799bde5b2e6847d4de3db6273fedda
Author: Ritesh Ghorse 
AuthorDate: Mon Nov 20 13:19:58 2023 -0500

[RRIO][Python] Replace call with __call__ to make caller callable (#29497)

* replace call with __call__ to caller callable

* update doc comment
---
 sdks/python/apache_beam/io/requestresponseio.py |  4 ++--
 sdks/python/apache_beam/io/requestresponseio_it_test.py | 14 +++---
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/sdks/python/apache_beam/io/requestresponseio.py 
b/sdks/python/apache_beam/io/requestresponseio.py
index 912fbb73194..fc742fa00ca 100644
--- a/sdks/python/apache_beam/io/requestresponseio.py
+++ b/sdks/python/apache_beam/io/requestresponseio.py
@@ -40,10 +40,10 @@ class UserCodeTimeoutException(UserCodeExecutionException):
 class Caller(metaclass=abc.ABCMeta):
   """Interfaces user custom code intended for API calls."""
   @abc.abstractmethod
-  def call(self, request: RequestT) -> ResponseT:
+  def __call__(self, request: RequestT, *args, **kwargs) -> ResponseT:
 """Calls a Web API with the ``RequestT``  and returns a
 ``ResponseT``. ``RequestResponseIO`` expects implementations of the
-call method to throw either a ``UserCodeExecutionException``,
+``__call__`` method to throw either a ``UserCodeExecutionException``,
 ``UserCodeQuotaException``, or ``UserCodeTimeoutException``.
 """
 pass
diff --git a/sdks/python/apache_beam/io/requestresponseio_it_test.py 
b/sdks/python/apache_beam/io/requestresponseio_it_test.py
index 608722a2ff6..f291ff96a4d 100644
--- a/sdks/python/apache_beam/io/requestresponseio_it_test.py
+++ b/sdks/python/apache_beam/io/requestresponseio_it_test.py
@@ -79,7 +79,7 @@ class EchoHTTPCaller(Caller):
   def __init__(self, url: str):
 self.url = url + _HTTP_PATH
 
-  def call(self, request: EchoRequest) -> EchoResponse:
+  def __call__(self, request: EchoRequest, *args, **kwargs) -> EchoResponse:
 """Overrides ``Caller``'s call method invoking the
 ``EchoServiceGrpc``'s HTTP handler with an ``EchoRequest``, returning
 either a successful ``EchoResponse`` or throwing either a
@@ -130,9 +130,9 @@ class EchoHTTPCallerTestIT(unittest.TestCase):
 req = EchoRequest(id=options.should_exceed_quota_id, payload=_PAYLOAD)
 try:
   # The following is needed to exceed the API
-  client.call(req)
-  client.call(req)
-  client.call(req)
+  client(req)
+  client(req)
+  client(req)
 except UserCodeExecutionException as e:
   if not isinstance(e, UserCodeQuotaException):
 raise e
@@ -148,7 +148,7 @@ class EchoHTTPCallerTestIT(unittest.TestCase):
 
 req = EchoRequest(id=options.never_exceed_quota_id, payload=_PAYLOAD)
 
-response: EchoResponse = client.call(req)
+response: EchoResponse = client(req)
 
 self.assertEqual(req.id, response.id)
 self.assertEqual(req.payload, response.payload)
@@ -158,14 +158,14 @@ class EchoHTTPCallerTestIT(unittest.TestCase):
 
 req = EchoRequest(id=options.should_exceed_quota_id, payload=_PAYLOAD)
 
-self.assertRaises(UserCodeQuotaException, lambda: client.call(req))
+self.assertRaises(UserCodeQuotaException, lambda: client(req))
 
   def test_not_found_should_raise(self):
 client, _ = EchoHTTPCallerTestIT._get_client_and_options()
 
 req = EchoRequest(id='i-dont-exist-quota-id', payload=_PAYLOAD)
 self.assertRaisesRegex(
-UserCodeExecutionException, "Not Found", lambda: client.call(req))
+UserCodeExecutionException, "Not Found", lambda: client(req))
 
 
 if __name__ == '__main__':

(beam) branch master updated (c1b83d2365c -> c4e211b000a)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from c1b83d2365c Copy edit the Custom Remote Inference notebook (#29460)
 add c4e211b000a Fix a bunch of no-op typos (#29294)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/beam/examples/complete/AutoComplete.java |  6 +++---
 examples/notebooks/beam-ml/image_processing_tensorflow.ipynb |  2 +-
 examples/notebooks/beam-ml/nlp_tensorflow_streaming.ipynb|  2 +-
 .../backend/internal/environment/environment_service.go  |  2 +-
 .../wrappers/streaming/stableinput/BufferingDoFnRunner.java  |  2 +-
 .../org/apache/beam/runners/dataflow/DataflowRunner.java |  2 +-
 .../runners/dataflow/options/DataflowPipelineOptions.java|  2 +-
 .../apache/beam/runners/dataflow/util/RandomAccessData.java  |  6 +++---
 .../java/org/apache/beam/runners/dataflow/util/TimeUtil.java |  2 +-
 .../dataflow/worker/DataflowExecutionStateRegistry.java  |  2 +-
 .../runners/fnexecution/artifact/ArtifactStagingService.java |  6 +++---
 .../main/java/org/apache/beam/sdk/coders/AtomicCoder.java|  2 +-
 .../java/org/apache/beam/sdk/coders/IterableLikeCoder.java   |  2 +-
 .../java/org/apache/beam/sdk/coders/RowCoderGenerator.java   |  2 +-
 .../src/main/java/org/apache/beam/sdk/io/FileBasedSink.java  |  2 +-
 .../src/main/java/org/apache/beam/sdk/testing/PAssert.java   |  2 +-
 .../src/main/java/org/apache/beam/sdk/transforms/Regex.java  |  2 +-
 .../src/main/java/org/apache/beam/sdk/transforms/Sets.java   |  2 +-
 .../sdk/transforms/reflect/ByteBuddyDoFnInvokerFactory.java  |  2 +-
 .../src/main/java/org/apache/beam/sdk/util/ApiSurface.java   |  8 
 .../java/org/apache/beam/sdk/util/common/ReflectHelpers.java |  2 +-
 .../beam/sdk/io/BoundedReadFromUnboundedSourceTest.java  |  2 +-
 .../test/java/org/apache/beam/sdk/transforms/ParDoTest.java  | 10 +-
 .../org/apache/beam/sdk/transforms/windowing/WindowTest.java |  2 +-
 .../org/apache/beam/sdk/util/common/ReflectHelpersTest.java  |  4 ++--
 .../apache/beam/sdk/expansion/service/ExpansionService.java  | 10 +-
 .../euphoria/core/annotation/operator/package-info.java  |  2 +-
 .../euphoria/core/client/operator/ReduceWindow.java  |  2 +-
 .../euphoria/core/client/operator/base/Builders.java |  2 +-
 .../extensions/euphoria/core/translate/JoinTranslator.java   |  2 +-
 .../core/translate/provider/GenericTranslatorProvider.java   |  2 +-
 .../euphoria/core/docs/DocumentationExamplesTest.java| 12 ++--
 .../sdk/extensions/sql/impl/rel/BeamLogicalConvention.java   |  2 +-
 .../beam/sdk/extensions/sql/impl/rel/BeamUnnestRel.java  |  2 +-
 .../sql/impl/udf/BeamSalUhfSpecialTypeAndValueTest.java  |  6 +++---
 .../sql/meta/provider/bigtable/BigtableTableFlatTest.java|  2 +-
 .../extensions/sql/meta/provider/kafka/KafkaTestTable.java   |  8 
 .../apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java|  2 +-
 .../apache/beam/sdk/io/gcp/bigquery/AppendClientInfo.java|  2 +-
 .../java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java |  4 ++--
 .../apache/beam/sdk/io/gcp/bigquery/BigQueryIOMetadata.java  |  2 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryResourceNaming.java |  2 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java   |  8 
 .../org/apache/beam/sdk/io/gcp/bigquery/RetryManager.java|  2 +-
 ...ommmitterImplTest.java => BlockingCommitterImplTest.java} |  2 +-
 sdks/python/apache_beam/io/gcp/bigquery_avro_tools.py|  2 +-
 sdks/python/apache_beam/testing/analyzers/__init__.py|  2 +-
 47 files changed, 79 insertions(+), 79 deletions(-)
 rename 
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsublite/internal/{BlockingCommmitterImplTest.java
 => BlockingCommitterImplTest.java} (98%)

(beam) branch master updated: Bump golang.org/x/text from 0.13.0 to 0.14.0 in /sdks (#29324)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 190237817ba Bump golang.org/x/text from 0.13.0 to 0.14.0 in /sdks 
(#29324)
190237817ba is described below

commit 190237817ba4f84acf725285ce999aabb11072b8
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Nov 16 12:30:35 2023 -0500

Bump golang.org/x/text from 0.13.0 to 0.14.0 in /sdks (#29324)

Bumps [golang.org/x/text](https://github.com/golang/text) from 0.13.0 to 
0.14.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 8103dfc43b0..ed08423691d 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -58,7 +58,7 @@ require (
golang.org/x/oauth2 v0.13.0
golang.org/x/sync v0.5.0
golang.org/x/sys v0.14.0
-   golang.org/x/text v0.13.0
+   golang.org/x/text v0.14.0
google.golang.org/api v0.150.0
google.golang.org/genproto v0.0.0-20231016165738-49dd2c1f3d0b
google.golang.org/grpc v1.59.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 4a5d989dbe9..cbf54e36f00 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -629,8 +629,8 @@ golang.org/x/text v0.3.6/go.mod 
h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
 golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
 golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
 golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
-golang.org/x/text v0.13.0/go.mod 
h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
+golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
+golang.org/x/text v0.14.0/go.mod 
h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
 golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod 
h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod 
h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod 
h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=

(beam) branch master updated: Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#29449)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 6235a0c9f05 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in 
/sdks (#29449)
6235a0c9f05 is described below

commit 6235a0c9f050859695ad0e5a2db5f695b05b0fb7
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Nov 16 11:22:49 2023 -0500

Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#29449)

Bumps 
[github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2)
 from 1.13.1 to 1.13.8.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.13.1...service/mq/v1.13.8)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 38 +++
 sdks/go.sum | 76 ++---
 2 files changed, 57 insertions(+), 57 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 1b5ce5aba9b..8103dfc43b0 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,12 +30,12 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.51.0
cloud.google.com/go/storage v1.34.1
-   github.com/aws/aws-sdk-go-v2 v1.22.2
-   github.com/aws/aws-sdk-go-v2/config v1.22.0
-   github.com/aws/aws-sdk-go-v2/credentials v1.15.1
-   github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.1
-   github.com/aws/aws-sdk-go-v2/service/s3 v1.42.0
-   github.com/aws/smithy-go v1.16.0
+   github.com/aws/aws-sdk-go-v2 v1.23.0
+   github.com/aws/aws-sdk-go-v2/config v1.25.1
+   github.com/aws/aws-sdk-go-v2/credentials v1.16.1
+   github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.13.8
+   github.com/aws/aws-sdk-go-v2/service/s3 v1.42.2
+   github.com/aws/smithy-go v1.17.0
github.com/docker/go-connections v0.4.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
@@ -105,19 +105,19 @@ require (
github.com/apache/arrow/go/v12 v12.0.0 // indirect
github.com/apache/thrift v0.16.0 // indirect
github.com/aws/aws-sdk-go v1.34.0 // indirect
-   github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.0 // indirect
-   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.2 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.1 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.1 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/ini v1.5.0 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.1 // indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.0 
// indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.1 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.1 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.1 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.17.0 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.19.0 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.25.0 // indirect
+   github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.1 // indirect
+   github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.4 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.3 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.3 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/ini v1.7.0 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.3 // indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.1 
// indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.3 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.3 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.3 // 
indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.17.2 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.19.2 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.25.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index a393569c4b9..4a5d989dbe9 100644
--- a/sdks/go.sum
+++ b

(beam) branch master updated: [RRIO] [testing] Downgrade mock-apis grpcio-tools dependencies to remove Beam SDK version conflict (#29320)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 29c0b41afec [RRIO] [testing] Downgrade mock-apis grpcio-tools 
dependencies to remove Beam SDK version conflict  (#29320)
29c0b41afec is described below

commit 29c0b41afeca1cb0fb97362a40d869d46735593a
Author: Damon 
AuthorDate: Mon Nov 6 16:58:16 2023 -0800

[RRIO] [testing] Downgrade mock-apis grpcio-tools dependencies to remove 
Beam SDK version conflict  (#29320)

* Constrain grpc* versons to >=1.53.0

* Downgrade grpcio dependencies

* Bump project version

* Make version constraint >=

* Add poetry.lock to rat
---
 .test-infra/mock-apis/poetry.lock| 19 +--
 .test-infra/mock-apis/pyproject.toml |  6 +++---
 build.gradle.kts |  3 +++
 3 files changed, 7 insertions(+), 21 deletions(-)

diff --git a/.test-infra/mock-apis/poetry.lock 
b/.test-infra/mock-apis/poetry.lock
index e9bcdbb4750..b36baff7a74 100644
--- a/.test-infra/mock-apis/poetry.lock
+++ b/.test-infra/mock-apis/poetry.lock
@@ -1,20 +1,3 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements.  See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
-#
-#http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
 # This file is automatically @generated by Poetry 1.7.0 and should not be 
changed by hand.
 
 [[package]]
@@ -233,4 +216,4 @@ files = [
 [metadata]
 lock-version = "2.0"
 python-versions = "^3.8"
-content-hash = 
"1f2dadd2821a62cdfb7562f211be17c8bd12b762551bdb923954f9e7404087ec"
+content-hash = 
"35ed5a98dd3f951bbfc44b949ad9148634159976cb54ac6f257d119c12d9d924"
diff --git a/.test-infra/mock-apis/pyproject.toml 
b/.test-infra/mock-apis/pyproject.toml
index ed3f035cbf4..680bf489ba1 100644
--- a/.test-infra/mock-apis/pyproject.toml
+++ b/.test-infra/mock-apis/pyproject.toml
@@ -17,7 +17,7 @@
 
 [tool.poetry]
 name = "mock-apis"
-version = "0.1.0"
+version = "0.1.1"
 authors = ["Ritesh Ghorse , Damon Douglas <>"]
 license = "Apache-2.0"
 description = ""
@@ -29,8 +29,8 @@ packages = [
 [tool.poetry.dependencies]
 python = "^3.8"
 google = "^3.0.0"
-grpcio = "^1.59.2"
-grpcio-tools = "^1.59.2"
+grpcio = "^1.53.0"
+grpcio-tools = "^1.53.0"
 
 
 [build-system]
diff --git a/build.gradle.kts b/build.gradle.kts
index b330bd07861..59161809f37 100644
--- a/build.gradle.kts
+++ b/build.gradle.kts
@@ -202,6 +202,9 @@ tasks.rat {
 
 // Ignore buf autogenerated files.
 "**/buf.lock",
+
+// Ignore poetry autogenerated files.
+"**/poetry.lock",
   )
 
   // Add .gitignore excludes to the Apache Rat exclusion list. We re-create 
the behavior

(beam) branch master updated: Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 in /sdks (#29308)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 34b93c0d10d Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 in /sdks 
(#29308)
34b93c0d10d is described below

commit 34b93c0d10ddd67d081b401a59a2f77be0ad5a89
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Nov 6 09:19:34 2023 -0500

Bump github.com/spf13/cobra from 1.7.0 to 1.8.0 in /sdks (#29308)

Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.7.0 
to 1.8.0.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 5618da7ec14..0ee2c233dab 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -46,7 +46,7 @@ require (
github.com/lib/pq v1.10.9
github.com/linkedin/goavro/v2 v2.12.0
github.com/proullon/ramsql v0.1.3
-   github.com/spf13/cobra v1.7.0
+   github.com/spf13/cobra v1.8.0
github.com/testcontainers/testcontainers-go v0.25.0
github.com/tetratelabs/wazero v1.5.0
github.com/xitongsys/parquet-go v1.6.2
diff --git a/sdks/go.sum b/sdks/go.sum
index dd53a66dfa6..6343b568926 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -158,7 +158,7 @@ github.com/coreos/go-systemd/v22 v22.3.2/go.mod 
h1:Y58oyj3AT4RCenI/lSvhwexgC+NSV
 github.com/cpuguy83/dockercfg v0.3.1 
h1:/FpZ+JaygUR/lZP2NlFI2DVfrOEMAIKP5wWEJdoYe9E=
 github.com/cpuguy83/dockercfg v0.3.1/go.mod 
h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
 github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod 
h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
-github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod 
h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
+github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod 
h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/creack/pty v1.1.9/go.mod 
h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY=
 github.com/cyphar/filepath-securejoin v0.2.3/go.mod 
h1:aPGpWjXOXUn2NCNjFvBE6aRxGGx79pTxQpKOJNYHHl4=
@@ -408,8 +408,8 @@ github.com/sirupsen/logrus v1.9.3 
h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ
 github.com/sirupsen/logrus v1.9.3/go.mod 
h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
 github.com/spf13/afero v1.2.1/go.mod 
h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
 github.com/spf13/afero v1.2.2/go.mod 
h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
-github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
-github.com/spf13/cobra v1.7.0/go.mod 
h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
+github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
+github.com/spf13/cobra v1.8.0/go.mod 
h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod 
h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/objx v0.1.0/go.mod 
h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=

(beam) branch master updated: [RRIO][Python] Add python grpc binding for mock API (#29129)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new cc2e63ea75a [RRIO][Python] Add python grpc binding for mock API 
(#29129)
cc2e63ea75a is described below

commit cc2e63ea75a6ff31fc6c97c5feb63191e81ebf53
Author: Ritesh Ghorse 
AuthorDate: Fri Nov 3 14:54:54 2023 -0400

[RRIO][Python] Add python grpc binding for mock API (#29129)

* add python grpc binding for mock api

* add licenses

* update python version for buf plugin

* add poetry for python dependencies

* add license

* del author,desc

* update package for pyproject

* update python path
---
 .test-infra/mock-apis/README.md|   1 +
 .test-infra/mock-apis/buf.gen.yaml |   6 +-
 .test-infra/mock-apis/poetry.lock  | 236 +
 .test-infra/mock-apis/pyproject.toml   |  38 
 .../src/main/python/proto/echo/v1/echo_pb2.py  |  47 
 .../src/main/python/proto/echo/v1/echo_pb2_grpc.py |  87 
 6 files changed, 414 insertions(+), 1 deletion(-)

diff --git a/.test-infra/mock-apis/README.md b/.test-infra/mock-apis/README.md
index 9c4911a0d63..ec94eb45a19 100644
--- a/.test-infra/mock-apis/README.md
+++ b/.test-infra/mock-apis/README.md
@@ -62,6 +62,7 @@ TODO: See https://github.com/apache/beam/issues/28859
 | [go](https://go.dev)| For making code 
changes in this directory. See [go.mod](go.mod) for required version.  |
 | [buf](https://github.com/bufbuild/buf#installation) | Optional for when 
making changes to proto. |
 | [ko](https://ko.build/install/) | To easily build Go 
container images.   |
+| [poetry](https://python-poetry.org/) | To manage python 
dependencies.   |
 
 # Testing
 
diff --git a/.test-infra/mock-apis/buf.gen.yaml 
b/.test-infra/mock-apis/buf.gen.yaml
index 31e57ff2da1..e5f6e51c14d 100644
--- a/.test-infra/mock-apis/buf.gen.yaml
+++ b/.test-infra/mock-apis/buf.gen.yaml
@@ -37,4 +37,8 @@ plugins:
 - name: java
   out: src/main/java
 - name: grpc-java
-  out: src/main/java
\ No newline at end of file
+  out: src/main/java
+- plugin: buf.build/protocolbuffers/python:v24.4
+  out: src/main/python
+- plugin: buf.build/grpc/python:v1.59.1
+  out: src/main/python
\ No newline at end of file
diff --git a/.test-infra/mock-apis/poetry.lock 
b/.test-infra/mock-apis/poetry.lock
new file mode 100644
index 000..e9bcdbb4750
--- /dev/null
+++ b/.test-infra/mock-apis/poetry.lock
@@ -0,0 +1,236 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# This file is automatically @generated by Poetry 1.7.0 and should not be 
changed by hand.
+
+[[package]]
+name = "beautifulsoup4"
+version = "4.12.2"
+description = "Screen-scraping library"
+optional = false
+python-versions = ">=3.6.0"
+files = [
+{file = "beautifulsoup4-4.12.2-py3-none-any.whl", hash = 
"sha256:bd2520ca0d9d7d12694a53d44ac482d181b4ec1888909b035a3dbf40d0f57d4a"},
+{file = "beautifulsoup4-4.12.2.tar.gz", hash = 
"sha256:492bbc69dca35d12daac71c4db1bfff0c876c00ef4a2ffacce226d4638eb72da"},
+]
+
+[package.dependencies]
+soupsieve = ">1.2"
+
+[package.extras]
+html5lib = ["html5lib"]
+lxml = ["lxml"]
+
+[[package]]
+name = "google"
+version = "3.0.0"
+description = "Python bindings to the Google search engine."
+optional = false
+python-versions = "*"
+files = [
+{file = "google-3.0.0-py2.py3-none-any.whl", hash = 
"sha256:889cf695f84e4ae2c55fbc0cfdaf4c1e729417fa52ab1db0485202ba173e4935"},
+{file = "google-3.0.0.tar.gz", hash = 
"sha256:143530122ee5130509ad5e989f0512f7cb218b2d4eddbafbad40fd10e8d8ccbe"},
+]
+
+[package.dependencies]
+beautifulsoup4 = &qu

(beam) branch master updated (04df11c8fc1 -> f25c441111f)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 04df11c8fc1 Pass empty_match_treatment flag (#29251)
 add f25c44f Bump cloud.google.com/go/bigquery from 1.57.0 to 1.57.1 in 
/sdks (#29286)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

(beam) branch master updated (3d3a7afe0df -> 0113befc701)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 3d3a7afe0df Bugfix/kafka nullable header coders (#29244)
 add 0113befc701 Bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 
1.20.0 in /sdks (#29240)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 24 
 sdks/go.sum | 37 -
 2 files changed, 36 insertions(+), 25 deletions(-)

(beam) branch master updated: [Java][Dataflow Runner] Add precondition to make sure userAgent name and version are not empty (#29157)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 71a74b9e9be [Java][Dataflow Runner] Add precondition to make sure 
userAgent name and version are not empty (#29157)
71a74b9e9be is described below

commit 71a74b9e9be3bfbad11231b043819e7cea969f53
Author: Andrei Gurau 
AuthorDate: Fri Oct 27 10:39:16 2023 -0400

[Java][Dataflow Runner] Add precondition to make sure userAgent name and 
version are not empty (#29157)
---
 .../java/org/apache/beam/runners/dataflow/DataflowRunner.java | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
index 80b4e4cfd8b..6449053194f 100644
--- 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
+++ 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
@@ -403,11 +403,14 @@ public class DataflowRunner extends 
PipelineRunner {
 }
 
 DataflowRunnerInfo dataflowRunnerInfo = 
DataflowRunnerInfo.getDataflowRunnerInfo();
+String userAgentName = dataflowRunnerInfo.getName();
+Preconditions.checkArgument(
+!userAgentName.equals(""), "Dataflow runner's `name` property cannot 
be empty.");
+String userAgentVersion = dataflowRunnerInfo.getVersion();
+Preconditions.checkArgument(
+!userAgentVersion.equals(""), "Dataflow runner's `version` property 
cannot be empty.");
 String userAgent =
-String.format(
-"%s/%s%s",
-dataflowRunnerInfo.getName(), dataflowRunnerInfo.getVersion(), 
agentJavaVer)
-.replace(" ", "_");
+String.format("%s/%s%s", userAgentName, userAgentVersion, 
agentJavaVer).replace(" ", "_");
 dataflowOptions.setUserAgent(userAgent);
 
 return new DataflowRunner(dataflowOptions);

(beam) branch master updated: Bump github.com/google/uuid from 1.3.1 to 1.4.0 in /sdks (#29159)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new db664d8125b Bump github.com/google/uuid from 1.3.1 to 1.4.0 in /sdks 
(#29159)
db664d8125b is described below

commit db664d8125b3ed6cf989d9586c98df8dcdb585f0
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Oct 27 09:50:02 2023 -0400

Bump github.com/google/uuid from 1.3.1 to 1.4.0 in /sdks (#29159)

Bumps [github.com/google/uuid](https://github.com/google/uuid) from 1.3.1 
to 1.4.0.
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.3.1...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 6cd7b77be91..49968e3227d 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -41,7 +41,7 @@ require (
github.com/go-sql-driver/mysql v1.7.1
github.com/golang/protobuf v1.5.3 // TODO(danoliveira): Fully replace 
this with google.golang.org/protobuf
github.com/google/go-cmp v0.6.0
-   github.com/google/uuid v1.3.1
+   github.com/google/uuid v1.4.0
github.com/johannesboyne/gofakes3 v0.0.0-20221110173912-32fb85c5aed6
github.com/lib/pq v1.10.9
github.com/linkedin/goavro/v2 v2.12.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 27a55827c72..05fc3076eb2 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -278,8 +278,8 @@ github.com/google/s2a-go v0.1.7 
h1:60BLSyTrOV4/haCDW4zb1guZItoSq8foHCXrAnjBo/o=
 github.com/google/s2a-go v0.1.7/go.mod 
h1:50CgR4k1jNlWBu4UfS4AcfhVe1r6pdZPygJ3R8F0Qdw=
 github.com/google/uuid v1.1.2/go.mod 
h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/google/uuid v1.2.0/go.mod 
h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
-github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
-github.com/google/uuid v1.3.1/go.mod 
h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/google/uuid v1.4.0 h1:MtMxsa51/r9yyhkyLsVeVt0B+BGQZzpQiTQ4eHZ8bc4=
+github.com/google/uuid v1.4.0/go.mod 
h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
 github.com/googleapis/enterprise-certificate-proxy v0.3.1 
h1:SBWmZhjUDRorQxrN0nwzf+AHBxnbFjViHQS4P0yVpmQ=
 github.com/googleapis/enterprise-certificate-proxy v0.3.1/go.mod 
h1:VLSiSSBs/ksPL8kq3OBOQ6WRI2QnaFynd1DCjZ62+V0=
 github.com/googleapis/gax-go/v2 v2.0.4/go.mod 
h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=

[beam-starter-go] branch main updated (c34fa8f -> 5c52ca6)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git


from c34fa8f  Bump golang.org/x/net from 0.14.0 to 0.17.0 (#22)
 add 2e48d6c  Bump google.golang.org/grpc from 1.58.1 to 1.58.3
 new 5c52ca6  Merge pull request #24 from 
apache/dependabot/go_modules/google.golang.org/grpc-1.58.3

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

[beam-starter-go] 01/01: Merge pull request #24 from apache/dependabot/go_modules/google.golang.org/grpc-1.58.3

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/beam-starter-go.git

commit 5c52ca6babe5c27bacbe656b424441dc387d0ade
Merge: c34fa8f 2e48d6c
Author: Ritesh Ghorse 
AuthorDate: Thu Oct 26 09:27:34 2023 -0400

Merge pull request #24 from 
apache/dependabot/go_modules/google.golang.org/grpc-1.58.3

Bump google.golang.org/grpc from 1.58.1 to 1.58.3

 go.mod | 2 +-
 go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

[beam] branch master updated: Remove warning in dataframes io. (#29084)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 7610d46f745 Remove warning in dataframes io. (#29084)
7610d46f745 is described below

commit 7610d46f7456ccd7c311aed4c21105fcce27b7b5
Author: Robert Bradshaw 
AuthorDate: Fri Oct 20 06:17:11 2023 -0700

Remove warning in dataframes io. (#29084)
---
 sdks/python/apache_beam/dataframe/io.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/sdks/python/apache_beam/dataframe/io.py 
b/sdks/python/apache_beam/dataframe/io.py
index eb1b1b5ec15..fedc40c6071 100644
--- a/sdks/python/apache_beam/dataframe/io.py
+++ b/sdks/python/apache_beam/dataframe/io.py
@@ -281,8 +281,9 @@ class _ReadFromPandas(beam.PTransform):
   if not self.binary:
 handle = TextIOWrapper(handle)
   if self.incremental:
-sample = next(
-self.reader(handle, *self.args, **dict(self.kwargs, 
chunksize=100)))
+with self.reader(handle, *self.args, **dict(self.kwargs,
+chunksize=100)) as stream:
+  sample = next(stream)
   else:
 sample = self.reader(handle, *self.args, **self.kwargs)
 

[beam] branch master updated: Bump google.golang.org/api from 0.147.0 to 0.148.0 in /sdks (#29086)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 641b95df45d Bump google.golang.org/api from 0.147.0 to 0.148.0 in 
/sdks (#29086)
641b95df45d is described below

commit 641b95df45d138cff32f5a43f00d174b7cd1eef1
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Oct 20 09:16:14 2023 -0400

Bump google.golang.org/api from 0.147.0 to 0.148.0 in /sdks (#29086)

Bumps 
[google.golang.org/api](https://github.com/googleapis/google-api-go-client) 
from 0.147.0 to 0.148.0.
- [Release 
notes](https://github.com/googleapis/google-api-go-client/releases)
- 
[Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- 
[Commits](https://github.com/googleapis/google-api-go-client/compare/v0.147.0...v0.148.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 4 ++--
 sdks/go.sum | 8 
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 686a399bd80..b990ee33ea0 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -57,7 +57,7 @@ require (
golang.org/x/sync v0.4.0
golang.org/x/sys v0.13.0
golang.org/x/text v0.13.0
-   google.golang.org/api v0.147.0
+   google.golang.org/api v0.148.0
google.golang.org/genproto v0.0.0-20231002182017-d307bd883b97
google.golang.org/grpc v1.59.0
google.golang.org/protobuf v1.31.0
@@ -171,5 +171,5 @@ require (
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto/googleapis/api 
v0.0.0-20231002182017-d307bd883b97 // indirect
-   google.golang.org/genproto/googleapis/rpc 
v0.0.0-20231009173412-8bfb1ae86b6c // indirect
+   google.golang.org/genproto/googleapis/rpc 
v0.0.0-20231012201019-e917dd12ba7a // indirect
 )
diff --git a/sdks/go.sum b/sdks/go.sum
index 6428fbd82c9..d4dd5e1ae8e 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -671,8 +671,8 @@ google.golang.org/api v0.14.0/go.mod 
h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsb
 google.golang.org/api v0.15.0/go.mod 
h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
 google.golang.org/api v0.17.0/go.mod 
h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
 google.golang.org/api v0.18.0/go.mod 
h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
-google.golang.org/api v0.147.0 h1:Can3FaQo9LlVqxJCodNmeZW/ib3/qKAY3rFeXiHo5gc=
-google.golang.org/api v0.147.0/go.mod 
h1:pQ/9j83DcmPd/5C9e2nFOdjjNkDZ1G+zkbK2uvdkJMs=
+google.golang.org/api v0.148.0 h1:HBq4TZlN4/1pNcu0geJZ/Q50vIwIXT532UIMYoo0vOs=
+google.golang.org/api v0.148.0/go.mod 
h1:8/TBgwaKjfqTdacOJrOv2+2Q6fBDU1uHKK06oGSkxzU=
 google.golang.org/appengine v1.1.0/go.mod 
h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
 google.golang.org/appengine v1.4.0/go.mod 
h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
 google.golang.org/appengine v1.5.0/go.mod 
h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -702,8 +702,8 @@ google.golang.org/genproto 
v0.0.0-20231002182017-d307bd883b97 h1:SeZZZx0cP0fqUyA
 google.golang.org/genproto v0.0.0-20231002182017-d307bd883b97/go.mod 
h1:t1VqOqqvce95G3hIDCT5FeO3YUc6Q4Oe24L/+rNMxRk=
 google.golang.org/genproto/googleapis/api v0.0.0-20231002182017-d307bd883b97 
h1:W18sezcAYs+3tDZX4F80yctqa12jcP1PUS2gQu1zTPU=
 google.golang.org/genproto/googleapis/api 
v0.0.0-20231002182017-d307bd883b97/go.mod 
h1:iargEX0SFPm3xcfMI0d1domjg0ZF4Aa0p2awqyxhvF0=
-google.golang.org/genproto/googleapis/rpc v0.0.0-20231009173412-8bfb1ae86b6c 
h1:jHkCUWkseRf+W+edG5hMzr/Uh1xkDREY4caybAq4dpY=
-google.golang.org/genproto/googleapis/rpc 
v0.0.0-20231009173412-8bfb1ae86b6c/go.mod 
h1:4cYg8o5yUbm77w8ZX00LhMVNl/YVBFJRYWDc0uYWMs0=
+google.golang.org/genproto/googleapis/rpc v0.0.0-20231012201019-e917dd12ba7a 
h1:a2MQQVoTo96JC9PMGtGBymLp7+/RzpFc2yX/9WfFg1c=
+google.golang.org/genproto/googleapis/rpc 
v0.0.0-20231012201019-e917dd12ba7a/go.mod 
h1:4cYg8o5yUbm77w8ZX00LhMVNl/YVBFJRYWDc0uYWMs0=
 google.golang.org/grpc v1.19.0/go.mod 
h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
 google.golang.org/grpc v1.20.1/go.mod 
h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
 google.golang.org/grpc v1.21.1/go.mod 
h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=

[beam] branch master updated: Bump urllib3 from 1.26.17 to 1.26.18 in /sdks/python/container/py310 (#29045)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 9745ea22497 Bump urllib3 from 1.26.17 to 1.26.18 in 
/sdks/python/container/py310 (#29045)
9745ea22497 is described below

commit 9745ea224976af888049d5553a7190f9c5877fbd
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Oct 18 11:11:47 2023 -0400

Bump urllib3 from 1.26.17 to 1.26.18 in /sdks/python/container/py310 
(#29045)

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.17 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.17...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/python/container/py310/base_image_requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/python/container/py310/base_image_requirements.txt 
b/sdks/python/container/py310/base_image_requirements.txt
index 548c5637170..82b210005fe 100644
--- a/sdks/python/container/py310/base_image_requirements.txt
+++ b/sdks/python/container/py310/base_image_requirements.txt
@@ -134,7 +134,7 @@ tqdm==4.66.1
 typing_extensions==4.8.0
 tzlocal==5.0.1
 uritemplate==4.1.1
-urllib3==1.26.17
+urllib3==1.26.18
 websocket-client==1.6.3
 wrapt==1.15.0
 zstandard==0.21.0

[beam] branch master updated: Bump google.golang.org/grpc from 1.58.3 to 1.59.0 in /sdks (#29047)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 574255a0411 Bump google.golang.org/grpc from 1.58.3 to 1.59.0 in /sdks 
(#29047)
574255a0411 is described below

commit 574255a04116a7b188393a08e7d1e9c94d412b5c
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Oct 18 09:19:50 2023 -0400

Bump google.golang.org/grpc from 1.58.3 to 1.59.0 in /sdks (#29047)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.58.3 
to 1.59.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.58.3...v1.59.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 51b6ac696e9..192802f77ea 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -59,7 +59,7 @@ require (
golang.org/x/text v0.13.0
google.golang.org/api v0.147.0
google.golang.org/genproto v0.0.0-20231002182017-d307bd883b97
-   google.golang.org/grpc v1.58.3
+   google.golang.org/grpc v1.59.0
google.golang.org/protobuf v1.31.0
gopkg.in/retry.v1 v1.0.3
gopkg.in/yaml.v2 v2.4.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 7c3af169cee..457fe2bf1ff 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -713,8 +713,8 @@ google.golang.org/grpc v1.26.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8
 google.golang.org/grpc v1.27.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.27.1/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.33.2/go.mod 
h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.58.3 h1:BjnpXut1btbtgN/6sp+brB2Kbm2LjNXnidYujAVbSoQ=
-google.golang.org/grpc v1.58.3/go.mod 
h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
+google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
+google.golang.org/grpc v1.59.0/go.mod 
h1:aUPDwccQo6OTjy7Hct4AfBPD1GptF4fyUjIkQ9YtF98=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod 
h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod 
h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod 
h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=

[beam] branch master updated (8de14655f78 -> 6b32a3fb7d9)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 8de14655f78 [RRIO] Stub the RequestResponseIO transform (#28950)
 add 6b32a3fb7d9 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in 
/sdks (#29002)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 30 +++---
 sdks/go.sum | 61 ++---
 2 files changed, 45 insertions(+), 46 deletions(-)

[beam] branch master updated (a342d2817a9 -> 2dc766512c0)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from a342d2817a9 Github Workflow Replacement for Jenkins Jobs, 
beam_LoadTests_Python_GBK_Flink_Batch; beam_Java_LoadTests_Combine_Smoke_PR 
(#28897)
 add 2dc766512c0 Bump github.com/aws/aws-sdk-go-v2 from 1.21.1 to 1.21.2 in 
/sdks (#28978)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 2 +-
 sdks/go.sum | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

[beam] branch master updated: Bump google.golang.org/grpc from 1.58.2 to 1.58.3 in /sdks (#28938)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new e3c1591bbb8 Bump google.golang.org/grpc from 1.58.2 to 1.58.3 in /sdks 
(#28938)
e3c1591bbb8 is described below

commit e3c1591bbb8c2b91313267fcd39d77720c768b75
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Wed Oct 11 09:27:52 2023 -0400

Bump google.golang.org/grpc from 1.58.2 to 1.58.3 in /sdks (#28938)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.58.2 
to 1.58.3.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.58.2...v1.58.3)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 0bc60b630bf..761798d57c9 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -59,7 +59,7 @@ require (
golang.org/x/text v0.13.0
google.golang.org/api v0.146.0
google.golang.org/genproto v0.0.0-20230913181813-007df8e322eb
-   google.golang.org/grpc v1.58.2
+   google.golang.org/grpc v1.58.3
google.golang.org/protobuf v1.31.0
gopkg.in/retry.v1 v1.0.3
gopkg.in/yaml.v2 v2.4.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 7743b2bc324..ac862972b2a 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -712,8 +712,8 @@ google.golang.org/grpc v1.26.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8
 google.golang.org/grpc v1.27.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.27.1/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.33.2/go.mod 
h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.58.2 h1:SXUpjxeVF3FKrTYQI4f4KvbGD5u2xccdYdurwowix5I=
-google.golang.org/grpc v1.58.2/go.mod 
h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
+google.golang.org/grpc v1.58.3 h1:BjnpXut1btbtgN/6sp+brB2Kbm2LjNXnidYujAVbSoQ=
+google.golang.org/grpc v1.58.3/go.mod 
h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod 
h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod 
h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod 
h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=

[beam] branch master updated: remove unused os import (#28914)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new e01de664594 remove unused os import (#28914)
e01de664594 is described below

commit e01de66459416f8f40dc3fbe64243fef9694ffc3
Author: Ritesh Ghorse 
AuthorDate: Tue Oct 10 10:42:23 2023 -0400

remove unused os import (#28914)
---
 sdks/go/test/integration/integration.go | 1 -
 1 file changed, 1 deletion(-)

diff --git a/sdks/go/test/integration/integration.go 
b/sdks/go/test/integration/integration.go
index d3eb1c3ee7c..5b7473fb561 100644
--- a/sdks/go/test/integration/integration.go
+++ b/sdks/go/test/integration/integration.go
@@ -38,7 +38,6 @@ package integration
 import (
"fmt"
"math/rand"
-   "os"
"regexp"
"strings"
"testing"

[beam] branch master updated: Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 in /sdks (#28912)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 152f9c52b57 Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 in /sdks 
(#28912)
152f9c52b57 is described below

commit 152f9c52b57e8cc3c7d791ab787217c333e71f4d
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Oct 10 10:17:26 2023 -0400

Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 in /sdks (#28912)

Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.12.0 
to 0.13.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index a2b5ed09764..53eae7db453 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -53,7 +53,7 @@ require (
github.com/xitongsys/parquet-go-source 
v0.0.0-20220315005136-aec0fe3e777c
go.mongodb.org/mongo-driver v1.12.1
golang.org/x/net v0.16.0
-   golang.org/x/oauth2 v0.12.0
+   golang.org/x/oauth2 v0.13.0
golang.org/x/sync v0.4.0
golang.org/x/sys v0.13.0
golang.org/x/text v0.13.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 516adec9387..5dc085b859a 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -548,8 +548,8 @@ golang.org/x/oauth2 
v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4Iltr
 golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
 golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod 
h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.12.0 h1:smVPGxink+n1ZI5pkQa8y6fZT0RW0MgCO5bFpepy4B4=
-golang.org/x/oauth2 v0.12.0/go.mod 
h1:A74bZ3aGXgCY0qaIC9Ahg6Lglin4AMAco8cIv9baba4=
+golang.org/x/oauth2 v0.13.0 h1:jDDenyj+WgFtmV3zYVoi8aE2BwtXFLWOA67ZfNWftiY=
+golang.org/x/oauth2 v0.13.0/go.mod 
h1:/JMhi4ZRXAf4HG9LiNmxvk+45+96RUlVThiH8FzNBn0=
 golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod 
h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=

[beam] branch master updated: Catch EOFError along with StopIteration (#28896)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 21f822f6580 Catch EOFError along with StopIteration (#28896)
21f822f6580 is described below

commit 21f822f6580f04c3bd581005d93846e555982367
Author: Ritesh Ghorse 
AuthorDate: Mon Oct 9 14:14:07 2023 -0400

Catch EOFError along with StopIteration (#28896)

* catch EOF instead of StopIteration

* add stop iteration to close

* catch both stop iteration and EOF
---
 sdks/python/apache_beam/io/gcp/bigquery.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/python/apache_beam/io/gcp/bigquery.py 
b/sdks/python/apache_beam/io/gcp/bigquery.py
index 986919fd6b8..184138af752 100644
--- a/sdks/python/apache_beam/io/gcp/bigquery.py
+++ b/sdks/python/apache_beam/io/gcp/bigquery.py
@@ -1310,7 +1310,7 @@ class _ReadReadRowsResponsesWithFastAvro():
   def __next__(self):
 try:
   return fastavro.schemaless_reader(self.bytes_reader, self.avro_schema)
-except StopIteration:
+except (StopIteration, EOFError):
   self.read_rows_response = next(self.read_rows_iterator, None)
   if self.read_rows_response is not None:
 self.bytes_reader = io.BytesIO(

[beam] branch master updated: Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 in /sdks (#28891)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 92d5bf87796 Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 in 
/sdks (#28891)
92d5bf87796 is described below

commit 92d5bf877969cd00a229a97d35575f7f5d1fc09b
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Oct 9 09:10:00 2023 -0400

Bump github.com/aws/aws-sdk-go-v2 from 1.21.0 to 1.21.1 in /sdks (#28891)

Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) 
from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.21.0...v1.21.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 4 ++--
 sdks/go.sum | 6 --
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index d1c98259f1e..b8dd03de6b0 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -30,12 +30,12 @@ require (
cloud.google.com/go/pubsub v1.33.0
cloud.google.com/go/spanner v1.50.0
cloud.google.com/go/storage v1.33.0
-   github.com/aws/aws-sdk-go-v2 v1.21.0
+   github.com/aws/aws-sdk-go-v2 v1.21.1
github.com/aws/aws-sdk-go-v2/config v1.18.43
github.com/aws/aws-sdk-go-v2/credentials v1.13.41
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.87
github.com/aws/aws-sdk-go-v2/service/s3 v1.40.0
-   github.com/aws/smithy-go v1.14.2
+   github.com/aws/smithy-go v1.15.0
github.com/docker/go-connections v0.4.0
github.com/dustin/go-humanize v1.0.1
github.com/go-sql-driver/mysql v1.7.1
diff --git a/sdks/go.sum b/sdks/go.sum
index 1c39718964d..f177dd8debb 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -81,8 +81,9 @@ github.com/aws/aws-sdk-go v1.30.19/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZve
 github.com/aws/aws-sdk-go v1.34.0 
h1:brux2dRrlwCF5JhTL7MUT3WUwo9zfDHZZp3+g3Mvlmo=
 github.com/aws/aws-sdk-go v1.34.0/go.mod 
h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
 github.com/aws/aws-sdk-go-v2 v1.7.1/go.mod 
h1:L5LuPC1ZgDr2xQS7AmIec/Jlc7O/Y1u2KxJyNVab250=
-github.com/aws/aws-sdk-go-v2 v1.21.0 
h1:gMT0IW+03wtYJhRqTVYn0wLzwdnK9sRMcxmtfGzRdJc=
 github.com/aws/aws-sdk-go-v2 v1.21.0/go.mod 
h1:/RfNgGmRxI+iFOB1OeJUyxiU+9s88k3pfHvDagGEp0M=
+github.com/aws/aws-sdk-go-v2 v1.21.1 
h1:wjHYshtPpYOZm+/mu3NhVgRRc0baM6LJZOmxPZ5Cwzs=
+github.com/aws/aws-sdk-go-v2 v1.21.1/go.mod 
h1:ErQhvNuEMhJjweavOYhxVkn2RUx7kQXVATHrjKtxIpM=
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13 
h1:OPLEkmhXf6xFPiz0bLeDArZIDx1NNS4oJyG4nv3Gct0=
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13/go.mod 
h1:gpAbvyDGQFozTEmlTFO8XcQKHzubdq0LzRyJpG6MiXM=
 github.com/aws/aws-sdk-go-v2/config v1.5.0/go.mod 
h1:RWlPOAW3E3tbtNAqTwvSW54Of/yP3oiZXMI0xfUdjyA=
@@ -133,8 +134,9 @@ github.com/aws/aws-sdk-go-v2/service/sts v1.22.0/go.mod 
h1:VC7JDqsqiwXukYEDjoHh9
 github.com/aws/aws-sdk-go-v2/service/sts v1.23.0 
h1:pyvfUqkNLMipdKNAtu7OVbRxUrR2BMaKccIPpk/Hkak=
 github.com/aws/aws-sdk-go-v2/service/sts v1.23.0/go.mod 
h1:VC7JDqsqiwXukYEDjoHh9U0fOJtNWh04FPQz4ct4GGU=
 github.com/aws/smithy-go v1.6.0/go.mod 
h1:SObp3lf9smib00L/v3U2eAKG8FyQ7iLrJnQiAmR5n+E=
-github.com/aws/smithy-go v1.14.2 
h1:MJU9hqBGbvWZdApzpvoF2WAIJDbtjK2NDJSiJP7HblQ=
 github.com/aws/smithy-go v1.14.2/go.mod 
h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
+github.com/aws/smithy-go v1.15.0 
h1:PS/durmlzvAFpQHDs4wi4sNNP9ExsqZh6IlfdHXgKK8=
+github.com/aws/smithy-go v1.15.0/go.mod 
h1:Tg+OJXh4MB2R/uN61Ko2f6hTZwB/ZYGOtib8J3gBHzA=
 github.com/cenkalti/backoff/v4 v4.2.1 
h1:y4OZtCnogmCPw98Zjyt5a6+QwPLGkiQsYW5oUqylYbM=
 github.com/cenkalti/backoff/v4 v4.2.1/go.mod 
h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
 github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod 
h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=

[beam] branch master updated (7462069f8ce -> 2be0605aeb7)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 7462069f8ce Github Workflow Replacement for Jenkins Jobs, 
beam_LoadTests_Python_GBK_Dataflow_* (#28827)
 add 2be0605aeb7 Bump github.com/proullon/ramsql from 0.1.2 to 0.1.3 in 
/sdks (#28857)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

[beam] branch master updated: Bump cloud.google.com/go/spanner from 1.49.0 to 1.50.0 in /sdks (#28838)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 0b938306d13 Bump cloud.google.com/go/spanner from 1.49.0 to 1.50.0 in 
/sdks (#28838)
0b938306d13 is described below

commit 0b938306d135bbaadb3123daf0455f67baee4a9a
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Thu Oct 5 09:42:10 2023 -0400

Bump cloud.google.com/go/spanner from 1.49.0 to 1.50.0 in /sdks (#28838)

Bumps 
[cloud.google.com/go/spanner](https://github.com/googleapis/google-cloud-go) 
from 1.49.0 to 1.50.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- 
[Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- 
[Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.49.0...spanner/v1.50.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/spanner
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index e17427227eb..89f119a5490 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -28,7 +28,7 @@ require (
cloud.google.com/go/datastore v1.14.0
cloud.google.com/go/profiler v0.3.1
cloud.google.com/go/pubsub v1.33.0
-   cloud.google.com/go/spanner v1.49.0
+   cloud.google.com/go/spanner v1.50.0
cloud.google.com/go/storage v1.33.0
github.com/aws/aws-sdk-go-v2 v1.21.0
github.com/aws/aws-sdk-go-v2/config v1.18.43
diff --git a/sdks/go.sum b/sdks/go.sum
index 71c1c4545c8..582f3d69f88 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -38,8 +38,8 @@ cloud.google.com/go/pubsub v1.1.0/go.mod 
h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+
 cloud.google.com/go/pubsub v1.2.0/go.mod 
h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
 cloud.google.com/go/pubsub v1.33.0 
h1:6SPCPvWav64tj0sVX/+npCBKhUi/UjJehy9op/V3p2g=
 cloud.google.com/go/pubsub v1.33.0/go.mod 
h1:f+w71I33OMyxf9VpMVcZbnG5KSUkCOUHYpFd5U1GdRc=
-cloud.google.com/go/spanner v1.49.0 
h1:+HY8C4uztU7XyLz3xMi/LCXdetLEOExhvRFJu2NiVXM=
-cloud.google.com/go/spanner v1.49.0/go.mod 
h1:eGj9mQGK8+hkgSVbHNQ06pQ4oS+cyc4tXXd6Dif1KoM=
+cloud.google.com/go/spanner v1.50.0 
h1:QrJFOpaxCXdXF+GkiruLz642PHxkdj68PbbnLw3O2Zw=
+cloud.google.com/go/spanner v1.50.0/go.mod 
h1:eGj9mQGK8+hkgSVbHNQ06pQ4oS+cyc4tXXd6Dif1KoM=
 cloud.google.com/go/storage v1.0.0/go.mod 
h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
 cloud.google.com/go/storage v1.5.0/go.mod 
h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
 cloud.google.com/go/storage v1.6.0/go.mod 
h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=

[beam] branch master updated (12b07140203 -> c01b41f9758)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 12b07140203 Add notebook for per key models (#28327)
 add c01b41f9758 Bump urllib3 from 1.26.16 to 1.26.17 in 
/sdks/python/container/py310 (#28785)

No new revisions were added by this update.

Summary of changes:
 sdks/python/container/py310/base_image_requirements.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[beam] branch master updated: Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28790)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 25024064ed2 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28790)
25024064ed2 is described below

commit 25024064ed23b37664f165dd6af8a24c972a3148
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Oct 3 09:47:55 2023 -0400

Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28790)

Bumps 
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) 
from 1.18.42 to 1.18.43.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.42...config/v1.18.43)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod |  8 
 sdks/go.sum | 12 
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 9c911948c48..e17427227eb 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -31,8 +31,8 @@ require (
cloud.google.com/go/spanner v1.49.0
cloud.google.com/go/storage v1.33.0
github.com/aws/aws-sdk-go-v2 v1.21.0
-   github.com/aws/aws-sdk-go-v2/config v1.18.42
-   github.com/aws/aws-sdk-go-v2/credentials v1.13.40
+   github.com/aws/aws-sdk-go-v2/config v1.18.43
+   github.com/aws/aws-sdk-go-v2/credentials v1.13.41
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.87
github.com/aws/aws-sdk-go-v2/service/s3 v1.40.0
github.com/aws/smithy-go v1.14.2
@@ -107,9 +107,9 @@ require (
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.36 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.35 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.15.4 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.14.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.15.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.17.1 // indirect
-   github.com/aws/aws-sdk-go-v2/service/sts v1.22.0 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sts v1.23.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index f1516bdf4a5..71c1c4545c8 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -86,11 +86,13 @@ github.com/aws/aws-sdk-go-v2 v1.21.0/go.mod 
h1:/RfNgGmRxI+iFOB1OeJUyxiU+9s88k3pf
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13 
h1:OPLEkmhXf6xFPiz0bLeDArZIDx1NNS4oJyG4nv3Gct0=
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13/go.mod 
h1:gpAbvyDGQFozTEmlTFO8XcQKHzubdq0LzRyJpG6MiXM=
 github.com/aws/aws-sdk-go-v2/config v1.5.0/go.mod 
h1:RWlPOAW3E3tbtNAqTwvSW54Of/yP3oiZXMI0xfUdjyA=
-github.com/aws/aws-sdk-go-v2/config v1.18.42 
h1:28jHROB27xZwU0CB88giDSjz7M1Sba3olb5JBGwina8=
 github.com/aws/aws-sdk-go-v2/config v1.18.42/go.mod 
h1:4AZM3nMMxwlG+eZlxvBKqwVbkDLlnN2a4UGTL6HjaZI=
+github.com/aws/aws-sdk-go-v2/config v1.18.43 
h1:IgdUtTRvUDC6eiJBqU6vh7bHFNAEBjQ8S+qJ7zVhDOs=
+github.com/aws/aws-sdk-go-v2/config v1.18.43/go.mod 
h1:NiFev8qlgg8MPzw3fO/EwzMZeZwlJEKGwfpjRPA9Nvw=
 github.com/aws/aws-sdk-go-v2/credentials v1.3.1/go.mod 
h1:r0n73xwsIVagq8RsxmZbGSRQFj9As3je72C2WzUIToc=
-github.com/aws/aws-sdk-go-v2/credentials v1.13.40 
h1:s8yOkDh+5b1jUDhMBtngF6zKWLDs84chUk2Vk0c38Og=
 github.com/aws/aws-sdk-go-v2/credentials v1.13.40/go.mod 
h1:VtEHVAAqDWASwdOqj/1huyT6uHbs5s8FUHfDQdky/Rs=
+github.com/aws/aws-sdk-go-v2/credentials v1.13.41 
h1:dgbKq1tamtboYAKSXWbqL0lKO9rmEzEhbZFh9JQW/Bg=
+github.com/aws/aws-sdk-go-v2/credentials v1.13.41/go.mod 
h1:cc3Fn7DkKbJalPtQnudHGZZ8ml9+hwtbc1CJONsYYqk=
 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.3.0/go.mod 
h1:2LAuqPx1I6jNfaGDucWfA2zqQCYCOMCDHiCOciALyNw=
 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11 
h1:uDZJF1hu0EVT/4bogChk8DyjSF6fof6uL/0Y26Ma7Fg=
 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11/go.mod 
h1:TEPP4tENqBGO99KwVpV9MlOX4NSrSLP8u3KRy2CDwA8=
@@ -121,13 +123,15 @@ github.com/aws/aws-sdk-go-v2/service/s3 v1.11.1/go.mod 
h1:XLAGFrEjbvMCLvAtWLLP32
 github.com/aws/aws-sdk-go-v2/service/s3 v1.40.0 
h1:wl5dxN1NONhTDQD9uaEvNsDRX29cBmGED/nl0jkWlt4=
 github.com/aws/aws-sdk-go-v2/service/s3 v1.40.0/go.mod 
h1:rDGMZA7f4pbmTtPOk5v5UM2lmX6UAbRnMDJeDvnH7AM=
 github.com/aws/aws-sdk-go-v2/service/sso v1.3.1/go.mod 
h1:J3A3RGUvuCZjvSuZEcOpHDnzZP/sKbh

[beam] branch master updated: Bump cloud.google.com/go/bigtable from 1.19.0 to 1.20.0 in /sdks (#28789)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 1c5982567b3 Bump cloud.google.com/go/bigtable from 1.19.0 to 1.20.0 in 
/sdks (#28789)
1c5982567b3 is described below

commit 1c5982567b3f90e922285900d71b085a6d96c203
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Oct 3 09:47:26 2023 -0400

Bump cloud.google.com/go/bigtable from 1.19.0 to 1.20.0 in /sdks (#28789)

Bumps 
[cloud.google.com/go/bigtable](https://github.com/googleapis/google-cloud-go) 
from 1.19.0 to 1.20.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- 
[Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- 
[Commits](https://github.com/googleapis/google-cloud-go/compare/video/v1.19.0...video/v1.20.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/bigtable
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 869db285534..9c911948c48 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -24,7 +24,7 @@ go 1.20
 
 require (
cloud.google.com/go/bigquery v1.55.0
-   cloud.google.com/go/bigtable v1.19.0
+   cloud.google.com/go/bigtable v1.20.0
cloud.google.com/go/datastore v1.14.0
cloud.google.com/go/profiler v0.3.1
cloud.google.com/go/pubsub v1.33.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 52512ffd424..f1516bdf4a5 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -15,8 +15,8 @@ cloud.google.com/go/bigquery v1.3.0/go.mod 
h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNF
 cloud.google.com/go/bigquery v1.4.0/go.mod 
h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
 cloud.google.com/go/bigquery v1.55.0 
h1:hs44Xxov3XLWQiCx2J8lK5U/ihLqnpm4RVVl5fdtLLI=
 cloud.google.com/go/bigquery v1.55.0/go.mod 
h1:9Y5I3PN9kQWuid6183JFhOGOW3GcirA5LpsKCUn+2ec=
-cloud.google.com/go/bigtable v1.19.0 
h1:wiq9LT0kukfInzvy1joMDijCw/OD1UChpSbORXYn0LI=
-cloud.google.com/go/bigtable v1.19.0/go.mod 
h1:xl5kPa8PTkJjdBxg6qdGH88464nNqmbISHSRU+D2yFE=
+cloud.google.com/go/bigtable v1.20.0 
h1:NqZC/WcesSn4O8L0I2JmuNsUigSyBQifVLYgM9LMQeQ=
+cloud.google.com/go/bigtable v1.20.0/go.mod 
h1:upJDn8frsjzpRMfybiWkD1PG6WCCL7CRl26MgVeoXY4=
 cloud.google.com/go/compute v1.23.0 
h1:tP41Zoavr8ptEqaW6j+LQOnyBBhO7OkOMAGrgLopTwY=
 cloud.google.com/go/compute v1.23.0/go.mod 
h1:4tCnrn48xsqlwSAiLf1HXMQk8CONslYbdiEZc9FEIbM=
 cloud.google.com/go/compute/metadata v0.2.3 
h1:mg4jlk7mCAj6xXp9UJ4fjI9VUI5rubuGBW5aJ7UnBMY=

[beam] branch master updated (2b7df3b64b3 -> e6dcd372123)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 2b7df3b64b3 Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks 
(#28674)
 add e6dcd372123 Update example in Tour of Beam (#28718)

No new revisions were added by this update.

Summary of changes:
 .../setting-pipeline/python-example/task.py | 17 ++---
 1 file changed, 6 insertions(+), 11 deletions(-)

[beam] branch master updated: Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28639)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 437204e6817 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28639)
437204e6817 is described below

commit 437204e681760978e691549b9e87d0a9fdd3ab97
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Mon Sep 25 10:12:04 2023 -0400

Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28639)

Bumps 
[github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) 
from 1.18.40 to 1.18.42.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- 
[Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.40...config/v1.18.42)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 10 +-
 sdks/go.sum | 15 ++-
 2 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index 73b99c8344e..3fa3a62445c 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -31,8 +31,8 @@ require (
cloud.google.com/go/spanner v1.49.0
cloud.google.com/go/storage v1.33.0
github.com/aws/aws-sdk-go-v2 v1.21.0
-   github.com/aws/aws-sdk-go-v2/config v1.18.40
-   github.com/aws/aws-sdk-go-v2/credentials v1.13.38
+   github.com/aws/aws-sdk-go-v2/config v1.18.42
+   github.com/aws/aws-sdk-go-v2/credentials v1.13.40
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.84
github.com/aws/aws-sdk-go-v2/service/s3 v1.38.5
github.com/aws/smithy-go v1.14.2
@@ -101,14 +101,14 @@ require (
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.41 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.35 // indirect
-   github.com/aws/aws-sdk-go-v2/internal/ini v1.3.42 // indirect
+   github.com/aws/aws-sdk-go-v2/internal/ini v1.3.43 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.1.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.9.14 
// indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.36 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.35 // 
indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.15.4 // 
indirect
-   github.com/aws/aws-sdk-go-v2/service/sso v1.14.0 // indirect
-   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.16.0 // indirect
+   github.com/aws/aws-sdk-go-v2/service/sso v1.14.1 // indirect
+   github.com/aws/aws-sdk-go-v2/service/ssooidc v1.17.1 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.22.0 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
diff --git a/sdks/go.sum b/sdks/go.sum
index 7e26168a360..ce2031747c7 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -86,11 +86,13 @@ github.com/aws/aws-sdk-go-v2 v1.21.0/go.mod 
h1:/RfNgGmRxI+iFOB1OeJUyxiU+9s88k3pf
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13 
h1:OPLEkmhXf6xFPiz0bLeDArZIDx1NNS4oJyG4nv3Gct0=
 github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13/go.mod 
h1:gpAbvyDGQFozTEmlTFO8XcQKHzubdq0LzRyJpG6MiXM=
 github.com/aws/aws-sdk-go-v2/config v1.5.0/go.mod 
h1:RWlPOAW3E3tbtNAqTwvSW54Of/yP3oiZXMI0xfUdjyA=
-github.com/aws/aws-sdk-go-v2/config v1.18.40 
h1:dbu1llI/nTIL+r6sYHMeVLl99DM8J8/o1I4EPurnhLg=
 github.com/aws/aws-sdk-go-v2/config v1.18.40/go.mod 
h1:JjrCZQwSPGCoZRQzKHyZNNueaKO+kFaEy2sR6mCzd90=
+github.com/aws/aws-sdk-go-v2/config v1.18.42 
h1:28jHROB27xZwU0CB88giDSjz7M1Sba3olb5JBGwina8=
+github.com/aws/aws-sdk-go-v2/config v1.18.42/go.mod 
h1:4AZM3nMMxwlG+eZlxvBKqwVbkDLlnN2a4UGTL6HjaZI=
 github.com/aws/aws-sdk-go-v2/credentials v1.3.1/go.mod 
h1:r0n73xwsIVagq8RsxmZbGSRQFj9As3je72C2WzUIToc=
-github.com/aws/aws-sdk-go-v2/credentials v1.13.38 
h1:gDAuCdVlA4lmmgQhvpZlscwicloCqH44vkxLklGkQLA=
 github.com/aws/aws-sdk-go-v2/credentials v1.13.38/go.mod 
h1:sD4G/Ybgp6s89mWIES3Xn97CsRLpxvz9uVSdv0UxY8I=
+github.com/aws/aws-sdk-go-v2/credentials v1.13.40 
h1:s8yOkDh+5b1jUDhMBtngF6zKWLDs84chUk2Vk0c38Og=
+github.com/aws/aws-sdk-go-v2/credentials v1.13.40/go.mod 
h1:VtEHVAAqDWASwdOqj/1huyT6uHbs5s8FUHfDQdky/Rs=
 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.3.0/go.mod 
h1:2LAuqPx1I6jNfaGDucWfA2zqQCYCOMCDHiCOciALyNw=
 github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11 
h1:uDZJF1hu0EVT/4bogChk8DyjSF6fof6uL/0Y26Ma7Fg=
 github.com/aws/aws-sdk-g

[beam] branch master updated: Bump google.golang.org/grpc from 1.58.1 to 1.58.2 in /sdks (#28607)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new c2677e28c66 Bump google.golang.org/grpc from 1.58.1 to 1.58.2 in /sdks 
(#28607)
c2677e28c66 is described below

commit c2677e28c66e6b91b841c3ad5c843b50b6eb1528
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Fri Sep 22 15:10:58 2023 -0400

Bump google.golang.org/grpc from 1.58.1 to 1.58.2 in /sdks (#28607)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.58.1 
to 1.58.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.58.1...v1.58.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] 
Co-authored-by: dependabot[bot] 
<49699333+dependabot[bot]@users.noreply.github.com>
---
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/sdks/go.mod b/sdks/go.mod
index ac7db574791..73b99c8344e 100644
--- a/sdks/go.mod
+++ b/sdks/go.mod
@@ -59,7 +59,7 @@ require (
golang.org/x/text v0.13.0
google.golang.org/api v0.142.0
google.golang.org/genproto v0.0.0-20230821184602-ccc8af3d0e93
-   google.golang.org/grpc v1.58.1
+   google.golang.org/grpc v1.58.2
google.golang.org/protobuf v1.31.0
gopkg.in/retry.v1 v1.0.3
gopkg.in/yaml.v2 v2.4.0
diff --git a/sdks/go.sum b/sdks/go.sum
index 2b404990716..7e26168a360 100644
--- a/sdks/go.sum
+++ b/sdks/go.sum
@@ -712,8 +712,8 @@ google.golang.org/grpc v1.26.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8
 google.golang.org/grpc v1.27.0/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.27.1/go.mod 
h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
 google.golang.org/grpc v1.33.2/go.mod 
h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
-google.golang.org/grpc v1.58.1 h1:OL+Vz23DTtrrldqHK49FUOPHyY75rvFqJfXC84NYW58=
-google.golang.org/grpc v1.58.1/go.mod 
h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
+google.golang.org/grpc v1.58.2 h1:SXUpjxeVF3FKrTYQI4f4KvbGD5u2xccdYdurwowix5I=
+google.golang.org/grpc v1.58.2/go.mod 
h1:tgX3ZQDlNJGU96V6yHh1T/JeoBQ2TXdr43YbYSsCJk0=
 google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod 
h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
 google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod 
h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
 google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod 
h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=

[beam] branch master updated (a0d014ae36f -> bb24ca71b62)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from a0d014ae36f Merge pull request #28262: [runners-flink] #28258 Wait 
infinitely if the duration value is less than 1 ms
 add bb24ca71b62 Bump github.com/testcontainers/testcontainers-go in /sdks 
(#28576)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 2 +-
 sdks/go.sum | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

[beam] branch dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.24.1 created (now 7f6110ee024)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch 
dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.24.1
in repository https://gitbox.apache.org/repos/asf/beam.git


  at 7f6110ee024 Bump github.com/testcontainers/testcontainers-go in /sdks

No new revisions were added by this update.

[beam] branch master updated (79d0a8d1109 -> 635f9ef5a0e)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 79d0a8d1109 tooltips, footer, humanize (#28538)
 add 635f9ef5a0e Bump github.com/testcontainers/testcontainers-go in /sdks 
(#28550)

No new revisions were added by this update.

Summary of changes:
 sdks/go.mod | 19 +++
 sdks/go.sum | 41 -
 2 files changed, 47 insertions(+), 13 deletions(-)

[beam] branch master updated: Update HuggingFace api doc and add text2audio pipeline task (#28474)

[riteshghorse]

This is an automated email from the ASF dual-hosted git repository.

riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4184f5ea521 Update HuggingFace api doc and add text2audio pipeline 
task (#28474)
4184f5ea521 is described below

commit 4184f5ea521941fcba6b90769b2d7f35b7262471
Author: Ritesh Ghorse 
AuthorDate: Tue Sep 19 22:40:15 2023 -0400

Update HuggingFace api doc and add text2audio pipeline task (#28474)

* update api doc and add text2audio pipeline task

* update doc

* update doc

* fix indent

* correct example snippet
---
 .../apache_beam/ml/inference/huggingface_inference.py  | 18 ++
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/sdks/python/apache_beam/ml/inference/huggingface_inference.py 
b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
index aee61336378..3ec063808ae 100644
--- a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
+++ b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
@@ -98,6 +98,7 @@ class PipelineTask(str, Enum):
   TextClassification = 'text-classification'
   TextGeneration = 'text-generation'
   Text2TextGeneration = 'text2text-generation'
+  TextToAudio = 'text-to-audio'
   TokenClassification = 'token-classification'
   Translation = 'translation'
   VideoClassification = 'video-classification'
@@ -570,7 +571,7 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
   def __init__(
   self,
   task: Union[str, PipelineTask] = "",
-  model=None,
+  model: str = "",
   *,
   inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
   load_pipeline_args: Optional[Dict[str, Any]] = None,
@@ -593,9 +594,18 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
 Args:
   task (str or enum.Enum): task supported by HuggingFace Pipelines.
 Accepts a string task or an enum.Enum from PipelineTask.
-  model : path to pretrained model on Hugging Face Models Hub to use custom
-model for the chosen task. If the model already defines the task then
-no need to specify the task parameter.
+  model (str): path to the pretrained *model-id* on Hugging Face Models Hub
+to use custom model for the chosen task. If the `model` already defines
+the task then no need to specify the `task` parameter.
+Use the *model-id* string instead of an actual model here.
+Model-specific kwargs for `from_pretrained(..., **model_kwargs)` can be
+specified with `model_kwargs` using `load_pipeline_args`.
+
+Example Usage::
+  model_handler = HuggingFacePipelineModelHandler(
+task="text-generation", model="meta-llama/Llama-2-7b-hf",
+load_pipeline_args={'model_kwargs':{'quantization_map':config}})
+
   inference_fn: the inference function to use during RunInference.
 Default is _default_pipeline_inference_fn.
   load_pipeline_args (Dict[str, Any]): keyword arguments to provide load