[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 233d9dcf4e5 Updating config from bot
233d9dcf4e5 is described below

commit 233d9dcf4e536dc385427670475558d4b013ca8d
Author: github-actions 
AuthorDate: Fri Mar 17 08:44:22 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25876.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25876.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25876.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25876.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 0ef09ee9a90 Updating config from bot
0ef09ee9a90 is described below

commit 0ef09ee9a900815a47feb7d5087fddedec987063
Author: github-actions 
AuthorDate: Fri Mar 17 09:33:57 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25861.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25861.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25861.json
new file mode 100644
index 000..6272b4e797d
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25861.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"python": "AnandInguva"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new f1d7551db93 Updating config from bot
f1d7551db93 is described below

commit f1d7551db93933c43e13a9ee81178d08315dcc74
Author: github-actions 
AuthorDate: Fri Mar 17 09:33:59 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 16b8d6aece3..205b47517ce 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -1,7 +1,7 @@
 {
   "label": "python",
   "dateOfLastReviewAssignment": {
-"AnandInguva": 1678959242992,
+"AnandInguva": 1679045635592,
 "yeandy": 1665802753763,
 "TheNeuralBit": 1667896849319,
 "ryanthompson591": 1670002443548,

[beam-starter-typescript] branch dependabot/npm_and_yarn/typescript-5.0.2 created (now 37937e1)

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch dependabot/npm_and_yarn/typescript-5.0.2
in repository https://gitbox.apache.org/repos/asf/beam-starter-typescript.git


  at 37937e1  Bump typescript from 4.7.3 to 5.0.2

No new revisions were added by this update.

[beam] branch master updated (213b3ed6bf1 -> cbc1a7661bd)

2023-03-17 Thread yhu 
This is an automated email from the ASF dual-hosted git repository.

yhu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 213b3ed6bf1 Use self hosted hadoop docker image in hdfsIntegrationTest 
(#25867)
 add cbc1a7661bd Revert "Update sqlalchemy requirement in /sdks/python 
(#25855)" (#25872)

No new revisions were added by this update.

Summary of changes:
 sdks/python/setup.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[beam] branch dependabot/pip/sdks/python/pytest-xdist-gte-2.5.0-and-lt-4 updated (9f1c216ead8 -> 72474f63988)

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch 
dependabot/pip/sdks/python/pytest-xdist-gte-2.5.0-and-lt-4
in repository https://gitbox.apache.org/repos/asf/beam.git


 discard 9f1c216ead8 Update pytest-xdist requirement in /sdks/python
 add eac926878c2 Use autoSharding from config when BigQueryIO.Write is 
created. (#25862)
 add 738825c90fb Revert "Adding support for DLQ for ZetaSQL (#25426)"
 add bb5e5d0 Merge pull request #25863: Revert "Adding support for DLQ 
for ZetaSQL (#25426)"
 add 213b3ed6bf1 Use self hosted hadoop docker image in hdfsIntegrationTest 
(#25867)
 add cbc1a7661bd Revert "Update sqlalchemy requirement in /sdks/python 
(#25855)" (#25872)
 add 72474f63988 Update pytest-xdist requirement in /sdks/python

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (9f1c216ead8)
\
 N -- N -- N   
refs/heads/dependabot/pip/sdks/python/pytest-xdist-gte-2.5.0-and-lt-4 
(72474f63988)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../apache/beam/sdk/coders/RowCoderGenerator.java  |  12 +--
 .../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 112 +
 .../sql/zetasql/BeamZetaSqlCalcRelTest.java|  70 -
 .../sql/zetasql/ZetaSqlDialectSpecTest.java|  14 +--
 .../io/gcp/bigquery/BigQuerySchemaIOProvider.java  |   9 +-
 ...ueryStorageWriteApiSchemaTransformProvider.java |  23 +++--
 .../io/hdfs_integration_test/docker-compose.yml|   4 +-
 sdks/python/setup.py   |   2 +-
 8 files changed, 58 insertions(+), 188 deletions(-)

[beam] branch master updated: Minor fix: move the getattr out of the loop as it is not needed to be inside (#25859)

2023-03-17 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new b2afbe0b26f Minor fix: move the getattr out of the loop as it is not 
needed to be inside (#25859)
b2afbe0b26f is described below

commit b2afbe0b26f7ea27d9073e582f913dd912b6d82c
Author: Amir Fayazi 
AuthorDate: Fri Mar 17 05:00:36 2023 -0700

Minor fix: move the getattr out of the loop as it is not needed to be 
inside (#25859)
---
 sdks/python/apache_beam/ml/inference/pytorch_inference.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/python/apache_beam/ml/inference/pytorch_inference.py 
b/sdks/python/apache_beam/ml/inference/pytorch_inference.py
index 818dd8325dc..cc6f3ea6be7 100644
--- a/sdks/python/apache_beam/ml/inference/pytorch_inference.py
+++ b/sdks/python/apache_beam/ml/inference/pytorch_inference.py
@@ -380,7 +380,7 @@ def make_keyed_tensor_model_fn(model_fn: str) -> 
KeyedTensorInferenceFn:
 batched_tensors = torch.stack(key_to_tensor_list[key])
 batched_tensors = _convert_to_device(batched_tensors, device)
 key_to_batched_tensors[key] = batched_tensors
-pred_fn = getattr(model, model_fn)
+  pred_fn = getattr(model, model_fn)
   predictions = pred_fn(**key_to_batched_tensors, **inference_args)
 return utils._convert_to_result(batch, predictions, model_id)

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new fb3b8f3da03 Updating config from bot
fb3b8f3da03 is described below

commit fb3b8f3da033650289dafc708b22e62325448a27
Author: github-actions 
AuthorDate: Fri Mar 17 12:13:55 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25632.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25632.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25632.json
index a7c19975bb1..e5769456548 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-25632.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25632.json
@@ -1,8 +1,8 @@
 {
   "commentedAboutFailingChecks": false,
   "reviewersAssignedForLabels": {
-"java": "robertwb",
-"io": "Abacn"
+"java": "kennknowles",
+"io": "johnjcasey"
   },
   "nextAction": "Reviewers",
   "stopReviewerNotifications": false,

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 2af1a985f9c Updating config from bot
2af1a985f9c is described below

commit 2af1a985f9c7d22fb8c600ac4f342d5c067af5ad
Author: github-actions 
AuthorDate: Fri Mar 17 12:14:00 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index 0b5db1fb436..1a73d7e75cf 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -2,7 +2,7 @@
   "label": "io",
   "dateOfLastReviewAssignment": {
 "chamikaramj": 1678989904221,
-"johnjcasey": 1678822446183,
+"johnjcasey": 1679055233734,
 "pabloem": 1678831493228,
 "Abacn": 1678977301745,
 "ahmedabu98": 1678995247594

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new ea329f9f39c Updating config from bot
ea329f9f39c is described below

commit ea329f9f39cee2ca4d71479a6d10a91b029fd830
Author: github-actions 
AuthorDate: Fri Mar 17 12:13:58 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index e21e2c7f4f8..162a5162dd0 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -2,7 +2,7 @@
   "label": "java",
   "dateOfLastReviewAssignment": {
 "lukecwik": 1679014126796,
-"kennknowles": 1678989904221,
+"kennknowles": 1679055233118,
 "robertwb": 1679006042236,
 "kileys": 1674428678843,
 "apilloud": 1678822446183

[beam] branch master updated (b2afbe0b26f -> 821a77beb00)

2023-03-17 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from b2afbe0b26f Minor fix: move the getattr out of the loop as it is not 
needed to be inside (#25859)
 add 821a77beb00 Multi language/runinference example (#25083)

No new revisions were added by this update.

Summary of changes:
 .../inference/multi_language_inference/README.md   |  69 +++
 .../beam/examples/MultiLangRunInference.java   |  92 ++
 .../multi_language_custom_transform}/__init__.py   |   3 +-
 .../composite_transform.py | 122 +++
 .../multi_language_custom_transform/setup.py}  |  12 +-
 .../documentation/ml/multi-language-inference.md   | 133 +
 .../site/content/en/documentation/ml/overview.md   |   5 +-
 .../partials/section-menu/en/documentation.html|   3 +-
 8 files changed, 428 insertions(+), 11 deletions(-)
 create mode 100644 
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md
 create mode 100644 
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/apache/beam/examples/MultiLangRunInference.java
 copy {.test-infra/jenkins/dependency_check => 
sdks/python/apache_beam/examples/inference/multi_language_inference/multi_language_custom_transform/multi_language_custom_transform}/__init__.py
 (99%)
 create mode 100644 
sdks/python/apache_beam/examples/inference/multi_language_inference/multi_language_custom_transform/multi_language_custom_transform/composite_transform.py
 copy sdks/python/apache_beam/{io/gcp/pubsublite/__init__.py => 
examples/inference/multi_language_inference/multi_language_custom_transform/setup.py}
 (84%)
 create mode 100644 
website/www/site/content/en/documentation/ml/multi-language-inference.md

[beam] branch master updated (821a77beb00 -> d9f81b8592d)

2023-03-17 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 821a77beb00 Multi language/runinference example (#25083)
 add d9f81b8592d Upgrade tox from 3.x.x to 4.x.x (#25785)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/python_tests.yml  |  6 +++---
 release/src/main/scripts/build_release_candidate.sh |  3 +--
 sdks/python/MANIFEST.in |  1 +
 sdks/python/gen_protos.py   | 13 -
 sdks/python/tox.ini | 18 +-
 5 files changed, 18 insertions(+), 23 deletions(-)

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 7f43e53bb9d Updating config from bot
7f43e53bb9d is described below

commit 7f43e53bb9ddbb77d2ed2f31a1a7c257045767f4
Author: github-actions 
AuthorDate: Fri Mar 17 14:05:13 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 205b47517ce..3a3ccd76bde 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -8,7 +8,7 @@
 "tvalentyn": 1678995263753,
 "pabloem": 1678995247594,
 "y1chi": 1667002607045,
-"damccorm": 1678979106036,
+"damccorm": 1679061908460,
 "jrmccluskey": 1679014132643
   }
 }
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 1d38dfe3530 Updating config from bot
1d38dfe3530 is described below

commit 1d38dfe3530ae9ad45ebcb23820993327e1bc005
Author: github-actions 
AuthorDate: Fri Mar 17 14:05:10 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25833.json | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25833.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25833.json
index 6272b4e797d..24d0e10a64c 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-25833.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25833.json
@@ -1,10 +1,10 @@
 {
   "commentedAboutFailingChecks": false,
   "reviewersAssignedForLabels": {
-"python": "AnandInguva"
+"python": "damccorm"
   },
   "nextAction": "Reviewers",
   "stopReviewerNotifications": false,
   "remindAfterTestsPass": [],
-  "committerAssigned": false
+  "committerAssigned": true
 }
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 18137020fcc Updating config from bot
18137020fcc is described below

commit 18137020fccee6e7d22e2f1a3d28b5873f00209b
Author: github-actions 
AuthorDate: Fri Mar 17 14:12:18 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25877.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25877.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25877.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25877.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch master updated (d9f81b8592d -> 167a4c5fb59)

2023-03-17 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from d9f81b8592d Upgrade tox from 3.x.x to 4.x.x (#25785)
 add 167a4c5fb59 Remove changes from MANIFEST.in (#25877)

No new revisions were added by this update.

Summary of changes:
 sdks/python/MANIFEST.in | 1 -
 1 file changed, 1 deletion(-)

[beam] branch master updated (167a4c5fb59 -> b4ee5bd60e0)

2023-03-17 Thread damccorm 
This is an automated email from the ASF dual-hosted git repository.

damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 167a4c5fb59 Remove changes from MANIFEST.in (#25877)
 add b4ee5bd60e0 Update pytest-xdist requirement in /sdks/python (#25833)

No new revisions were added by this update.

Summary of changes:
 sdks/python/setup.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 62c96d63600 Updating config from bot
62c96d63600 is described below

commit 62c96d636001f0c23febe375d88d0e7d1dc0150f
Author: github-actions 
AuthorDate: Fri Mar 17 17:28:55 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25886.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25886.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25886.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25886.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new a3959b919ae Updating config from bot
a3959b919ae is described below

commit a3959b919aebc1c73377d9c2c3952c04582beb63
Author: github-actions 
AuthorDate: Fri Mar 17 18:05:10 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25885.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25885.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25885.json
new file mode 100644
index 000..d26f23226c7
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25885.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"java": "robertwb"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 71c86fde341 Updating config from bot
71c86fde341 is described below

commit 71c86fde341dc5af98ed063cb0c49b3dd590a79e
Author: github-actions 
AuthorDate: Fri Mar 17 18:05:13 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 162a5162dd0..66c8e626c2a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -3,7 +3,7 @@
   "dateOfLastReviewAssignment": {
 "lukecwik": 1679014126796,
 "kennknowles": 1679055233118,
-"robertwb": 1679006042236,
+"robertwb": 1679076308290,
 "kileys": 1674428678843,
 "apilloud": 1678822446183
   }

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 09d4e9c7913 Updating config from bot
09d4e9c7913 is described below

commit 09d4e9c79138d348042272fac1bfc1c6b7bc347b
Author: github-actions 
AuthorDate: Fri Mar 17 18:35:44 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25216.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25216.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25216.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25216.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": true,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch master updated: Onboard Performance tests using the Change point analysis tool. (#25754)

2023-03-17 Thread tvalentyn 
This is an automated email from the ASF dual-hosted git repository.

tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 6f82f34eeb1 Onboard Performance tests using the Change point analysis 
tool. (#25754)
6f82f34eeb1 is described below

commit 6f82f34eeb14097810f3e8faee10221ac1c54aa3
Author: Anand Inguva <34158215+ananding...@users.noreply.github.com>
AuthorDate: Fri Mar 17 14:49:07 2023 -0400

Onboard Performance tests using the Change point analysis tool. (#25754)
---
 .github/workflows/run_perf_alert_tool.yml  | 24 +
 .../python/apache_beam/testing/analyzers/README.md |  9 ++--
 .../apache_beam/testing/analyzers/constants.py |  2 +-
 .../testing/analyzers/github_issues_utils.py   |  3 +-
 .../apache_beam/testing/analyzers/perf_analysis.py | 17 ---
 .../testing/analyzers/perf_analysis_utils.py   | 20 
 .../testing/analyzers/tests_config.yaml| 59 ++
 .../benchmarks/cloudml/cloudml_benchmark_test.py   | 24 -
 8 files changed, 93 insertions(+), 65 deletions(-)

diff --git a/.github/workflows/run_perf_alert_tool.yml 
b/.github/workflows/run_perf_alert_tool.yml
index 30779e765f1..65e89ca5f5f 100644
--- a/.github/workflows/run_perf_alert_tool.yml
+++ b/.github/workflows/run_perf_alert_tool.yml
@@ -17,12 +17,12 @@
 
 # To learn more about GitHub Actions in Apache Beam check the CI.md
 
-name: Run performance alerting tool on Python load/performance/benchmark tests.
+name: Performance alerting tool on Python load/performance/benchmark tests.
 
 on:
+  workflow_dispatch:
   schedule:
 - cron: '5 22 * * *'
-
 jobs:
   python_run_change_point_analysis:
 name: Run Change Point Analysis.
@@ -34,6 +34,11 @@ jobs:
 uses: actions/setup-python@v4
 with:
   python-version: 3.8
+  - name: Authenticate on GCP
+uses: google-github-actions/setup-gcloud@v0
+with:
+  service_account_key: ${{ secrets.GCP_SA_KEY }}
+  export_default_credentials: true
   - name: Get Apache Beam Build dependencies
 working-directory: ./sdks/python
 run: pip install pip setuptools --upgrade && pip install -r 
build-requirements.txt
@@ -43,17 +48,16 @@ jobs:
   - name: Install signal-processing-algorithms
 run: pip install signal-processing-algorithms
   - name: Install pandas, yaml, requests
-run: pip install pandas PyYAML requests
-#  - name: Run Change Point Analysis.
-#working-directory: ./sdks/python/apache_beam/testing/analyzers
-#shell: bash
-#run: python analysis.py
-#env:
-#  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+run: pip install pandas db-dtypes PyYAML requests
+  - name: Run Change Point Analysis.
+working-directory: ./sdks/python/apache_beam/testing/analyzers
+shell: bash
+run: python perf_analysis.py
+env:
+  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
   - name: Run change point analysis tests.
 working-directory: ./sdks/python/apache_beam/testing/analyzers
 shell: bash
 run: pytest perf_analysis_test.py
 env:
   GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-
diff --git a/sdks/python/apache_beam/testing/analyzers/README.md 
b/sdks/python/apache_beam/testing/analyzers/README.md
index 9f50d9797b5..71351fe3e57 100644
--- a/sdks/python/apache_beam/testing/analyzers/README.md
+++ b/sdks/python/apache_beam/testing/analyzers/README.md
@@ -41,7 +41,8 @@ please follow the below structure.
 ```
 # the test_1 must be a unique id.
 test_1:
-  test_name: 
apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks
+  test_name: Pytorch image classification on 50k images of size 224 x 224 with 
resnet 152
+  test_target: 
apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks
   source: big_query
   metrics_dataset: beam_run_inference
   metrics_table: torch_inference_imagenet_results_resnet152
@@ -53,9 +54,9 @@ test_1:
   num_runs_in_change_point_window: 30 # optional parameter
 ```
 
-**NOTE**: `test_name` should be in the format `apache_beam.foo.bar`. It should 
point to a single test target.
+**NOTE**: `test_target` is optional. It is used for identifying the test that 
was causing the regression.
 
-**Note**: If the source is **BigQuery**, the metrics_dataset, metrics_table, 
project and metric_name should match with the values defined for 
performance/load tests.
+**Note**: If the source is **BigQuery**, the `metrics_dataset`, 
`metrics_table`, `project` and `metric_name` should match with the values 
defined for performance/load tests.
 The above example uses this [test 
configuration](https://github.com/apache/beam/blob/0a91d139dea4276dc46176c4cdcdfce210fc50c4/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L30)
 to fill u

[beam] 01/01: Merge pull request #25871: Update default splitting behavior to get fewer, larger, splits

2023-03-17 Thread kenn 
This is an automated email from the ASF dual-hosted git repository.

kenn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit de578e606e5d07eb9efe3ffc993ed0d9d11e
Merge: 6f82f34eeb1 a9826ee2db1
Author: Kenneth Knowles 
AuthorDate: Fri Mar 17 12:10:18 2023 -0700

Merge pull request #25871: Update default splitting behavior to get fewer, 
larger, splits

 .../core/src/main/java/org/apache/beam/sdk/io/Read.java| 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

[beam] branch master updated (6f82f34eeb1 -> de578e606e5)

2023-03-17 Thread kenn 
This is an automated email from the ASF dual-hosted git repository.

kenn pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


from 6f82f34eeb1 Onboard Performance tests using the Change point analysis 
tool. (#25754)
 add a9826ee2db1 Update default splitting behavior to get fewer, larger, 
splits
 new de578e606e5 Merge pull request #25871: Update default splitting 
behavior to get fewer, larger, splits

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../core/src/main/java/org/apache/beam/sdk/io/Read.java| 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

[beam] branch master updated: Make conversion to Iterable tighter (#25866)

2023-03-17 Thread jrmccluskey 
This is an automated email from the ASF dual-hosted git repository.

jrmccluskey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 7555d989799 Make conversion to Iterable tighter (#25866)
7555d989799 is described below

commit 7555d9897996362a82e6e5d25202ddf29931338d
Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com>
AuthorDate: Fri Mar 17 15:23:55 2023 -0400

Make conversion to Iterable tighter (#25866)
---
 sdks/python/apache_beam/typehints/native_type_compatibility.py| 2 +-
 .../apache_beam/typehints/native_type_compatibility_test.py   | 8 
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/sdks/python/apache_beam/typehints/native_type_compatibility.py 
b/sdks/python/apache_beam/typehints/native_type_compatibility.py
index f0ef1b2e2c3..d03d5db4045 100644
--- a/sdks/python/apache_beam/typehints/native_type_compatibility.py
+++ b/sdks/python/apache_beam/typehints/native_type_compatibility.py
@@ -201,7 +201,7 @@ def convert_collections_to_typing(typ):
   typ = typing.Iterator[typ.__args__]
 elif hasattr(typ, 'send') and hasattr(typ, 'throw'):
   typ = typing.Generator[typ.__args__]
-else:
+elif _match_is_exactly_iterable(typ):
   typ = typing.Iterable[typ.__args__]
   return typ
 
diff --git 
a/sdks/python/apache_beam/typehints/native_type_compatibility_test.py 
b/sdks/python/apache_beam/typehints/native_type_compatibility_test.py
index 013fa354b82..89045ae7a25 100644
--- a/sdks/python/apache_beam/typehints/native_type_compatibility_test.py
+++ b/sdks/python/apache_beam/typehints/native_type_compatibility_test.py
@@ -156,6 +156,14 @@ class NativeTypeCompatibilityTest(unittest.TestCase):
   'nested iterable',
   tuple[bytes, collections.abc.Iterable[int]],
   typehints.Tuple[bytes, typehints.Iterable[int]]),
+  (
+  'iterable over tuple',
+  collections.abc.Iterable[tuple[str, int]],
+  typehints.Iterable[typehints.Tuple[str, int]]),
+  (
+  'mapping not caught',
+  collections.abc.Mapping[str, int],
+  collections.abc.Mapping[str, int]),
   ]
 
   for test_case in test_cases:

[beam] branch master updated: Add support for CPython 3.11 opcodes (#25795)

2023-03-17 Thread jrmccluskey 
This is an automated email from the ASF dual-hosted git repository.

jrmccluskey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new fad7f96dc85 Add support for CPython 3.11 opcodes (#25795)
fad7f96dc85 is described below

commit fad7f96dc85047eb1a01536f817565d6f959ee35
Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com>
AuthorDate: Fri Mar 17 16:14:44 2023 -0400

Add support for CPython 3.11 opcodes (#25795)

* Add support for CPython 3.11 opcodes

* Remove unused utility function

* Uncomment test

* Drop camel case form

* Add addition documentation  comments
---
 sdks/python/apache_beam/typehints/opcodes.py   |  36 +-
 .../apache_beam/typehints/trivial_inference.py | 129 +++--
 2 files changed, 151 insertions(+), 14 deletions(-)

diff --git a/sdks/python/apache_beam/typehints/opcodes.py 
b/sdks/python/apache_beam/typehints/opcodes.py
index 1ac3538c691..5a35b56b932 100644
--- a/sdks/python/apache_beam/typehints/opcodes.py
+++ b/sdks/python/apache_beam/typehints/opcodes.py
@@ -178,7 +178,9 @@ binary_lshift = inplace_lshift = binary_rshift = 
inplace_rshift = pop_top
 
 binary_and = inplace_and = symmetric_binary_op
 binary_xor = inplace_xor = symmetric_binary_op
-binary_or = inpalce_or = symmetric_binary_op
+binary_or = inplace_or = symmetric_binary_op
+
+binary_op = symmetric_binary_op
 
 
 def store_subscr(unused_state, unused_args):
@@ -403,6 +405,8 @@ import_from = push_value(Any)
 
 
 def load_global(state, arg):
+  if (sys.version_info.major, sys.version_info.minor) >= (3, 11):
+arg = arg >> 1
   state.stack.append(state.get_global(arg))
 
 
@@ -428,10 +432,18 @@ def gen_start(state, arg):
 
 
 def load_closure(state, arg):
+  # The arg is no longer offset by len(covar_names) as of 3.11
+  # See https://docs.python.org/3/library/dis.html#opcode-LOAD_CLOSURE
+  if (sys.version_info.major, sys.version_info.minor) >= (3, 11):
+arg -= len(state.co.co_varnames)
   state.stack.append(state.get_closure(arg))
 
 
 def load_deref(state, arg):
+  # The arg is no longer offset by len(covar_names) as of 3.11
+  # See https://docs.python.org/3/library/dis.html#opcode-LOAD_DEREF
+  if (sys.version_info.major, sys.version_info.minor) >= (3, 11):
+arg -= len(state.co.co_varnames)
   state.stack.append(state.closure_type(arg))
 
 
@@ -440,9 +452,19 @@ def make_function(state, arg):
   """
   # TODO(luke-zhu): Handle default argument types
   globals = state.f.__globals__  # Inherits globals from the current frame
-  func_name = state.stack[-1].value
-  func_code = state.stack[-2].value
-  pop_count = 2
+  tos = state.stack[-1].value
+  # In Python 3.11 lambdas no longer have fully qualified names on the stack,
+  # so we check for this case (AKA the code is top of stack.)
+  if isinstance(tos, types.CodeType):
+func_name = None
+func_code = tos
+pop_count = 1
+is_lambda = True
+  else:
+func_name = tos
+func_code = state.stack[-2].value
+pop_count = 2
+is_lambda = False
   closure = None
   # arg contains flags, with corresponding stack values if positive.
   # https://docs.python.org/3.6/library/dis.html#opcode-MAKE_FUNCTION
@@ -450,8 +472,12 @@ def make_function(state, arg):
   if arg & 0x08:
 # Convert types in Tuple constraint to a tuple of CPython cells.
 # https://stackoverflow.com/a/44670295
+if is_lambda:
+  closureTuplePos = -2
+else:
+  closureTuplePos = -3
 closure = tuple((lambda _: lambda: _)(t).__closure__[0]
-for t in state.stack[-3].tuple_types)
+for t in state.stack[closureTuplePos].tuple_types)
 
   func = types.FunctionType(func_code, globals, name=func_name, 
closure=closure)
 
diff --git a/sdks/python/apache_beam/typehints/trivial_inference.py 
b/sdks/python/apache_beam/typehints/trivial_inference.py
index f69a87192e3..f4b350e8f05 100644
--- a/sdks/python/apache_beam/typehints/trivial_inference.py
+++ b/sdks/python/apache_beam/typehints/trivial_inference.py
@@ -126,11 +126,12 @@ class Const(object):
 class FrameState(object):
   """Stores the state of the frame at a particular point of execution.
   """
-  def __init__(self, f, local_vars=None, stack=()):
+  def __init__(self, f, local_vars=None, stack=(), kw_names=None):
 self.f = f
 self.co = f.__code__
 self.vars = list(local_vars)
 self.stack = list(stack)
+self.kw_names = kw_names
 
   def __eq__(self, other):
 return isinstance(other, FrameState) and self.__dict__ == other.__dict__
@@ -139,7 +140,7 @@ class FrameState(object):
 return hash(tuple(sorted(self.__dict__.items(
 
   def copy(self):
-return FrameState(self.f, self.vars, self.stack)
+return FrameState(self.f, self.vars, self.stack, self.kw_names)
 
   def const_type(self, i):
 return Const(self.co.co_consts[i])
@@ -35

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 4353db95c7a Updating config from bot
4353db95c7a is described below

commit 4353db95c7aa286d38d502c28f457d63060661d8
Author: github-actions 
AuthorDate: Fri Mar 17 21:37:34 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25873.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25873.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25873.json
index 499fd94a3a0..ffa9ca4fec0 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-25873.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25873.json
@@ -4,7 +4,7 @@
 "java": "lukecwik"
   },
   "nextAction": "Reviewers",
-  "stopReviewerNotifications": false,
+  "stopReviewerNotifications": true,
   "remindAfterTestsPass": [],
   "committerAssigned": false
 }
\ No newline at end of file

[beam] branch master updated: Adding DLQ support for ZetaSQL (#25873)

2023-03-17 Thread pabloem 
This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new e8fad9ca2ba Adding DLQ support for ZetaSQL (#25873)
e8fad9ca2ba is described below

commit e8fad9ca2ba80220b4817ddba95aabc0a48067dd
Author: Pablo Estrada 
AuthorDate: Fri Mar 17 18:14:03 2023 -0400

Adding DLQ support for ZetaSQL (#25873)
---
 .../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 126 -
 1 file changed, 97 insertions(+), 29 deletions(-)

diff --git 
a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRel.java
 
b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRel.java
index 744fbd0bcd4..dee87d370e6 100644
--- 
a/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRel.java
+++ 
b/sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcRel.java
@@ -37,18 +37,24 @@ import org.apache.beam.sdk.annotations.Internal;
 import org.apache.beam.sdk.extensions.sql.impl.BeamSqlPipelineOptions;
 import org.apache.beam.sdk.extensions.sql.impl.QueryPlanner.QueryParameters;
 import org.apache.beam.sdk.extensions.sql.impl.rel.AbstractBeamCalcRel;
+import org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils;
 import org.apache.beam.sdk.extensions.sql.impl.utils.CalciteUtils;
 import 
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BeamBigQuerySqlDialect;
 import 
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BeamSqlUnparseContext;
 import org.apache.beam.sdk.schemas.FieldAccessDescriptor;
 import org.apache.beam.sdk.schemas.Schema;
+import org.apache.beam.sdk.schemas.utils.SelectHelpers;
 import org.apache.beam.sdk.transforms.DoFn;
 import org.apache.beam.sdk.transforms.PTransform;
 import org.apache.beam.sdk.transforms.ParDo;
 import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
 import org.apache.beam.sdk.values.PCollection;
 import org.apache.beam.sdk.values.PCollectionList;
+import org.apache.beam.sdk.values.PCollectionTuple;
+import org.apache.beam.sdk.values.POutput;
 import org.apache.beam.sdk.values.Row;
+import org.apache.beam.sdk.values.TupleTag;
+import org.apache.beam.sdk.values.TupleTagList;
 import 
org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptCluster;
 import 
org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelTraitSet;
 import org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode;
@@ -64,7 +70,6 @@ import 
org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.sql.fun.SqlStdO
 import 
org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.sql.parser.SqlParserPos;
 import 
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions;
 import 
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.collect.ImmutableList;
-import org.checkerframework.checker.nullness.qual.NonNull;
 import org.checkerframework.checker.nullness.qual.Nullable;
 import org.joda.time.Duration;
 import org.joda.time.Instant;
@@ -83,6 +88,9 @@ public class BeamZetaSqlCalcRel extends AbstractBeamCalcRel {
   private static final int MAX_PENDING_WINDOW = 32;
   private final BeamSqlUnparseContext context;
 
+  private static final TupleTag rows = new TupleTag("output") {};
+  private static final TupleTag errors = new TupleTag("errors") {};
+
   private static String columnName(int i) {
 return "_" + i;
   }
@@ -101,21 +109,36 @@ public class BeamZetaSqlCalcRel extends 
AbstractBeamCalcRel {
 
   @Override
   public PTransform, PCollection> buildPTransform() {
-return new Transform();
+return buildPTransform(null);
+  }
+
+  @Override
+  public PTransform, PCollection> buildPTransform(
+  @Nullable PTransform, ? extends POutput> 
errorsTransformer) {
+return new Transform(errorsTransformer);
   }
 
   @AutoValue
   abstract static class TimestampedFuture {
-private static TimestampedFuture create(Instant t, Future f) {
-  return new AutoValue_BeamZetaSqlCalcRel_TimestampedFuture(t, f);
+private static TimestampedFuture create(Instant t, Future f, Row r) 
{
+  return new AutoValue_BeamZetaSqlCalcRel_TimestampedFuture(t, f, r);
 }
 
 abstract Instant timestamp();
 
 abstract Future future();
+
+abstract Row row();
   }
 
   private class Transform extends PTransform, 
PCollection> {
+
+private final @Nullable PTransform, ? extends POutput> 
errorsTransformer;
+
+Transform(@Nullable PTransform, ? extends POutput> 
errorsTransformer) {
+  this.errorsTransformer = errorsTransformer;
+}
+
 @Override
 public PCollection expand(PCollectionList pinput) {
   Preconditions.checkArgument(
@@ -135,9 +158,10 @@ public class BeamZetaSqlCalcRel extends 
AbstractBeamC

[beam] branch master updated: Optimize change stream connector with more efficient batching and blind writes, and add transaction/query tags (#25718)

2023-03-17 Thread pabloem 
This is an automated email from the ASF dual-hosted git repository.

pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new ac57b8709d3 Optimize change stream connector with more efficient 
batching and blind writes, and add transaction/query tags (#25718)
ac57b8709d3 is described below

commit ac57b8709d3f2db597a5d9e6339545b0f2036a3a
Author: ChangyuLi28 <35211213+changyul...@users.noreply.github.com>
AuthorDate: Fri Mar 17 15:13:27 2023 -0700

Optimize change stream connector with more efficient batching and blind 
writes, and add transaction/query tags (#25718)

* Optimize change stream connector with more efficient batching and blind 
writes, and add transaction/query tags

* Optimize change stream connector with more efficient batching and blind 
writes, and add transaction/query tags

* Apply commit deadline only for metadata table writes

-

Co-authored-by: Changyu Li 
---
 .../MetadataSpannerConfigFactory.java  |  2 +-
 .../action/ChildPartitionsRecordAction.java|  3 +-
 .../action/DetectNewPartitionsAction.java  | 19 --
 .../changestreams/dao/PartitionMetadataDao.java| 68 +++---
 .../action/ChildPartitionsRecordActionTest.java|  4 +-
 .../dao/PartitionMetadataDaoTest.java  | 29 +++--
 6 files changed, 75 insertions(+), 50 deletions(-)

diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/MetadataSpannerConfigFactory.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/MetadataSpannerConfigFactory.java
index 83965b1bfaa..56c67f3194f 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/MetadataSpannerConfigFactory.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/MetadataSpannerConfigFactory.java
@@ -84,7 +84,7 @@ public class MetadataSpannerConfigFactory {
 
 ValueProvider commitDeadline = primaryConfig.getCommitDeadline();
 if (commitDeadline != null) {
-  config = 
config.withCommitDeadline(StaticValueProvider.of(commitDeadline.get()));
+  config = 
config.withCommitDeadline(StaticValueProvider.of(Duration.standardSeconds(60)));
 }
 
 ValueProvider maxCumulativeBackoff = 
primaryConfig.getMaxCumulativeBackoff();
diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/ChildPartitionsRecordAction.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/ChildPartitionsRecordAction.java
index 70286b41778..7fb69d0e7ab 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/ChildPartitionsRecordAction.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/ChildPartitionsRecordAction.java
@@ -155,7 +155,8 @@ public class ChildPartitionsRecordAction {
   } else {
 return false;
   }
-})
+},
+"InsertChildPartition")
 .getResult();
 if (insertedRow && isSplit) {
   metrics.incPartitionRecordSplitCount();
diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/DetectNewPartitionsAction.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/DetectNewPartitionsAction.java
index 934210250f5..73967d2a2a7 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/DetectNewPartitionsAction.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/action/DetectNewPartitionsAction.java
@@ -146,15 +146,22 @@ public class DetectNewPartitionsAction {
   OutputReceiver receiver,
   Timestamp minWatermark,
   TreeMap> batches) {
+List batchPartitionsDifferentCreatedAt = new 
ArrayList<>();
+int numTimestampsHandledSofar = 0;
 for (Map.Entry> batch : 
batches.entrySet()) {
+  numTimestampsHandledSofar++;
   final Timestamp batchCreatedAt = batch.getKey();
-  final List batchPartitions = batch.getValue();
-
-  final Timestamp scheduledAt = updateBatchToScheduled(batchPartitions);
-  if (!tracker.tryClaim(batchCreatedAt)) {
-return ProcessContinuation.stop();
+  final List batchPartitionsSameCreatedAt = 
batch.getValue();
+  batchPartitionsDifferentCreatedAt.addAll(batchPartitionsSameCreatedAt);
+  if (batchPartitionsDifferentCreatedAt.size() >= 200
+  || numTimestampsH

[beam] branch asf-site updated: Publishing website 2023/03/17 22:17:23 at commit e8fad9c

2023-03-17 Thread git-site-role 
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new ad9fd1e0c8b Publishing website 2023/03/17 22:17:23 at commit e8fad9c
ad9fd1e0c8b is described below

commit ad9fd1e0c8bc7bed7e486204818d33fb6625a1bb
Author: jenkins 
AuthorDate: Fri Mar 17 22:17:24 2023 +

Publishing website 2023/03/17 22:17:23 at commit e8fad9c
---
 website/generated-content/sitemap.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/generated-content/sitemap.xml 
b/website/generated-content/sitemap.xml
index 5ee9202fbf7..77029a1c729 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.46.0/2023-03-17T10:40:29-04:00/categories/blog/2023-03-17T10:40:29-04:00/blog/2023-03-17T10:40:29-04:00/categories/2023-03-17T10:40:29-04:00/catego
 [...]
\ No newline at end of file
+http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.46.0/2023-03-17T18:14:03-04:00/categories/blog/2023-03-17T18:14:03-04:00/blog/2023-03-17T18:14:03-04:00/categories/2023-03-17T18:14:03-04:00/catego
 [...]
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new 0c3fb3fe36a Updating config from bot
0c3fb3fe36a is described below

commit 0c3fb3fe36ac07a155f7e53fb0673144d6118ec8
Author: github-actions 
AuthorDate: Fri Mar 17 23:37:23 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25891.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25891.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25891.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25891.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new ef37b6a9726 Updating config from bot
ef37b6a9726 is described below

commit ef37b6a97268c5c95a39f18bca8d35c948e5ddbf
Author: github-actions 
AuthorDate: Sat Mar 18 00:49:09 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25890.json | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25890.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25890.json
new file mode 100644
index 000..62db1666e3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25890.json
@@ -0,0 +1,10 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {
+"python": "pabloem"
+  },
+  "nextAction": "Reviewers",
+  "stopReviewerNotifications": false,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new d84775ff581 Updating config from bot
d84775ff581 is described below

commit d84775ff581e70885207a1e0345da8202e0fa595
Author: github-actions 
AuthorDate: Sat Mar 18 00:49:11 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json 
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 3a3ccd76bde..8287f4956ab 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -6,7 +6,7 @@
 "TheNeuralBit": 1667896849319,
 "ryanthompson591": 1670002443548,
 "tvalentyn": 1678995263753,
-"pabloem": 1678995247594,
+"pabloem": 1679100548105,
 "y1chi": 1667002607045,
 "damccorm": 1679061908460,
 "jrmccluskey": 1679014132643

[beam] branch pr-bot-state updated: Updating config from bot

2023-03-17 Thread github-bot 
This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/pr-bot-state by this push:
 new a1ceda206e8 Updating config from bot
a1ceda206e8 is described below

commit a1ceda206e8354edd5762dc289b993583f693cf3
Author: github-actions 
AuthorDate: Sat Mar 18 01:08:46 2023 +

Updating config from bot
---
 scripts/ci/pr-bot/state/pr-state/pr-25893.json | 8 
 1 file changed, 8 insertions(+)

diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25893.json 
b/scripts/ci/pr-bot/state/pr-state/pr-25893.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25893.json
@@ -0,0 +1,8 @@
+{
+  "commentedAboutFailingChecks": false,
+  "reviewersAssignedForLabels": {},
+  "nextAction": "Author",
+  "stopReviewerNotifications": true,
+  "remindAfterTestsPass": [],
+  "committerAssigned": false
+}
\ No newline at end of file

[beam] branch master updated: Disable firestore integration tests temporarily (#25891)

2023-03-17 Thread yhu 
This is an automated email from the ASF dual-hosted git repository.

yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
 new 4e96e700c2d Disable firestore integration tests temporarily (#25891)
4e96e700c2d is described below

commit 4e96e700c2d08865ee05b6ac19de76232a8d78ad
Author: Kenneth Knowles 
AuthorDate: Fri Mar 17 20:37:27 2023 -0700

Disable firestore integration tests temporarily (#25891)
---
 sdks/java/io/google-cloud-platform/build.gradle | 5 +
 1 file changed, 5 insertions(+)

diff --git a/sdks/java/io/google-cloud-platform/build.gradle 
b/sdks/java/io/google-cloud-platform/build.gradle
index 8b264a9c8b3..fa696306f9e 100644
--- a/sdks/java/io/google-cloud-platform/build.gradle
+++ b/sdks/java/io/google-cloud-platform/build.gradle
@@ -203,6 +203,11 @@ task integrationTest(type: Test, dependsOn: 
processTestResources) {
   exclude '**/BigQueryIOStorageWriteIT.class'
   exclude '**/BigQueryToTableIT.class'
   exclude '**/BigQueryIOJsonTest.class'
+
+  // Failing due to Firestore service-side changes
+  // https://github.com/apache/beam/issues/25851
+  exclude '**/firestore/it/**/*.class'
+
   maxParallelForks 4
   classpath = sourceSets.test.runtimeClasspath
   testClassesDirs = sourceSets.test.output.classesDirs

[beam] branch asf-site updated: Publishing website 2023/03/18 04:15:45 at commit 4e96e70

2023-03-17 Thread git-site-role 
This is an automated email from the ASF dual-hosted git repository.

git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 2a365356c84 Publishing website 2023/03/18 04:15:45 at commit 4e96e70
2a365356c84 is described below

commit 2a365356c84dad2de072c86126415cae9c6c2092
Author: jenkins 
AuthorDate: Sat Mar 18 04:15:45 2023 +

Publishing website 2023/03/18 04:15:45 at commit 4e96e70
---
 website/generated-content/sitemap.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/website/generated-content/sitemap.xml 
b/website/generated-content/sitemap.xml
index 77029a1c729..8b74f7dc6e6 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.46.0/2023-03-17T18:14:03-04:00/categories/blog/2023-03-17T18:14:03-04:00/blog/2023-03-17T18:14:03-04:00/categories/2023-03-17T18:14:03-04:00/catego
 [...]
\ No newline at end of file
+http://www.sitemaps.org/schemas/sitemap/0.9"; 
xmlns:xhtml="http://www.w3.org/1999/xhtml";>/blog/beam-2.46.0/2023-03-17T20:37:27-07:00/categories/blog/2023-03-17T20:37:27-07:00/blog/2023-03-17T20:37:27-07:00/categories/2023-03-17T20:37:27-07:00/catego
 [...]
\ No newline at end of file