[beam] branch tvalentyn-patch-3 updated (f8d05928c4d -> 1ffabf4ea78)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git omit f8d05928c4d Conflict resolution. omit 0e24d973f41 Further restrict allowed Pandas versions. add 1ffabf4ea78 Further restrict allowed Pandas versions. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (f8d05928c4d) \ N -- N -- N refs/heads/tvalentyn-patch-3 (1ffabf4ea78) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: sdks/python/test-suites/tox/py37/build.gradle | 14 -- sdks/python/tox.ini | 7 ++- 2 files changed, 2 insertions(+), 19 deletions(-)
[beam] branch nightly-refs/heads/master updated (aee2c844ad8 -> d20d0b01c3c)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from aee2c844ad8 Clarify llm download/loading instructions (#25145) add bb582d8db12 Allow for setMaxRetryJobs in BigQueryIO to be configurable (#25224) add fd4ea9b6bce delete file used in internal testing (#25339) add bbe27e96de0 Don't discard output column names add 9178456b38f Merge pull request #25290: Don't discard output column names add cd8eb0c9c68 Fix output timestamp for multi output receiver in FnApiDoFnRunner #25344 (#25349) add dc632029f0c Change UnboundedScheduledExecutorService to avoid creating threads when (#25300) add d20d0b01c3c Upgrading spring-expression to latest patch version (#25348) No new revisions were added by this update. Summary of changes: playground/backend/containers/java/Dockerfile | 2 +- .../go/pkg/beam/core/runtime/xlangx/expand_test.go | 41 .../util/UnboundedScheduledExecutorService.java| 28 +-- .../UnboundedScheduledExecutorServiceTest.java | 54 ++ .../extensions/sql/impl/planner/BeamRuleSets.java | 4 -- .../sql/impl/rel/AbstractBeamCalcRel.java | 10 .../sql/impl/rule/LogicalCalcMergeRule.java| 42 - .../extensions/sql/BeamSqlMultipleSchemasTest.java | 21 + .../sql/impl/rule/IOPushDownRuleTest.java | 2 - ...tTableProviderWithFilterAndProjectPushDown.java | 2 - .../test/TestTableProviderWithFilterPushDown.java | 2 - .../test/TestTableProviderWithProjectPushDown.java | 2 - .../sql/zetasql/ZetaSQLQueryPlanner.java | 15 +- .../apache/beam/fn/harness/FnApiDoFnRunner.java| 7 ++- .../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 12 - .../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 14 ++ sdks/java/io/kafka/build.gradle| 2 +- 17 files changed, 140 insertions(+), 120 deletions(-) delete mode 100644 sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go delete mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/LogicalCalcMergeRule.java
[beam] branch tvalentyn-patch-3 updated (0e24d973f41 -> f8d05928c4d)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git from 0e24d973f41 Further restrict allowed Pandas versions. add f8d05928c4d Conflict resolution. No new revisions were added by this update. Summary of changes: sdks/python/test-suites/tox/py37/build.gradle | 29 --- sdks/python/test-suites/tox/py38/build.gradle | 19 -- 2 files changed, 21 insertions(+), 27 deletions(-)
[beam] branch tvalentyn-patch-3 updated (a5193922220 -> 0e24d973f41)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git discard a519390 Further restrict allowed Pandas versions. omit d118e4ffe39 Require Pandas 1.2.0 or above. omit 1a5fe93d2bf Add a note to update build-requirements.txt omit f2ce6cc04ba Restore the lower bound. omit 18075f5c2e0 Update upper bound for numpy. add 7ad44c84585 Handle schema updates in Storage API writes. add f5020e7ac2b Merge pull request #24145: Handle updates to table schema when using Storage API writes. add 9cecec3f74c Bump cloud.google.com/go/storage from 1.28.1 to 1.29.0 in /sdks (#25095) add 5e1ebee8b47 Allow to set timeout for finishing a remote bundle in Samza portable runner (#25031) add e379c23c885 Fix truncate copy job when WRITE_TRUNCATE in BigQuery batch load (#25101) add 482401411b7 fix(sec): upgrade torch to 1.13.1 (#24933) add 4dad3c696d4 [#24515] Delete the JRH (#24967) add cd20288318d Support DoFn metrics in portable Samza Runner (#25068) add 5cfc09cd65d Add I/O Standards Page (#24962) add 897fffdeda5 Add more info in documentation (#25099) add b3aa2e89489 Fix Python PreCommit dependency compatibility test not run or redundant setup (#25115) add 6029a57a4d7 Move CombineValues override to non portable overrides (#25106) add 54aad800535 Remove kileys from Java reviewers add 459d6b2543a Merge pull request #25124 from kileys/remove add 5e3b41788ab Exclude UsesMultimapState in Samza validatesPortableRunner tests (#25121) add 633c51242e9 Add GraalSystems (#23651) add 93ca2fe53ce Bump timeout for default transient expansion service (#25111) add b1aba9d24e8 #20812 handle @RequiresStableInput in portable flink (#22889) add 6ba647333c4 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#25128) add 6b8f8c09f5e Initial start of a yaml-based declarative way of building pipelines. (#24667) add 3d9318bff08 Remove deprecated first implementations of data channel connector code on the runner side (#25104) add 9aa2c529c51 Adding autoservice annotations to all SchemaTransformProviders (#25107) add 89cd0599708 Add working example for Per Entity Training (#25081) add 860b05ff66e Add documentation on typescript IOs. add d9145257b37 Merge pull request #25071 Restore documentation on typescript IOs. add 0bcb50edde5 Adds a reference x-lang RunInference Javadoc add b7c07e26472 Merge pull request #25148: Adds a reference x-lang RunInference Javadoc add 299a7b01308 Drop '2' from BeamFnDataGrpcMultiplexer2 and BeamFnDataInboundObserver2 (#25144) add a96afe2c57c [BEAM-12164] Enable postgres IT for change streams (#25152) add b49ef57ddd2 update cloudpickle version (#25142) add 72d2776bf3a [ToB][Frontend] Gradle tasks for ToB frontend (#24919) add 6d6f9958734 Remove databaseRole from metadata DB requests in SpannerIO.ReadChangeStream (#25108) (#25141) add d688d425938 Update typehint code to allow primitive composite types in python 3.9+ (#25147) add 91efbd3a933 Fix intruction -> instruction typo add e8484a790e3 Fix inconsistent cache bug add 1ecb4ef6d6f Merge pull request #25159: Fix inconsistent cache bug add b0c594167b4 Increase default expansion service timeout add 48712b26e67 Merge pull request #25169: Increase default expansion service timeout add d39e9c5779b Further increase transient expansion service timeout (#25176) add 0c2a1683e5f Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#25182) add b497cc4a98a Adds the SampleDataRequest/Response (#25065) add 76f3a5919fe Update chromedriver-binary requirement in /sdks/python (#25178) add 6676646c8b1 Bump google.golang.org/grpc from 1.52.0 to 1.52.3 in /sdks (#25181) add 67b5b368353 Initial commit of boilerplate setup of change stream pipeline for bigtable add f565a94b9b3 Merge pull request #25153 from tonytanger/initial_cdc add 13a489efc9a Fix SchemaTransform identifiers add 2c45ac4519b Merge pull request #25175: Fix SchemaTransform identifiers add f07241b0983 Fix a couple typos caught by an internal linter (#25188) add 61045b670bf Update Dataflow container versions (#25192) add 40243ffc284 update GCP cloud libraries BOM to 26.5.0 add 3bf17574c6f Merge pull request #25194: update GCP cloud libraries BOM to 26.5.0 add bfb6a8ea0a8 Fixing a table existence verifier for BQ add 2bcec868da0 Merge pull request #25195 from pabloem/fix-verify-table-bq add 1c71463118c Add a bunch of convenience methods for nullable top level schema fields to make schemas a bit easier to read in Java add 0b7394b4486 Updated to use existing addNullableField vs doing it directly add 423cf9aded6 Merge branch 'apache:master' into schema-convenience-functions
[beam] branch tvalentyn-patch-3 updated (d118e4ffe39 -> a5193922220)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git from d118e4ffe39 Require Pandas 1.2.0 or above. add a519390 Further restrict allowed Pandas versions. No new revisions were added by this update. Summary of changes: sdks/python/setup.py | 11 --- sdks/python/test-suites/tox/py37/build.gradle | 17 + sdks/python/test-suites/tox/py38/build.gradle | 12 sdks/python/tox.ini | 3 ++- 4 files changed, 23 insertions(+), 20 deletions(-)
[beam] branch tvalentyn-patch-3 updated (f2ce6cc04ba -> d118e4ffe39)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch tvalentyn-patch-3 in repository https://gitbox.apache.org/repos/asf/beam.git from f2ce6cc04ba Restore the lower bound. add 1a5fe93d2bf Add a note to update build-requirements.txt add d118e4ffe39 Require Pandas 1.2.0 or above. No new revisions were added by this update. Summary of changes: sdks/python/build-requirements.txt | 2 +- sdks/python/setup.py | 6 +++--- sdks/python/tox.ini| 3 +-- 3 files changed, 5 insertions(+), 6 deletions(-)
[beam-starter-typescript] branch main updated: typo
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/beam-starter-typescript.git The following commit(s) were added to refs/heads/main by this push: new e542f8f typo e542f8f is described below commit e542f8f8f70e8bf0b321c05315c97ebcc7e2c3f8 Author: Robert Bradshaw AuthorDate: Mon Feb 6 13:54:40 2023 -0800 typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index da60dc5..3b80045 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ This can be done with npm install ``` -The pipeline is then build with +The pipeline is then built with ```sh npm run build
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c6102dd26ca Updating config from bot c6102dd26ca is described below commit c6102dd26ca9a2e253b9ba8399fd6c93db78ee5a Author: github-actions AuthorDate: Mon Feb 6 21:14:19 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index cb6db22f10f..68d84ef2b7c 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -4,7 +4,7 @@ "chamikaramj": 1675477355154, "johnjcasey": 1675647135455, "pabloem": 1675611261019, -"Abacn": 1675472198459, +"Abacn": 1675718029620, "ahmedabu98": 1675663534387 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 58881ca4f17 Updating config from bot
58881ca4f17 is described below
commit 58881ca4f1799f4e1149bb72b3774ae01eaf5805
Author: github-actions
AuthorDate: Mon Feb 6 21:14:07 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 0af34d4a4d0..ff9476a2e40 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1675699528474,
-"jrmccluskey": 1675444397309,
+"jrmccluskey": 1675718029620,
"youngoli": 1657688896155,
"damccorm": 1675454717804,
"riteshghorse": 1675472198459
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ce2bb7f2c40 Updating config from bot
ce2bb7f2c40 is described below
commit ce2bb7f2c40825f0193408a452b187b3e56460ae
Author: github-actions
AuthorDate: Mon Feb 6 21:13:56 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25352.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25352.json
b/scripts/ci/pr-bot/state/pr-state/pr-25352.json
new file mode 100644
index 000..f71d29b31d3
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25352.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey",
+"io": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] 01/01: fix doc on bq sxtorage
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a commit to branch pabloem-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git commit 6d971168e8c908797d4366e3c706914949abe7ce Author: Pablo Estrada AuthorDate: Mon Feb 6 15:41:50 2023 -0500 fix doc on bq sxtorage --- .../www/site/content/en/documentation/io/built-in/google-bigquery.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/www/site/content/en/documentation/io/built-in/google-bigquery.md b/website/www/site/content/en/documentation/io/built-in/google-bigquery.md index 6f12a4f6bcf..8ab3b0a9a04 100644 --- a/website/www/site/content/en/documentation/io/built-in/google-bigquery.md +++ b/website/www/site/content/en/documentation/io/built-in/google-bigquery.md @@ -336,8 +336,8 @@ BigQuery. SDK versions before 2.25.0 support the BigQuery Storage API as an and use the pre-GA BigQuery Storage API surface. Callers should migrate pipelines which use the BigQuery Storage API to use SDK version 2.25.0 or later. -The Beam SDK for Python does not support the BigQuery Storage API. See -[Issue 20687](https://github.com/apache/beam/issues/20687)). +The Beam SDK for Python supports the BigQuery Storage API. Enable it +by passing a `method` parameter to it. Updating your code
[beam] branch pabloem-patch-2 created (now 6d971168e8c)
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a change to branch pabloem-patch-2 in repository https://gitbox.apache.org/repos/asf/beam.git at 6d971168e8c fix doc on bq sxtorage This branch includes the following new commits: new 6d971168e8c fix doc on bq sxtorage The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] branch master updated: Upgrading spring-expression to latest patch version (#25348)
This is an automated email from the ASF dual-hosted git repository.
pabloem pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new d20d0b01c3c Upgrading spring-expression to latest patch version
(#25348)
d20d0b01c3c is described below
commit d20d0b01c3c6bcde551420f36e13d794c930f1e2
Author: Pablo Estrada
AuthorDate: Mon Feb 6 12:36:22 2023 -0800
Upgrading spring-expression to latest patch version (#25348)
---
playground/backend/containers/java/Dockerfile | 2 +-
sdks/java/io/kafka/build.gradle | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/playground/backend/containers/java/Dockerfile
b/playground/backend/containers/java/Dockerfile
index 2a1561c3b0b..759118ddb99 100644
--- a/playground/backend/containers/java/Dockerfile
+++ b/playground/backend/containers/java/Dockerfile
@@ -53,7 +53,7 @@ RUN mvn dependency:copy-dependencies
FROM apache/beam_java8_sdk:$BEAM_VERSION
ARG BEAM_VERSION
-ARG SPRING_VERSION=5.3.18
+ARG SPRING_VERSION=5.3.25
ARG KAFKA_CLIENTS_VERSION=2.3.1
ENV SERVER_IP=0.0.0.0
ENV SERVER_PORT=8080
diff --git a/sdks/java/io/kafka/build.gradle b/sdks/java/io/kafka/build.gradle
index e9b3b255b9b..8d64cf0bf97 100644
--- a/sdks/java/io/kafka/build.gradle
+++ b/sdks/java/io/kafka/build.gradle
@@ -70,7 +70,7 @@ dependencies {
implementation library.java.joda_time
implementation library.java.jackson_annotations
implementation library.java.jackson_databind
- implementation "org.springframework:spring-expression:5.3.18"
+ implementation "org.springframework:spring-expression:5.3.25"
implementation ("io.confluent:kafka-avro-serializer:5.3.2") {
// zookeeper depends on "spotbugs-annotations:3.1.9" which clashes with
current
// "spotbugs-annotations:3.1.12" used in Beam. Not required.
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6ff482e60ef Updating config from bot 6ff482e60ef is described below commit 6ff482e60ef78e774e8174437c3cecfe4a29746f Author: github-actions AuthorDate: Mon Feb 6 19:16:06 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 99fd1022d40..a99ad0a0420 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -6,7 +6,7 @@ "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, "tvalentyn": 1675710904325, -"pabloem": 1675488850784, +"pabloem": 1675710947280, "y1chi": 1667002607045, "damccorm": 1675611261019, "jrmccluskey": 1675663534385
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 3e1ec8bd7b8 Updating config from bot
3e1ec8bd7b8 is described below
commit 3e1ec8bd7b8fb634aec493abdcf2fffa1e46c086
Author: github-actions
AuthorDate: Mon Feb 6 19:15:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25226.json | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25226.json
b/scripts/ci/pr-bot/state/pr-state/pr-25226.json
index 6272b4e797d..80394fc1997 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-25226.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25226.json
@@ -1,10 +1,10 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"python": "AnandInguva"
+"python": "pabloem"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
- "committerAssigned": false
+ "committerAssigned": true
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f5cc7d4675a Updating config from bot f5cc7d4675a is described below commit f5cc7d4675aa8451766009881d9e66f20f6fa913 Author: github-actions AuthorDate: Mon Feb 6 19:15:16 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 2a29abb0f3d..99fd1022d40 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1675400711771, +"tvalentyn": 1675710904325, "pabloem": 1675488850784, "y1chi": 1667002607045, "damccorm": 1675611261019,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b942b51650d Updating config from bot
b942b51650d is described below
commit b942b51650d5ad0d3a6bf2b3f8503193fce29857
Author: github-actions
AuthorDate: Mon Feb 6 19:15:11 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25351.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25351.json
b/scripts/ci/pr-bot/state/pr-state/pr-25351.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25351.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Change UnboundedScheduledExecutorService to avoid creating threads when (#25300)
This is an automated email from the ASF dual-hosted git repository.
lcwik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new dc632029f0c Change UnboundedScheduledExecutorService to avoid creating
threads when (#25300)
dc632029f0c is described below
commit dc632029f0cce4d9ee6ee139cad7236388cd7b71
Author: Sam Whittle
AuthorDate: Mon Feb 6 19:57:14 2023 +0100
Change UnboundedScheduledExecutorService to avoid creating threads when
(#25300)
there is contention on the SynchronousQueue.
ThreadPoolExecutor creates a new thread if offer(Runnable) returns false.
With a SynchronousQueue under contention, this may return false even if
there are already idle threads in the pool. To avoid creating a thread
unnecessarily and growing the pool we modify offer to wait with a small
timeout of 100ms. This may delay initial execution on the pool but that
should be rare as it is encountered only once the high-watermark is
exceeded.
Also add a timeout to the non-core threads so that the pool reduces size.
Both of these changes are motivated to reduce memory usage due to per-thread
buffers in Beam and AvroCoder.
Add a test contending adds on the pool which previously showed the pool
growing
beyond the necessary amount (ie created 104 threads when 100 is sufficient).
---
.../util/UnboundedScheduledExecutorService.java| 28 +--
.../UnboundedScheduledExecutorServiceTest.java | 54 ++
2 files changed, 77 insertions(+), 5 deletions(-)
diff --git
a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorService.java
b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorService.java
index 9b38604a7a8..1c243ca2ada 100644
---
a/sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorService.java
+++
b/sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorService.java
@@ -17,6 +17,8 @@
*/
package org.apache.beam.sdk.util;
+import static java.util.concurrent.TimeUnit.HOURS;
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
import static java.util.concurrent.TimeUnit.NANOSECONDS;
import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
@@ -63,7 +65,6 @@ import org.checkerframework.checker.nullness.qual.Nullable;
*
*/
public final class UnboundedScheduledExecutorService implements
ScheduledExecutorService {
-
/**
* A {@link FutureTask} that handles periodically rescheduling tasks.
*
@@ -181,7 +182,7 @@ public final class UnboundedScheduledExecutorService
implements ScheduledExecuto
private final AtomicLong sequencer = new AtomicLong();
private final NanoClock clock;
- private final ThreadPoolExecutor threadPoolExecutor;
+ @VisibleForTesting final ThreadPoolExecutor threadPoolExecutor;
@VisibleForTesting final PriorityQueue> tasks;
private final AbstractExecutorService invokeMethodsAdapter;
private final Future launchTasks;
@@ -201,9 +202,26 @@ public final class UnboundedScheduledExecutorService
implements ScheduledExecuto
new ThreadPoolExecutor(
0,
Integer.MAX_VALUE, // Allow an unlimited number of re-usable
threads.
-Long.MAX_VALUE,
-TimeUnit.NANOSECONDS, // Keep non-core threads alive forever.
-new SynchronousQueue<>(),
+// Put a high-timeout on non-core threads. This reduces memory for
per-thread caches
+// over time.
+1,
+HOURS,
+new SynchronousQueue() {
+ @Override
+ public boolean offer(Runnable r) {
+try {
+ // By blocking for a little we hope to delay thread creation
if there are existing
+ // threads that will eventually return. We expect this
timeout to be very rarely
+ // hit as the high-watermark of necessary threads will
remain for up to an hour.
+ if (offer(r, 10, MILLISECONDS)) {
+return true;
+ }
+} catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+}
+return false;
+ }
+},
threadFactoryBuilder.build());
// Create an internal adapter so that execute does not re-wrap the
ScheduledFutureTask again
diff --git
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorServiceTest.java
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorServiceTest.java
index ededf69c914..b8efa292bd2 100644
---
a/sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnboundedScheduledExecutorServiceTest.java
+++
b/sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnboundedS
[beam] branch master updated: Fix output timestamp for multi output receiver in FnApiDoFnRunner #25344 (#25349)
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new cd8eb0c9c68 Fix output timestamp for multi output receiver in FnApiDoFnRunner #25344 (#25349) cd8eb0c9c68 is described below commit cd8eb0c9c68f115a868b8e9203e0a2e3a1acdf0d Author: Luke Cwik AuthorDate: Mon Feb 6 10:33:03 2023 -0800 Fix output timestamp for multi output receiver in FnApiDoFnRunner #25344 (#25349) * Fix output timestamp for multi output receiver in FnApiDoFnRunner #25344 Fixes #25344 --- .../src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java b/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java index 13d85d27006..0cfcb0a84f2 100644 --- a/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java +++ b/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java @@ -2529,8 +2529,7 @@ public class FnApiDoFnRunner
[beam] branch master updated (fd4ea9b6bce -> 9178456b38f)
This is an automated email from the ASF dual-hosted git repository. apilloud pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from fd4ea9b6bce delete file used in internal testing (#25339) add bbe27e96de0 Don't discard output column names add 9178456b38f Merge pull request #25290: Don't discard output column names No new revisions were added by this update. Summary of changes: .../extensions/sql/impl/planner/BeamRuleSets.java | 4 --- .../sql/impl/rel/AbstractBeamCalcRel.java | 10 ++ .../sql/impl/rule/LogicalCalcMergeRule.java| 42 -- .../extensions/sql/BeamSqlMultipleSchemasTest.java | 21 +++ .../sql/impl/rule/IOPushDownRuleTest.java | 2 -- ...tTableProviderWithFilterAndProjectPushDown.java | 2 -- .../test/TestTableProviderWithFilterPushDown.java | 2 -- .../test/TestTableProviderWithProjectPushDown.java | 2 -- .../sql/zetasql/ZetaSQLQueryPlanner.java | 15 ++-- 9 files changed, 33 insertions(+), 67 deletions(-) delete mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/LogicalCalcMergeRule.java
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b426ffeee15 Updating config from bot b426ffeee15 is described below commit b426ffeee15bb1ac2292b48163fee586d26252dc Author: github-actions AuthorDate: Mon Feb 6 17:55:26 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-25342.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25342.json b/scripts/ci/pr-bot/state/pr-state/pr-25342.json index 8ea94810368..ae9024f9d4e 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-25342.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-25342.json @@ -4,7 +4,7 @@ "go": "lostluck" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch release-2.45.0 updated: delete file used in internal testing (#25342)
This is an automated email from the ASF dual-hosted git repository.
johncasey pushed a commit to branch release-2.45.0
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/release-2.45.0 by this push:
new b29e8f34710 delete file used in internal testing (#25342)
b29e8f34710 is described below
commit b29e8f34710a0259ab6db6c3892e5f909342b559
Author: Ritesh Ghorse
AuthorDate: Mon Feb 6 12:46:36 2023 -0500
delete file used in internal testing (#25342)
---
.../go/pkg/beam/core/runtime/xlangx/expand_test.go | 41 --
1 file changed, 41 deletions(-)
diff --git a/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
b/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
deleted file mode 100644
index f4949efc3f0..000
--- a/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
+++ /dev/null
@@ -1,41 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one or more
-// contributor license agreements. See the NOTICE file distributed with
-// this work for additional information regarding copyright ownership.
-// The ASF licenses this file to You under the Apache License, Version 2.0
-// (the "License"); you may not use this file except in compliance with
-// the License. You may obtain a copy of the License at
-//
-//http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package xlangx
-
-import (
- "strings"
- "testing"
-
- "github.com/apache/beam/sdks/v2/go/pkg/beam/core"
-)
-
-func TestStartAutomated(t *testing.T) {
- // TODO: https://github.com/apache/beam/issues/25303
- t.Skipf("need a release SDK version to test auto python expansion
service")
- if strings.HasSuffix(core.SdkVersion, ".dev") {
- t.Skipf("need a released SDK version to test auto python
expansion service, got: %s", core.SdkVersion)
- }
- sp, addr, err :=
startPythonExpansionService("apache_beam.runners.portability.expansion_service_main",
"")
- if err != nil {
- t.Fatal(err)
- }
- if addr == "" {
- t.Fatal("no address")
- }
- if err := sp(); err != nil {
- t.Fatal("error stoping service")
- }
-}
[beam] branch master updated: delete file used in internal testing (#25339)
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new fd4ea9b6bce delete file used in internal testing (#25339)
fd4ea9b6bce is described below
commit fd4ea9b6bcebfac1c161805fa8d741cf20cc38eb
Author: Ritesh Ghorse
AuthorDate: Mon Feb 6 12:38:34 2023 -0500
delete file used in internal testing (#25339)
---
.../go/pkg/beam/core/runtime/xlangx/expand_test.go | 41 --
1 file changed, 41 deletions(-)
diff --git a/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
b/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
deleted file mode 100644
index f4949efc3f0..000
--- a/sdks/go/pkg/beam/core/runtime/xlangx/expand_test.go
+++ /dev/null
@@ -1,41 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one or more
-// contributor license agreements. See the NOTICE file distributed with
-// this work for additional information regarding copyright ownership.
-// The ASF licenses this file to You under the Apache License, Version 2.0
-// (the "License"); you may not use this file except in compliance with
-// the License. You may obtain a copy of the License at
-//
-//http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package xlangx
-
-import (
- "strings"
- "testing"
-
- "github.com/apache/beam/sdks/v2/go/pkg/beam/core"
-)
-
-func TestStartAutomated(t *testing.T) {
- // TODO: https://github.com/apache/beam/issues/25303
- t.Skipf("need a release SDK version to test auto python expansion
service")
- if strings.HasSuffix(core.SdkVersion, ".dev") {
- t.Skipf("need a released SDK version to test auto python
expansion service, got: %s", core.SdkVersion)
- }
- sp, addr, err :=
startPythonExpansionService("apache_beam.runners.portability.expansion_service_main",
"")
- if err != nil {
- t.Fatal(err)
- }
- if addr == "" {
- t.Fatal("no address")
- }
- if err := sp(); err != nil {
- t.Fatal("error stoping service")
- }
-}
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 68fadf30f5d Updating config from bot
68fadf30f5d is described below
commit 68fadf30f5db01e515a7929885d7f2f6086aa63e
Author: github-actions
AuthorDate: Mon Feb 6 17:22:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25350.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25350.json
b/scripts/ci/pr-bot/state/pr-state/pr-25350.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25350.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ec665df0444 Updating config from bot
ec665df0444 is described below
commit ec665df0444fb8c794453893d015f0ae811b9584
Author: github-actions
AuthorDate: Mon Feb 6 17:05:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25349.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25349.json
b/scripts/ci/pr-bot/state/pr-state/pr-25349.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25349.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new e37bf1eb297 Updating config from bot
e37bf1eb297 is described below
commit e37bf1eb297878fc26a360a92473767c71cab830
Author: github-actions
AuthorDate: Mon Feb 6 16:49:13 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25321.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25321.json
b/scripts/ci/pr-bot/state/pr-state/pr-25321.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25321.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c1e9138035e Updating config from bot
c1e9138035e is described below
commit c1e9138035ead23c0c0414aeaf1be0c6fc30ec66
Author: github-actions
AuthorDate: Mon Feb 6 16:46:12 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25348.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25348.json
b/scripts/ci/pr-bot/state/pr-state/pr-25348.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25348.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f912c2fe949 Updating config from bot
f912c2fe949 is described below
commit f912c2fe94955d6b8b87ac4548bf8fc0d51f7056
Author: github-actions
AuthorDate: Mon Feb 6 16:05:33 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index cfb2a14a2cc..0af34d4a4d0 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1675381127768,
+"lostluck": 1675699528474,
"jrmccluskey": 1675444397309,
"youngoli": 1657688896155,
"damccorm": 1675454717804,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new ec2a98cff88 Updating config from bot
ec2a98cff88 is described below
commit ec2a98cff88553750a5c2fdc866626947417a638
Author: github-actions
AuthorDate: Mon Feb 6 16:05:30 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25342.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25342.json
b/scripts/ci/pr-bot/state/pr-state/pr-25342.json
new file mode 100644
index 000..8ea94810368
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25342.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch revert-25245-beam13015.5 created (now 8e9279c8a88)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch revert-25245-beam13015.5 in repository https://gitbox.apache.org/repos/asf/beam.git at 8e9279c8a88 Revert "Optimize to use cached output receiver instead of creating one on DoFn invocation #21250 (#25245)" This branch includes the following new commits: new 8e9279c8a88 Revert "Optimize to use cached output receiver instead of creating one on DoFn invocation #21250 (#25245)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Revert "Optimize to use cached output receiver instead of creating one on DoFn invocation #21250 (#25245)"
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch revert-25245-beam13015.5
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 8e9279c8a88107d863a8b698f074b0ec8cf8b573
Author: Yi Hu
AuthorDate: Mon Feb 6 10:27:22 2023 -0500
Revert "Optimize to use cached output receiver instead of creating one on
DoFn invocation #21250 (#25245)"
This reverts commit b1c9d8aec07ce72e946bd349eb4345417efbfc9c.
---
.../beam/sdk/transforms/DoFnOutputReceivers.java | 2 +-
.../apache/beam/fn/harness/FnApiDoFnRunner.java| 310 ++---
2 files changed, 17 insertions(+), 295 deletions(-)
diff --git
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnOutputReceivers.java
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnOutputReceivers.java
index 27fbb9754ec..a17264da35d 100644
---
a/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnOutputReceivers.java
+++
b/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/DoFnOutputReceivers.java
@@ -115,7 +115,7 @@ public class DoFnOutputReceivers {
checkState(outputCoder != null, "No output tag for " + tag);
checkState(
outputCoder instanceof SchemaCoder,
- "Output with tag " + tag + " must have a schema in order to call
getRowReceiver");
+ "Output with tag " + tag + " must have a schema in order to call " +
" getRowReceiver");
return DoFnOutputReceivers.rowReceiver(context, tag, (SchemaCoder)
outputCoder);
}
}
diff --git
a/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java
b/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java
index 13d85d27006..2b449e0200b 100644
---
a/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java
+++
b/sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnApiDoFnRunner.java
@@ -85,6 +85,7 @@ import org.apache.beam.sdk.transforms.DoFn;
import org.apache.beam.sdk.transforms.DoFn.BundleFinalizer;
import org.apache.beam.sdk.transforms.DoFn.MultiOutputReceiver;
import org.apache.beam.sdk.transforms.DoFn.OutputReceiver;
+import org.apache.beam.sdk.transforms.DoFnOutputReceivers;
import org.apache.beam.sdk.transforms.DoFnSchemaInformation;
import org.apache.beam.sdk.transforms.SerializableFunction;
import org.apache.beam.sdk.transforms.reflect.DoFnInvoker;
@@ -2412,7 +2413,7 @@ public class FnApiDoFnRunner.ProcessContext
- implements DoFnInvoker.ArgumentProvider,
OutputReceiver {
+ implements DoFnInvoker.ArgumentProvider {
private ProcessBundleContextBase() {
doFn.super();
@@ -2477,112 +2478,17 @@ public class FnApiDoFnRunner outputReceiver(DoFn doFn) {
- return this;
+ return DoFnOutputReceivers.windowedReceiver(this, null);
}
-private final OutputReceiver mainRowOutputReceiver =
-mainOutputSchemaCoder == null
-? null
-: new OutputReceiver() {
- private final SerializableFunction fromRowFunction
=
- mainOutputSchemaCoder.getFromRowFunction();
-
- @Override
- public void output(Row output) {
-ProcessBundleContextBase.this.outputWithTimestamp(
-fromRowFunction.apply(output),
currentElement.getTimestamp());
- }
-
- @Override
- public void outputWithTimestamp(Row output, Instant timestamp) {
-ProcessBundleContextBase.this.outputWithTimestamp(
-fromRowFunction.apply(output), timestamp);
- }
-};
-
@Override
public OutputReceiver outputRowReceiver(DoFn doFn) {
- checkState(
- mainOutputSchemaCoder != null,
- "Output with tag "
- + mainOutputTag
- + " must have a schema in order to call getRowReceiver");
- return mainRowOutputReceiver;
-}
-
-/** A {@link MultiOutputReceiver} which caches created instances to re-use
across bundles. */
-private final MultiOutputReceiver taggedOutputReceiver =
-new MultiOutputReceiver() {
- private final Map, OutputReceiver>
taggedOutputReceivers = new HashMap<>();
- private final Map, OutputReceiver>
taggedRowReceivers = new HashMap<>();
-
- private OutputReceiver createTaggedOutputReceiver(TupleTag
tag) {
-if (tag == null || mainOutputTag.equals(tag)) {
- return (OutputReceiver) ProcessBundleContextBase.this;
-}
-return new OutputReceiver() {
- @Override
- public void output(T output) {
-ProcessBundleContextBase.this.outputWithTimestamp(
-tag, output, currentElement.getTimestamp());
- }
-
- @Override
- public void outputWithTimestamp(T output, Instant timestamp) {
-ProcessBundleContextBas
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bd94c7b2fe0 Updating config from bot
bd94c7b2fe0 is described below
commit bd94c7b2fe0f293107dd3d746a70dc320a40e27b
Author: github-actions
AuthorDate: Mon Feb 6 15:24:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25339.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25339.json
b/scripts/ci/pr-bot/state/pr-state/pr-25339.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25339.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated (aee2c844ad8 -> bb582d8db12)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from aee2c844ad8 Clarify llm download/loading instructions (#25145) add bb582d8db12 Allow for setMaxRetryJobs in BigQueryIO to be configurable (#25224) No new revisions were added by this update. Summary of changes: .../org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java| 12 +++- .../beam/sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 14 ++ 2 files changed, 25 insertions(+), 1 deletion(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 9f83c190aec Updating config from bot 9f83c190aec is described below commit 9f83c190aec0f519dc6e5bada234051db0c04795 Author: github-actions AuthorDate: Mon Feb 6 14:30:25 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-25338.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25338.json b/scripts/ci/pr-bot/state/pr-state/pr-25338.json index ab445634a3b..fbd10f56033 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-25338.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-25338.json @@ -4,7 +4,7 @@ "java": "apilloud" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new f1fe736364c Updating config from bot f1fe736364c is described below commit f1fe736364c5e682d5f385ca4052db30d5a320e0 Author: github-actions AuthorDate: Mon Feb 6 14:05:17 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 545662954e2..1545d8fd867 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -5,6 +5,6 @@ "kennknowles": 1675472198459, "robertwb": 1675477355152, "kileys": 1674428678843, -"apilloud": 1675460269442 +"apilloud": 1675692311366 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new dbc6d8df554 Updating config from bot
dbc6d8df554 is described below
commit dbc6d8df5549962162284669ce8d9e7959ce7d41
Author: github-actions
AuthorDate: Mon Feb 6 14:05:14 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-25338.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-25338.json
b/scripts/ci/pr-bot/state/pr-state/pr-25338.json
new file mode 100644
index 000..ab445634a3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-25338.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "apilloud"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
