(beam) branch ml_transform_criteo updated (26bd7e70e5b -> 78c37f9e3f7)
This is an automated email from the ASF dual-hosted git repository.
anandinguva pushed a change to branch ml_transform_criteo
in repository https://gitbox.apache.org/repos/asf/beam.git
from 26bd7e70e5b Add criteo benchmark for MLTransform
add 78c37f9e3f7 Skip few tests
No new revisions were added by this update.
Summary of changes:
.../testing/benchmarks/cloudml/cloudml_benchmark_test.py | 5 -
.../testing/benchmarks/{cloudml => }/mltransform/__init__.py | 0
.../testing/benchmarks/{cloudml => }/mltransform/criteo.py | 0
.../testing/benchmarks/{cloudml => }/mltransform/criteo_test.py| 7 +--
4 files changed, 5 insertions(+), 7 deletions(-)
rename sdks/python/apache_beam/testing/benchmarks/{cloudml =>
}/mltransform/__init__.py (100%)
rename sdks/python/apache_beam/testing/benchmarks/{cloudml =>
}/mltransform/criteo.py (100%)
rename sdks/python/apache_beam/testing/benchmarks/{cloudml =>
}/mltransform/criteo_test.py (89%)
(beam) branch asf-site updated: Publishing website 2023/12/13 05:39:31 at commit 8be85d2
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 2baaeb062df Publishing website 2023/12/13 05:39:31 at commit 8be85d2 2baaeb062df is described below commit 2baaeb062df384f111c76724abad2c1197ca1e30 Author: runner AuthorDate: Wed Dec 13 05:39:32 2023 + Publishing website 2023/12/13 05:39:31 at commit 8be85d2 --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index a4dbbb7e351..950edbe5cea 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.52.0/2023-12-12T22:19:06+01:00/categories/blog/2023-12-12T22:19:06+01:00/blog/2023-12-12T22:19:06+01:00/categories/2023-12-12T22:19:06+01:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.52.0/2023-12-12T23:15:06-05:00/categories/blog/2023-12-12T23:15:06-05:00/blog/2023-12-12T23:15:06-05:00/categories/2023-12-12T23:15:06-05:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 05e5c64ab64 Updating config from bot
05e5c64ab64 is described below
commit 05e5c64ab645586e69ce7532dc57b16075e0c382
Author: github-actions
AuthorDate: Wed Dec 13 05:05:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29738.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29738.json
b/scripts/ci/pr-bot/state/pr-state/pr-29738.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29738.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f2d727ad426 Updating config from bot
f2d727ad426 is described below
commit f2d727ad426fcd202ede593f2056be6a1e033af6
Author: github-actions
AuthorDate: Wed Dec 13 05:05:50 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29739.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29739.json
b/scripts/ci/pr-bot/state/pr-state/pr-29739.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29739.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5475f06da6a Updating config from bot
5475f06da6a is described below
commit 5475f06da6a00e112ebdb3e2975ad31a1d1d655f
Author: github-actions
AuthorDate: Wed Dec 13 05:05:47 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29740.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29740.json
b/scripts/ci/pr-bot/state/pr-state/pr-29740.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29740.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a777a163e9c Updating config from bot
a777a163e9c is described below
commit a777a163e9c9059aa4c7414115567072921504ec
Author: github-actions
AuthorDate: Wed Dec 13 04:35:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 1fd77b57bf4..3b8ab224546 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1702272877414,
+"lostluck": 1702442101196,
"jrmccluskey": 1702404385813,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b5f9d8714d2 Updating config from bot
b5f9d8714d2 is described below
commit b5f9d8714d26669408695cead17869c76e7a5f65
Author: github-actions
AuthorDate: Wed Dec 13 04:35:03 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29734.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29734.json
b/scripts/ci/pr-bot/state/pr-state/pr-29734.json
new file mode 100644
index 000..8ea94810368
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29734.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1633bc192f2 Updating config from bot
1633bc192f2 is described below
commit 1633bc192f274e3c211392f51b3332d448296a88
Author: github-actions
AuthorDate: Wed Dec 13 04:35:07 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29733.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29733.json
b/scripts/ci/pr-bot/state/pr-state/pr-29733.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29733.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new bcac0a68db7 Updating config from bot bcac0a68db7 is described below commit bcac0a68db7538eba2e7961da051573bcdab6967 Author: github-actions AuthorDate: Wed Dec 13 04:35:09 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29732.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29732.json b/scripts/ci/pr-bot/state/pr-state/pr-29732.json index 2040c933220..b85bcf8e5c9 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29732.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29732.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch dependabot/gradle/com.gradle.enterprise-3.16 created (now e4ec43252db)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.gradle.enterprise-3.16 in repository https://gitbox.apache.org/repos/asf/beam.git at e4ec43252db Bump com.gradle.enterprise from 3.15.1 to 3.16 No new revisions were added by this update.
(beam) branch dependabot/gradle/org.javacc.javacc-3.0.2 created (now 42990e51026)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/org.javacc.javacc-3.0.2 in repository https://gitbox.apache.org/repos/asf/beam.git at 42990e51026 Bump org.javacc.javacc from 3.0.0 to 3.0.2 No new revisions were added by this update.
(beam) branch dependabot/gradle/com.diffplug.spotless-spotless-plugin-gradle-6.23.3 created (now 5ab13bdb2b4)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.diffplug.spotless-spotless-plugin-gradle-6.23.3 in repository https://gitbox.apache.org/repos/asf/beam.git at 5ab13bdb2b4 Bump com.diffplug.spotless:spotless-plugin-gradle from 5.6.1 to 6.23.3 No new revisions were added by this update.
(beam) branch criteo_mltransform deleted (was 12df8cf9a5f)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch criteo_mltransform in repository https://gitbox.apache.org/repos/asf/beam.git was 12df8cf9a5f Skip TFT test. revert commit once done The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch ml_transform_criteo created (now 26bd7e70e5b)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch ml_transform_criteo in repository https://gitbox.apache.org/repos/asf/beam.git at 26bd7e70e5b Add criteo benchmark for MLTransform No new revisions were added by this update.
(beam) branch master updated (276aa023903 -> 8be85d29f53)
This is an automated email from the ASF dual-hosted git repository. damondouglas pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 276aa023903 Add suffix for vocab files in tft transforms (#29720) add 8be85d29f53 fix: optimize segment reader (#29694) No new revisions were added by this update. Summary of changes: .../sdk/io/gcp/bigtable/BigtableServiceImpl.java | 9 +++- .../io/gcp/bigtable/BigtableServiceImplTest.java | 63 ++ 2 files changed, 70 insertions(+), 2 deletions(-)
(beam) branch dependabot/go_modules/sdks/google.golang.org/api-0.154.0 created (now a8f85a013ad)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/google.golang.org/api-0.154.0 in repository https://gitbox.apache.org/repos/asf/beam.git at a8f85a013ad Bump google.golang.org/api from 0.153.0 to 0.154.0 in /sdks No new revisions were added by this update.
(beam) branch dependabot/go_modules/sdks/github.com/google/uuid-1.5.0 created (now d6642b0b853)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/google/uuid-1.5.0 in repository https://gitbox.apache.org/repos/asf/beam.git at d6642b0b853 Bump github.com/google/uuid from 1.4.0 to 1.5.0 in /sdks No new revisions were added by this update.
(beam) branch nightly-refs/heads/master updated (bdaec7a13ed -> 276aa023903)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from bdaec7a13ed Merge pull request #29695 Enable keys values multimap protocol based on runner capabilities. add eff6c203991 Bump actions/setup-java from 3 to 4 (#29573) add 4431479fd90 Add InfluxDB credentials as env variables where they are missing add 8ecfaa64fa8 Merge pull request #29716:Add InfluxDB credentials as env variables add f9479c68190 Support BigQuery tables with spaces (#29649) add d24c40a125c Bump com.gradle.common-custom-user-data-gradle-plugin (#29632) add 5e00feea6c6 Update links to point to About Beam ML (#29709) add 12a4f677c69 Fixes dictionary size change in direct runners (#29122) add 649da779acd Update to most recent dataflow python container (#29717) add f4d4a0d4696 Add utility method to parse labels/base name from a mangled bigquery metric name (#29500) add 0e85810d0d6 Add HistogramData::getAndReset method (#29484) add 475a0c78690 Fix syntax (#29719) add efe7e6aad1f Deal with trailing slash in tempRoot (#29478) add d59e1920d23 Bump dataflowLegacyContainerVersion (#29723) add 90e79ae373a [Go SDK]: Implement natsio.Read transform for reading from NATS (#29410) add 276aa023903 Add suffix for vocab files in tft transforms (#29720) No new revisions were added by this update. Summary of changes: .github/workflows/beam_Java_JMH.yml| 2 + .../beam_PostCommit_Java_Jpms_Direct_Java21.yml| 2 +- .../beam_PostCommit_Java_Nexmark_Dataflow.yml | 2 + .../beam_PostCommit_Java_Nexmark_Dataflow_V2.yml | 2 + ...am_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml | 2 + .../beam_PostCommit_Java_Nexmark_Direct.yml| 2 + .../beam_PostCommit_Java_Nexmark_Flink.yml | 2 + .../beam_PostCommit_Java_Nexmark_Spark.yml | 2 + .../beam_PostCommit_Java_Tpcds_Dataflow.yml| 4 +- .../workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 + .../workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 + .github/workflows/build_release_candidate.yml | 10 +- .github/workflows/code_completion_plugin_tests.yml | 2 +- .github/workflows/run_rc_validation.yml| 4 +- CHANGES.md | 1 + examples/notebooks/beam-ml/README.md | 6 +- .../beam-ml/automatic_model_refresh.ipynb | 10 - examples/notebooks/beam-ml/mltransform_basic.ipynb | 2 +- examples/notebooks/beam-ml/per_key_models.ipynb| 2 +- .../notebooks/beam-ml/run_custom_inference.ipynb | 2 +- .../run_inference_pytorch_tensorflow_sklearn.ipynb | 4 +- .../beam-ml/speech_emotion_tensorflow.ipynb| 2 +- runners/google-cloud-dataflow-java/build.gradle| 2 +- .../beam/runners/dataflow/TestDataflowRunner.java | 14 +- sdks/go/pkg/beam/io/natsio/common.go | 4 + sdks/go/pkg/beam/io/natsio/end_estimator.go| 77 ++ sdks/go/pkg/beam/io/natsio/end_estimator_test.go | 78 ++ sdks/go/pkg/beam/io/natsio/example_test.go | 18 ++ sdks/go/pkg/beam/io/natsio/helper_test.go | 48 +++- sdks/go/pkg/beam/io/natsio/read.go | 289 + sdks/go/pkg/beam/io/natsio/read_option.go | 98 +++ sdks/go/pkg/beam/io/natsio/read_test.go| 212 +++ .../go/pkg/beam/io/natsio/time_policy.go | 39 ++- .../strategy.go => io/natsio/time_policy_test.go} | 41 ++- .../go/pkg/beam/io/natsio/watermark_estimator.go | 24 +- .../pkg/beam/io/natsio/watermark_estimator_test.go | 67 + sdks/go/pkg/beam/io/natsio/write_test.go | 4 +- .../java/org/apache/beam/sdk/io/FileSystems.java | 17 ++ .../org/apache/beam/sdk/util/HistogramData.java| 12 + .../org/apache/beam/sdk/io/FileSystemsTest.java| 20 ++ .../apache/beam/sdk/util/HistogramDataTest.java| 26 ++ .../beam/sdk/extensions/gcp/util/GcsUtilIT.java| 4 +- .../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 2 +- .../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 61 - .../sdk/io/gcp/bigquery/BigQueryClusteringIT.java | 4 +- .../beam/sdk/io/gcp/bigquery/BigQueryIOJsonIT.java | 4 +- .../beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java | 4 +- .../bigquery/BigQueryIOStorageReadTableRowIT.java | 4 +- .../beam/sdk/io/gcp/bigquery/BigQueryKmsKeyIT.java | 4 +- .../io/gcp/bigquery/BigQueryNestedRecordsIT.java | 4 +- .../bigquery/BigQuerySchemaUpdateOptionsIT.java| 7 +- .../io/gcp/bigquery/BigQuerySinkMetricsTest.java | 44 .../BigQueryTimePartitioningClusteringIT.java | 4 +- .../sdk/io/gcp/bigquery/BigQueryToTableIT.java | 10 +- .../beam/sdk/io/gcp/storage/GcsKmsKeyIT.java | 3 +- .../apache/beam/sdk/io/gcp/storage/GcsMatchIT.java | 3 +-
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 664b4d11540 Updating config from bot 664b4d11540 is described below commit 664b4d11540cb94b78bafa28b5b4241b7a4fc8e1 Author: github-actions AuthorDate: Wed Dec 13 01:29:47 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index 5d2b702400c..00a0ff7cea8 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -12,6 +12,6 @@ "riteshghorse": 1702076738591, "robertwb": 1700228072719, "tvalentyn": 1700246106570, -"damondouglas": 1700069673019 +"damondouglas": 1702430984870 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9f6b1e5a4a2 Updating config from bot
9f6b1e5a4a2 is described below
commit 9f6b1e5a4a2d52d8c6c91fc8947ea57da8cc569e
Author: github-actions
AuthorDate: Wed Dec 13 01:29:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29732.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29732.json
b/scripts/ci/pr-bot/state/pr-state/pr-29732.json
new file mode 100644
index 000..2040c933220
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29732.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "damondouglas"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated (90e79ae373a -> 276aa023903)
This is an automated email from the ASF dual-hosted git repository. anandinguva pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 90e79ae373a [Go SDK]: Implement natsio.Read transform for reading from NATS (#29410) add 276aa023903 Add suffix for vocab files in tft transforms (#29720) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/ml/transforms/tft.py | 28 +--- sdks/python/apache_beam/ml/transforms/tft_test.py | 81 ++- 2 files changed, 98 insertions(+), 11 deletions(-)
(beam) branch asf-site updated: Publishing website 2023/12/12 23:39:09 at commit 90e79ae
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/asf-site by this push: new 1851bc07b91 Publishing website 2023/12/12 23:39:09 at commit 90e79ae 1851bc07b91 is described below commit 1851bc07b91e667efbe44239d7b5f53cc622e4fc Author: runner AuthorDate: Tue Dec 12 23:39:09 2023 + Publishing website 2023/12/12 23:39:09 at commit 90e79ae --- website/generated-content/sitemap.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 2f159894ca0..a4dbbb7e351 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.52.0/2023-12-12T08:59:39-08:00/categories/blog/2023-12-12T08:59:39-08:00/blog/2023-12-12T08:59:39-08:00/categories/2023-12-12T08:59:39-08:00/catego [...] \ No newline at end of file +http://www.sitemaps.org/schemas/sitemap/0.9; xmlns:xhtml="http://www.w3.org/1999/xhtml;>/blog/beam-2.52.0/2023-12-12T22:19:06+01:00/categories/blog/2023-12-12T22:19:06+01:00/blog/2023-12-12T22:19:06+01:00/categories/2023-12-12T22:19:06+01:00/catego [...] \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1094757879c Updating config from bot
1094757879c is described below
commit 1094757879c0a72330addcc71440ad56214a5916
Author: github-actions
AuthorDate: Tue Dec 12 23:25:06 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29731.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29731.json
b/scripts/ci/pr-bot/state/pr-state/pr-29731.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29731.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new feef461bc15 Updating config from bot feef461bc15 is described below commit feef461bc159916d7d3b7036d83282402f404a8f Author: github-actions AuthorDate: Tue Dec 12 23:12:56 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29712.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29712.json b/scripts/ci/pr-bot/state/pr-state/pr-29712.json index aa3638a74ec..553b9b3a020 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29712.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29712.json @@ -4,7 +4,7 @@ "go": "riteshghorse" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f7da080fb6c Updating config from bot
f7da080fb6c is described below
commit f7da080fb6c1d712a1cd01d8075aacd57a6c1b73
Author: github-actions
AuthorDate: Tue Dec 12 22:16:42 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29561.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29561.json
b/scripts/ci/pr-bot/state/pr-state/pr-29561.json
index 242a48d7d3b..37f7ab41238 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29561.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29561.json
@@ -2,7 +2,7 @@
"commentedAboutFailingChecks": true,
"reviewersAssignedForLabels": {},
"nextAction": "Author",
- "stopReviewerNotifications": false,
+ "stopReviewerNotifications": true,
"remindAfterTestsPass": [],
"committerAssigned": false
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new d19ccdee940 Updating config from bot
d19ccdee940 is described below
commit d19ccdee940ae98e88dd943feb23646bd6231aa8
Author: github-actions
AuthorDate: Tue Dec 12 22:08:12 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29720.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29720.json
b/scripts/ci/pr-bot/state/pr-state/pr-29720.json
new file mode 100644
index 000..321b31c0f05
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29720.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 334029e6db3 Updating config from bot 334029e6db3 is described below commit 334029e6db30c78b686629b1baeb9de1721181dc Author: github-actions AuthorDate: Tue Dec 12 22:08:14 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 6262bc99c32..a50ac13dc31 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -5,7 +5,7 @@ "yeandy": 1665802753763, "TheNeuralBit": 1667896849319, "ryanthompson591": 1670002443548, -"tvalentyn": 1701994812757, +"tvalentyn": 1702418891117, "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1702071278157,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a45d517d06f Updating config from bot
a45d517d06f is described below
commit a45d517d06f0f1d28117059b05e586c5e9719b4d
Author: github-actions
AuthorDate: Tue Dec 12 22:05:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29727.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29727.json
b/scripts/ci/pr-bot/state/pr-state/pr-29727.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29727.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a6d9b997a9e Updating config from bot
a6d9b997a9e is described below
commit a6d9b997a9eb4824780d70925d9af6466bebda8c
Author: github-actions
AuthorDate: Tue Dec 12 22:01:19 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29728.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29728.json
b/scripts/ci/pr-bot/state/pr-state/pr-29728.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29728.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 25b83a1cb9b Updating config from bot
25b83a1cb9b is described below
commit 25b83a1cb9b4c83e5d5abd2f076a6985d3d5087e
Author: github-actions
AuthorDate: Tue Dec 12 21:34:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29712.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29712.json
b/scripts/ci/pr-bot/state/pr-state/pr-29712.json
index 242a48d7d3b..aa3638a74ec 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29712.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29712.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new fe45f248205 Updating config from bot fe45f248205 is described below commit fe45f2482056b67ce0ea688aab7b5926fcbc83b1 Author: github-actions AuthorDate: Tue Dec 12 21:34:51 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29694.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29694.json b/scripts/ci/pr-bot/state/pr-state/pr-29694.json index 4094f26494f..3ea6f7c8bb8 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29694.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29694.json @@ -8,5 +8,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new fb8285f6f10 Updating config from bot fb8285f6f10 is described below commit fb8285f6f104269ee99afd73aea5279956d61106 Author: github-actions AuthorDate: Tue Dec 12 21:34:48 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 8783707148a..1fd77b57bf4 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1702404385813, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1702272869219 +"riteshghorse": 1702416884948 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 973b2715a32 Updating config from bot
973b2715a32 is described below
commit 973b2715a32959016e85dd23ee7aec91591640c8
Author: github-actions
AuthorDate: Tue Dec 12 21:34:41 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29726.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29726.json
b/scripts/ci/pr-bot/state/pr-state/pr-29726.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29726.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated: [Go SDK]: Implement natsio.Read transform for reading from NATS (#29410)
This is an automated email from the ASF dual-hosted git repository.
lostluck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 90e79ae373a [Go SDK]: Implement natsio.Read transform for reading from
NATS (#29410)
90e79ae373a is described below
commit 90e79ae373ab38cf4e48e9854c28aaffb0938458
Author: Johanna Öjeling <51084516+johannaojel...@users.noreply.github.com>
AuthorDate: Tue Dec 12 22:19:06 2023 +0100
[Go SDK]: Implement natsio.Read transform for reading from NATS (#29410)
---
CHANGES.md | 1 +
sdks/go/pkg/beam/io/natsio/common.go | 4 +
sdks/go/pkg/beam/io/natsio/end_estimator.go| 77 ++
sdks/go/pkg/beam/io/natsio/end_estimator_test.go | 78 ++
sdks/go/pkg/beam/io/natsio/example_test.go | 18 ++
sdks/go/pkg/beam/io/natsio/helper_test.go | 48 +++-
sdks/go/pkg/beam/io/natsio/read.go | 289 +
sdks/go/pkg/beam/io/natsio/read_option.go | 98 +++
sdks/go/pkg/beam/io/natsio/read_test.go| 212 +++
.../beam/io/natsio/{common.go => time_policy.go} | 50 ++--
.../io/natsio/{common.go => time_policy_test.go} | 55 ++--
.../natsio/{common.go => watermark_estimator.go} | 43 +--
.../pkg/beam/io/natsio/watermark_estimator_test.go | 67 +
sdks/go/pkg/beam/io/natsio/write_test.go | 4 +-
14 files changed, 943 insertions(+), 101 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index 60b5a820cf3..70ce1a70b7e 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -68,6 +68,7 @@
* Adding support for LowCardinality DataType in ClickHouse (Java)
([#29533](https://github.com/apache/beam/pull/29533)).
* Added support for handling bad records to KafkaIO (Java)
([#29546](https://github.com/apache/beam/pull/29546))
* Add support for generating text embeddings in MLTransform for Vertex AI and
Hugging Face Hub models.([#29564](https://github.com/apache/beam/pull/29564))
+* NATS IO connector added (Go)
([#29000](https://github.com/apache/beam/issues/29000)).
## New Features / Improvements
diff --git a/sdks/go/pkg/beam/io/natsio/common.go
b/sdks/go/pkg/beam/io/natsio/common.go
index 53f59551698..72640894c76 100644
--- a/sdks/go/pkg/beam/io/natsio/common.go
+++ b/sdks/go/pkg/beam/io/natsio/common.go
@@ -31,6 +31,10 @@ type natsFn struct {
}
func (fn *natsFn) Setup() error {
+ if fn.nc != nil && fn.js != nil {
+ return nil
+ }
+
var opts []nats.Option
if fn.CredsFile != "" {
opts = append(opts, nats.UserCredentials(fn.CredsFile))
diff --git a/sdks/go/pkg/beam/io/natsio/end_estimator.go
b/sdks/go/pkg/beam/io/natsio/end_estimator.go
new file mode 100644
index 000..6b2f18e10ce
--- /dev/null
+++ b/sdks/go/pkg/beam/io/natsio/end_estimator.go
@@ -0,0 +1,77 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+//http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package natsio
+
+import (
+ "context"
+ "errors"
+ "fmt"
+
+ "github.com/nats-io/nats.go/jetstream"
+)
+
+type endEstimator struct {
+ js jetstream.JetStream
+ stream string
+ subject string
+}
+
+func newEndEstimator(js jetstream.JetStream, stream string, subject string)
*endEstimator {
+ return {
+ js: js,
+ stream: stream,
+ subject: subject,
+ }
+}
+
+func (e *endEstimator) Estimate() int64 {
+ ctx := context.Background()
+ end, err := e.getEndSeqNo(ctx)
+ if err != nil {
+ panic(err)
+ }
+ return end
+}
+
+func (e *endEstimator) getEndSeqNo(ctx context.Context) (int64, error) {
+ str, err := e.js.Stream(ctx, e.stream)
+ if err != nil {
+ return -1, fmt.Errorf("error getting stream: %v", err)
+ }
+
+ msg, err := str.GetLastMsgForSubject(ctx, e.subject)
+ if err != nil {
+ if isMessageNotFound(err) {
+ return 1, nil
+ }
+
+ return -1, fmt.Errorf("error getting last message: %v", err)
+ }
+
+ return int64(msg.Sequence) +
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 73003a927be Updating config from bot
73003a927be is described below
commit 73003a927beff5cba47c90e481dc5f49fbea1c3a
Author: github-actions
AuthorDate: Tue Dec 12 20:34:39 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29725.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29725.json
b/scripts/ci/pr-bot/state/pr-state/pr-29725.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29725.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated: Bump dataflowLegacyContainerVersion (#29723)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new d59e1920d23 Bump dataflowLegacyContainerVersion (#29723)
d59e1920d23 is described below
commit d59e1920d23546332368847597255051611e5bd5
Author: Yi Hu
AuthorDate: Tue Dec 12 14:52:17 2023 -0500
Bump dataflowLegacyContainerVersion (#29723)
---
runners/google-cloud-dataflow-java/build.gradle | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/runners/google-cloud-dataflow-java/build.gradle
b/runners/google-cloud-dataflow-java/build.gradle
index 5d967d90f2b..c7e5e1ba623 100644
--- a/runners/google-cloud-dataflow-java/build.gradle
+++ b/runners/google-cloud-dataflow-java/build.gradle
@@ -51,7 +51,7 @@ evaluationDependsOn(":sdks:java:container:java11")
ext.dataflowLegacyEnvironmentMajorVersion = '8'
ext.dataflowFnapiEnvironmentMajorVersion = '8'
-ext.dataflowLegacyContainerVersion = 'beam-master-20231026'
+ext.dataflowLegacyContainerVersion = 'beam-master-20231212'
ext.dataflowFnapiContainerVersion = 'beam-master-20231026'
ext.dataflowContainerBaseRepository = 'gcr.io/cloud-dataflow/v1beta3'
(beam) branch master updated (475a0c78690 -> efe7e6aad1f)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 475a0c78690 Fix syntax (#29719) add efe7e6aad1f Deal with trailing slash in tempRoot (#29478) No new revisions were added by this update. Summary of changes: .../beam/runners/dataflow/TestDataflowRunner.java| 14 +++--- .../java/org/apache/beam/sdk/io/FileSystems.java | 17 + .../java/org/apache/beam/sdk/io/FileSystemsTest.java | 20 .../beam/sdk/extensions/gcp/util/GcsUtilIT.java | 4 +++- .../sdk/io/gcp/bigquery/BigQueryClusteringIT.java| 4 +++- .../beam/sdk/io/gcp/bigquery/BigQueryIOJsonIT.java | 4 +++- .../beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java | 4 +++- .../bigquery/BigQueryIOStorageReadTableRowIT.java| 4 +++- .../beam/sdk/io/gcp/bigquery/BigQueryKmsKeyIT.java | 4 +++- .../sdk/io/gcp/bigquery/BigQueryNestedRecordsIT.java | 4 +++- .../gcp/bigquery/BigQuerySchemaUpdateOptionsIT.java | 7 +-- .../BigQueryTimePartitioningClusteringIT.java| 4 +++- .../beam/sdk/io/gcp/bigquery/BigQueryToTableIT.java | 10 +++--- .../apache/beam/sdk/io/gcp/storage/GcsKmsKeyIT.java | 3 ++- .../apache/beam/sdk/io/gcp/storage/GcsMatchIT.java | 3 ++- 15 files changed, 88 insertions(+), 18 deletions(-)
(beam) branch master updated: Fix syntax (#29719)
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 475a0c78690 Fix syntax (#29719)
475a0c78690 is described below
commit 475a0c7869051331a03f2bcb65af45113fd984a3
Author: Ritesh Ghorse
AuthorDate: Tue Dec 12 14:08:45 2023 -0500
Fix syntax (#29719)
* correct syntax
* remove unused inference_args
---
.../apache_beam/ml/inference/huggingface_inference.py | 19 +--
1 file changed, 1 insertion(+), 18 deletions(-)
diff --git a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
index 1bc92c462c9..25367d22eaa 100644
--- a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
+++ b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
@@ -221,7 +221,6 @@ class
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
*,
inference_fn: Optional[Callable[..., Iterable[PredictionResult]]] = None,
load_model_args: Optional[Dict[str, Any]] = None,
- inference_args: Optional[Dict[str, Any]] = None,
min_batch_size: Optional[int] = None,
max_batch_size: Optional[int] = None,
max_batch_duration_secs: Optional[int] = None,
@@ -250,10 +249,6 @@ class
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
load_model_args (Dict[str, Any]): (Optional) Keyword arguments to provide
load options while loading models from Hugging Face Hub.
Defaults to None.
- inference_args (Dict[str, Any]): (Optional) Non-batchable arguments
-required as inputs to the model's inference function. Unlike Tensors
-in `batch`, these parameters will not be dynamically batched.
-Defaults to None.
min_batch_size: the minimum batch size to use when batching inputs.
max_batch_size: the maximum batch size to use when batching inputs.
max_batch_duration_secs: the maximum amount of time to buffer a batch
@@ -273,7 +268,6 @@ class
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
self._device = device
self._inference_fn = inference_fn
self._model_config_args = load_model_args if load_model_args else {}
-self._inference_args = inference_args if inference_args else {}
self._batching_kwargs = {}
self._env_vars = kwargs.get("env_vars", {})
if min_batch_size is not None:
@@ -293,7 +287,7 @@ class
HuggingFaceModelHandlerKeyedTensor(ModelHandler[Dict[str,
model = self._model_class.from_pretrained(
self._model_uri, **self._model_config_args)
if self._framework == 'pt':
- if self._device == "GPU" and is_gpu_available_torch:
+ if self._device == "GPU" and is_gpu_available_torch():
model.to(torch.device("cuda"))
if callable(getattr(model, 'requires_grad_', None)):
model.requires_grad_(False)
@@ -407,7 +401,6 @@ class
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
*,
inference_fn: Optional[Callable[..., Iterable[PredictionResult]]] = None,
load_model_args: Optional[Dict[str, Any]] = None,
- inference_args: Optional[Dict[str, Any]] = None,
min_batch_size: Optional[int] = None,
max_batch_size: Optional[int] = None,
max_batch_duration_secs: Optional[int] = None,
@@ -436,10 +429,6 @@ class
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
load_model_args (Dict[str, Any]): (Optional) keyword arguments to provide
load options while loading models from Hugging Face Hub.
Defaults to None.
- inference_args (Dict[str, Any]): (Optional) Non-batchable arguments
-required as inputs to the model's inference function. Unlike Tensors
-in `batch`, these parameters will not be dynamically batched.
-Defaults to None.
min_batch_size: the minimum batch size to use when batching inputs.
max_batch_size: the maximum batch size to use when batching inputs.
max_batch_duration_secs: the maximum amount of time to buffer a batch
@@ -459,7 +448,6 @@ class
HuggingFaceModelHandlerTensor(ModelHandler[Union[tf.Tensor, torch.Tensor],
self._device = device
self._inference_fn = inference_fn
self._model_config_args = load_model_args if load_model_args else {}
-self._inference_args = inference_args if inference_args else {}
self._batching_kwargs = {}
self._env_vars = kwargs.get("env_vars", {})
if min_batch_size is not None:
@@ -586,7 +574,6 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
device: Optional[str] = None,
inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
load_pipeline_args: Optional[Dict[str, Any]] = None,
- inference_args: Optional[Dict[str, Any]] = None,
min_batch_size:
(beam) branch master updated (f4d4a0d4696 -> 0e85810d0d6)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from f4d4a0d4696 Add utility method to parse labels/base name from a mangled bigquery metric name (#29500) add 0e85810d0d6 Add HistogramData::getAndReset method (#29484) No new revisions were added by this update. Summary of changes: .../org/apache/beam/sdk/util/HistogramData.java| 12 ++ .../apache/beam/sdk/util/HistogramDataTest.java| 26 ++ 2 files changed, 38 insertions(+)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new c90705330cd Updating config from bot
c90705330cd is described below
commit c90705330cd101d12c31e9e91c7082b2db4458c5
Author: github-actions
AuthorDate: Tue Dec 12 19:05:48 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29723.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29723.json
b/scripts/ci/pr-bot/state/pr-state/pr-29723.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29723.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 5589671f69e Updating config from bot 5589671f69e is described below commit 5589671f69ea6de97b9982fef0d5503636e0dc42 Author: github-actions AuthorDate: Tue Dec 12 19:05:32 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29721.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29721.json b/scripts/ci/pr-bot/state/pr-state/pr-29721.json index 1fd18b7f4c0..7180e22cec9 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29721.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29721.json @@ -9,5 +9,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 300c73a08d7 Updating config from bot
300c73a08d7 is described below
commit 300c73a08d710d88676fec6186eb35471732a516
Author: github-actions
AuthorDate: Tue Dec 12 18:34:47 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29710.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29710.json
b/scripts/ci/pr-bot/state/pr-state/pr-29710.json
new file mode 100644
index 000..4b972b47d4c
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29710.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "bvolpato",
+"io": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 64c8e508943 Updating config from bot 64c8e508943 is described below commit 64c8e508943e201470ccf279b634678ebca79f54 Author: github-actions AuthorDate: Tue Dec 12 18:35:19 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index e64821d43ab..4a957a953ac 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -4,7 +4,7 @@ "chamikaramj": 1702373679889, "johnjcasey": 1701967055102, "pabloem": 1691787951165, -"Abacn": 1701761214632, +"Abacn": 1702406086322, "ahmedabu98": 1701912589475, "bvolpato": 1701900361224, "manavgarg": 1690826779210,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new b9e4e82abd3 Updating config from bot b9e4e82abd3 is described below commit b9e4e82abd32b5521d792c3df591afd2bf740afe Author: github-actions AuthorDate: Tue Dec 12 18:34:34 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29719.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29719.json b/scripts/ci/pr-bot/state/pr-state/pr-29719.json index 6272b4e797d..7220f359527 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29719.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29719.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new d8cad9c410e Updating config from bot d8cad9c410e is described below commit d8cad9c410ea62733a3f0c4f4a6dd1dc6fd0c59c Author: github-actions AuthorDate: Tue Dec 12 18:35:10 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json b/scripts/ci/pr-bot/state/reviewers-for-label-java.json index 1cd26f3d827..42f1995154a 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json @@ -7,7 +7,7 @@ "kileys": 1674428678843, "apilloud": 1678822446183, "Abacn": 1702037652272, -"bvolpato": 1701937217789, +"bvolpato": 1702406086322, "m-trieu": 1702244239472, "damondouglas": 1702076742198 }
(beam) branch users/damccorm/goVersion deleted (was 9cbb0b07fab)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/goVersion in repository https://gitbox.apache.org/repos/asf/beam.git was 9cbb0b07fab Allow go version override This change permanently discards the following revisions: discard 9cbb0b07fab Allow go version override
(beam) branch users/damccorm/gcsDep deleted (was 27def9967a8)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/gcsDep in repository https://gitbox.apache.org/repos/asf/beam.git was 27def9967a8 More recent client This change permanently discards the following revisions: discard 27def9967a8 More recent client discard 5216c1779dd Add gcs dep
(beam) branch users/damccorm/auth deleted (was 80a4e078b56)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/auth in repository https://gitbox.apache.org/repos/asf/beam.git was 80a4e078b56 Fix auth for beam_CleanUpPrebuiltSDKImages.yml This change permanently discards the following revisions: discard 80a4e078b56 Fix auth for beam_CleanUpPrebuiltSDKImages.yml
(beam) branch users/damccorm/dep-cp deleted (was 4dac8b1e056)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/dep-cp in repository https://gitbox.apache.org/repos/asf/beam.git was 4dac8b1e056 Declare signMavenJavaPublication dependency on copyPom The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch users/damccorm/partialRevert deleted (was 747e7fa4071)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/partialRevert in repository https://gitbox.apache.org/repos/asf/beam.git was 747e7fa4071 Partially revert changes from #29587 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 8bb7ee604f6 Updating config from bot
8bb7ee604f6 is described below
commit 8bb7ee604f67cb47f8cec53806cceb772077031a
Author: github-actions
AuthorDate: Tue Dec 12 18:18:28 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29708.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29708.json
b/scripts/ci/pr-bot/state/pr-state/pr-29708.json
index 242a48d7d3b..37f7ab41238 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29708.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29708.json
@@ -2,7 +2,7 @@
"commentedAboutFailingChecks": true,
"reviewersAssignedForLabels": {},
"nextAction": "Author",
- "stopReviewerNotifications": false,
+ "stopReviewerNotifications": true,
"remindAfterTestsPass": [],
"committerAssigned": false
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f51c129fc5b Updating config from bot
f51c129fc5b is described below
commit f51c129fc5bfd9b4489f6db64275ce37e33f4974
Author: github-actions
AuthorDate: Tue Dec 12 18:06:35 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29719.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29719.json
b/scripts/ci/pr-bot/state/pr-state/pr-29719.json
new file mode 100644
index 000..6272b4e797d
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29719.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "AnandInguva"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 71acd397398 Updating config from bot 71acd397398 is described below commit 71acd397398d2929547bf05695263643c648ec2d Author: github-actions AuthorDate: Tue Dec 12 18:06:28 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 215813e039f..8ce6e650386 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -9,7 +9,7 @@ "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1702071278157, -"jrmccluskey": 1701900361224, +"jrmccluskey": 1702404385813, "riteshghorse": 1702312674607, "liferoad": 1702399009803 }
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 4da61b045c6 Updating config from bot 4da61b045c6 is described below commit 4da61b045c602613785f3d69e414c35f1812a7fc Author: github-actions AuthorDate: Tue Dec 12 18:06:32 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-website.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-website.json b/scripts/ci/pr-bot/state/reviewers-for-label-website.json index 592818ec281..7508c01bb39 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-website.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-website.json @@ -4,6 +4,7 @@ "rszper": 1700231683152, "melap": 1701201946424, "damccorm": 1701390112035, -"liferoad": 1701394336823 +"liferoad": 1701394336823, +"kennknowles": 1702404385813 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a2ec787d323 Updating config from bot
a2ec787d323 is described below
commit a2ec787d3230ce0c855245c858b9f4de15fe37b2
Author: github-actions
AuthorDate: Tue Dec 12 18:06:27 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29721.json | 13 +
1 file changed, 13 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29721.json
b/scripts/ci/pr-bot/state/pr-state/pr-29721.json
new file mode 100644
index 000..1fd18b7f4c0
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29721.json
@@ -0,0 +1,13 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "jrmccluskey",
+"go": "jrmccluskey",
+"build": "Abacn",
+"website": "kennknowles"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1f6c016e2f8 Updating config from bot
1f6c016e2f8 is described below
commit 1f6c016e2f852e46aebb521a1f21d116a9f06235
Author: github-actions
AuthorDate: Tue Dec 12 18:06:30 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index bbdbdf5d838..8783707148a 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1702272877414,
-"jrmccluskey": 1702271156500,
+"jrmccluskey": 1702404385813,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1702272869219
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1344be64eca Updating config from bot
1344be64eca is described below
commit 1344be64eca64a2e702d25a03bb7c7abed4413bb
Author: github-actions
AuthorDate: Tue Dec 12 18:06:31 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index c8c0ba0f1a5..92058c3e4ee 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1702383230637,
-"Abacn": 1702037652612
+"Abacn": 1702404385813
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 321d50913eb Updating config from bot
321d50913eb is described below
commit 321d50913eb4cac16e2c21bab841b0ffcf3c9359
Author: github-actions
AuthorDate: Tue Dec 12 18:06:36 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
index 8ce6e650386..6262bc99c32 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json
@@ -1,7 +1,7 @@
{
"label": "python",
"dateOfLastReviewAssignment": {
-"AnandInguva": 1701916695379,
+"AnandInguva": 1702404394215,
"yeandy": 1665802753763,
"TheNeuralBit": 1667896849319,
"ryanthompson591": 1670002443548,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9cf8255d1b5 Updating config from bot
9cf8255d1b5 is described below
commit 9cf8255d1b5bed78e42d7d992c7f8fa177a91944
Author: github-actions
AuthorDate: Tue Dec 12 17:01:50 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29217.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29217.json
b/scripts/ci/pr-bot/state/pr-state/pr-29217.json
index 84a14da827e..88d1920a8bb 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29217.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29217.json
@@ -3,7 +3,7 @@
"reviewersAssignedForLabels": {
"python": "damccorm"
},
- "nextAction": "Author",
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
(beam) branch master updated (649da779acd -> f4d4a0d4696)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 649da779acd Update to most recent dataflow python container (#29717) add f4d4a0d4696 Add utility method to parse labels/base name from a mangled bigquery metric name (#29500) No new revisions were added by this update. Summary of changes: .../sdk/io/gcp/bigquery/BigQuerySinkMetrics.java | 61 +- .../io/gcp/bigquery/BigQuerySinkMetricsTest.java | 44 2 files changed, 104 insertions(+), 1 deletion(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 206298d9c83 Updating config from bot 206298d9c83 is described below commit 206298d9c833a616597871ac93e68fa03cf9dd5b Author: github-actions AuthorDate: Tue Dec 12 16:36:52 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index 941f9ade676..215813e039f 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -11,6 +11,6 @@ "damccorm": 1702071278157, "jrmccluskey": 1701900361224, "riteshghorse": 1702312674607, -"liferoad": 1701893148828 +"liferoad": 1702399009803 } } \ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1334ff26605 Updating config from bot
1334ff26605 is described below
commit 1334ff266053664062a5b6372eac002dd8ca7dd3
Author: github-actions
AuthorDate: Tue Dec 12 16:36:51 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29718.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29718.json
b/scripts/ci/pr-bot/state/pr-state/pr-29718.json
new file mode 100644
index 000..e1b3308f259
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29718.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "liferoad"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch users/damccorm/pyCont deleted (was fa0a408f515)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/pyCont in repository https://gitbox.apache.org/repos/asf/beam.git was fa0a408f515 Update to most recent dataflow python container The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Update to most recent dataflow python container (#29717)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 649da779acd Update to most recent dataflow python container (#29717) 649da779acd is described below commit 649da779acd9f0a7e443be6edf2f963f3b362b91 Author: Danny McCormick AuthorDate: Tue Dec 12 10:54:04 2023 -0500 Update to most recent dataflow python container (#29717) --- sdks/python/apache_beam/runners/dataflow/internal/names.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/python/apache_beam/runners/dataflow/internal/names.py b/sdks/python/apache_beam/runners/dataflow/internal/names.py index 82e2306ced3..b43f615189d 100644 --- a/sdks/python/apache_beam/runners/dataflow/internal/names.py +++ b/sdks/python/apache_beam/runners/dataflow/internal/names.py @@ -34,6 +34,6 @@ SERIALIZED_SOURCE_KEY = 'serialized_source' # Unreleased sdks use container image tag specified below. # Update this tag whenever there is a change that # requires changes to SDK harness container or SDK harness launcher. -BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231207' +BEAM_DEV_SDK_CONTAINER_TAG = 'beam-master-20231212' DATAFLOW_CONTAINER_IMAGE_REPOSITORY = 'gcr.io/cloud-dataflow/v1beta3'
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 46cfb28f59c Updating config from bot
46cfb28f59c is described below
commit 46cfb28f59c97920c002a0aa2088767160b61028
Author: github-actions
AuthorDate: Tue Dec 12 15:24:08 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29717.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29717.json
b/scripts/ci/pr-bot/state/pr-state/pr-29717.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29717.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch master updated (5e00feea6c6 -> 12a4f677c69)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 5e00feea6c6 Update links to point to About Beam ML (#29709) add 12a4f677c69 Fixes dictionary size change in direct runners (#29122) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/worker/sdk_worker.py | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-)
(beam) branch users/damccorm/pyCont created (now fa0a408f515)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/pyCont in repository https://gitbox.apache.org/repos/asf/beam.git at fa0a408f515 Update to most recent dataflow python container No new revisions were added by this update.
(beam) branch master updated: Update links to point to About Beam ML (#29709)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 5e00feea6c6 Update links to point to About Beam ML (#29709)
5e00feea6c6 is described below
commit 5e00feea6c6b355092a9746a0af0d0989a42f371
Author: Rebecca Szper <98840847+rsz...@users.noreply.github.com>
AuthorDate: Tue Dec 12 07:15:34 2023 -0800
Update links to point to About Beam ML (#29709)
* Update Beam ML links
* Update Beam ML links
---
examples/notebooks/beam-ml/README.md | 6 +++---
examples/notebooks/beam-ml/automatic_model_refresh.ipynb | 10 --
examples/notebooks/beam-ml/mltransform_basic.ipynb | 2 +-
examples/notebooks/beam-ml/per_key_models.ipynb| 2 +-
examples/notebooks/beam-ml/run_custom_inference.ipynb | 2 +-
.../beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb | 4 ++--
examples/notebooks/beam-ml/speech_emotion_tensorflow.ipynb | 2 +-
sdks/python/apache_beam/examples/inference/README.md | 2 +-
sdks/python/apache_beam/ml/inference/__init__.py | 2 +-
sdks/python/apache_beam/ml/inference/base.py | 14 +++---
website/www/site/content/en/documentation/ml/about-ml.md | 2 +-
.../site/content/en/documentation/ml/inference-overview.md | 4 ++--
.../content/en/documentation/ml/multi-model-pipelines.md | 2 +-
website/www/site/content/en/documentation/ml/overview.md | 4 ++--
website/www/site/content/en/documentation/sdks/python.md | 2 +-
.../transforms/python/elementwise/runinference.md | 2 +-
.../content/en/get-started/resources/learning-resources.md | 2 +-
17 files changed, 27 insertions(+), 37 deletions(-)
diff --git a/examples/notebooks/beam-ml/README.md
b/examples/notebooks/beam-ml/README.md
index 0ae937e9e28..5a90f1c68b4 100644
--- a/examples/notebooks/beam-ml/README.md
+++ b/examples/notebooks/beam-ml/README.md
@@ -25,13 +25,13 @@ transform.
This transform allows you to make predictions and inference on data with
machine learning (ML) models.
The model handler abstracts the user from the configuration needed for
specific frameworks, such as Tensorflow, PyTorch, and others. For a full list
of supported frameworks,
-see the Apache Beam [Machine
Learning](https://beam.apache.org/documentation/sdks/python-machine-learning)
page.
+see the [About Beam ML](https://beam.apache.org/documentation/ml/about-ml/)
page.
## Using The Notebooks
These notebooks illustrate ways to use Apache Beam's RunInference transforms,
as well as different
-use cases for
[ModelHandler](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelHandler)
implementations.
-Beam comes with [multiple ModelHandler
implementations](https://beam.apache.org/documentation/sdks/python-machine-learning/#modify-a-pipeline-to-use-an-ml-model).
+use cases for
[`ModelHandler`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelHandler)
implementations.
+Beam comes with [multiple `ModelHandler`
implementations](https://beam.apache.org/documentation/ml/about-ml/#modify-a-python-pipeline-to-use-an-ml-model).
### Loading the Notebooks
diff --git a/examples/notebooks/beam-ml/automatic_model_refresh.ipynb
b/examples/notebooks/beam-ml/automatic_model_refresh.ipynb
index cf05979c5b3..b0e9bc2f53e 100644
--- a/examples/notebooks/beam-ml/automatic_model_refresh.ipynb
+++ b/examples/notebooks/beam-ml/automatic_model_refresh.ipynb
@@ -15,16 +15,6 @@
}
},
"cells": [
-{
- "cell_type": "markdown",
- "metadata": {
-"id": "view-in-github",
-"colab_type": "text"
- },
- "source": [
-"https://colab.research.google.com/github/apache/beam/blob/master/examples/notebooks/beam-ml/automatic_model_refresh.ipynb\;
target=\"_parent\">https://colab.research.google.com/assets/colab-badge.svg\; alt=\"Open In
Colab\"/>"
- ]
-},
{
"cell_type": "code",
"source": [
diff --git a/examples/notebooks/beam-ml/mltransform_basic.ipynb
b/examples/notebooks/beam-ml/mltransform_basic.ipynb
index 43068624eb8..b0af96d0859 100644
--- a/examples/notebooks/beam-ml/mltransform_basic.ipynb
+++ b/examples/notebooks/beam-ml/mltransform_basic.ipynb
@@ -55,7 +55,7 @@
"id": "d3b81cf2-8603-42bd-995e-9e14631effd0"
},
"source": [
-"This notebook demonstrates how to use `MLTransform` to preprocess
data for machine learning workflows. Apache Beam provides a set of transforms
for preprocessing data for training and inference. The `MLTransform` class
wraps various transforms in one PTransform, simplifying your workflow. For a
list of available preprocessing transforms see the [Preprocess data with
(beam) 01/01: Fix auth for beam_CleanUpPrebuiltSDKImages.yml
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/auth
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 80a4e078b56a312a7cdce8ef3c0368dc3d20bbbf
Author: Danny McCormick
AuthorDate: Tue Dec 12 09:59:35 2023 -0500
Fix auth for beam_CleanUpPrebuiltSDKImages.yml
---
.github/workflows/beam_CleanUpPrebuiltSDKImages.yml | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/beam_CleanUpPrebuiltSDKImages.yml
b/.github/workflows/beam_CleanUpPrebuiltSDKImages.yml
index 4cc4b09a84c..a22363c2fd3 100644
--- a/.github/workflows/beam_CleanUpPrebuiltSDKImages.yml
+++ b/.github/workflows/beam_CleanUpPrebuiltSDKImages.yml
@@ -72,12 +72,13 @@ jobs:
- name: Setup environment
uses: ./.github/actions/setup-environment-action
- name: Authenticate on GCP
-id: auth
-uses: google-github-actions/auth@v1
+uses: google-github-actions/setup-gcloud@v0
with:
- credentials_json: ${{ secrets.GCP_SA_KEY }}
+ service_account_email: ${{ secrets.GCP_SA_EMAIL }}
+ service_account_key: ${{ secrets.GCP_SA_KEY }}
project_id: ${{ secrets.GCP_PROJECT_ID }}
+ export_default_credentials: true
- name: run remove stale sdk container images
uses: ./.github/actions/gradle-command-self-hosted-action
with:
- gradle-command: :beam-test-tools:removeStaleSDKContainerImages
\ No newline at end of file
+ gradle-command: :beam-test-tools:removeStaleSDKContainerImages
(beam) branch users/damccorm/auth created (now 80a4e078b56)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/auth in repository https://gitbox.apache.org/repos/asf/beam.git at 80a4e078b56 Fix auth for beam_CleanUpPrebuiltSDKImages.yml This branch includes the following new commits: new 80a4e078b56 Fix auth for beam_CleanUpPrebuiltSDKImages.yml The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) branch dependabot/go_modules/learning/tour-of-beam/backend/google.golang.org/grpc-1.56.3 deleted (was 0addf73b4d8)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/learning/tour-of-beam/backend/google.golang.org/grpc-1.56.3 in repository https://gitbox.apache.org/repos/asf/beam.git was 0addf73b4d8 Bump google.golang.org/grpc in /learning/tour-of-beam/backend The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 790c86fe856 Updating config from bot
790c86fe856 is described below
commit 790c86fe856799f50466a6f9ee12ed904d3aeff6
Author: github-actions
AuthorDate: Tue Dec 12 13:43:17 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29500.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29500.json
b/scripts/ci/pr-bot/state/pr-state/pr-29500.json
index 242a48d7d3b..37f7ab41238 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29500.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29500.json
@@ -2,7 +2,7 @@
"commentedAboutFailingChecks": true,
"reviewersAssignedForLabels": {},
"nextAction": "Author",
- "stopReviewerNotifications": false,
+ "stopReviewerNotifications": true,
"remindAfterTestsPass": [],
"committerAssigned": false
}
\ No newline at end of file
(beam) branch dependabot/gradle/org.checkerframework-checkerframework-gradle-plugin-0.6.36 deleted (was 40b917d176f)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/org.checkerframework-checkerframework-gradle-plugin-0.6.36 in repository https://gitbox.apache.org/repos/asf/beam.git was 40b917d176f Bump org.checkerframework:checkerframework-gradle-plugin The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/gradle/com.github.spotbugs.snom-spotbugs-gradle-plugin-6.0.2 deleted (was 332d7f48050)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.github.spotbugs.snom-spotbugs-gradle-plugin-6.0.2 in repository https://gitbox.apache.org/repos/asf/beam.git was 332d7f48050 Bump com.github.spotbugs.snom:spotbugs-gradle-plugin The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-1.12.1 deleted (was 501e6978eca)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/gradle/com.gradle.common-custom-user-data-gradle-plugin-1.12.1 in repository https://gitbox.apache.org/repos/asf/beam.git was 501e6978eca Bump com.gradle.common-custom-user-data-gradle-plugin The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated (f9479c68190 -> d24c40a125c)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from f9479c68190 Support BigQuery tables with spaces (#29649) add d24c40a125c Bump com.gradle.common-custom-user-data-gradle-plugin (#29632) No new revisions were added by this update. Summary of changes: settings.gradle.kts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(beam) branch master updated: Support BigQuery tables with spaces (#29649)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new f9479c68190 Support BigQuery tables with spaces (#29649)
f9479c68190 is described below
commit f9479c6819072fda6ab226d438dd799b9402ed2a
Author: Julian Braha
AuthorDate: Tue Dec 12 08:38:37 2023 -0500
Support BigQuery tables with spaces (#29649)
---
.../src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
index 86f7a7f122b..686c29c13e3 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
@@ -577,7 +577,7 @@ public class BigQueryIO {
private static final String DATASET_REGEXP = "[-\\w.]{1,1024}";
/** Regular expression that matches Table IDs. */
- private static final String TABLE_REGEXP = "[-\\w$@]{1,1024}";
+ private static final String TABLE_REGEXP = "[-\\w$@ ]{1,1024}";
/**
* Matches table specifications in the form {@code
"[project_id]:[dataset_id].[table_id]"} or
(beam) 01/01: Merge pull request #29716:Add InfluxDB credentials as env variables
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 8ecfaa64fa81a536665153973049bfb4e0e732c7 Merge: eff6c203991 4431479fd90 Author: Alexey Romanenko <33895511+aromanenko-...@users.noreply.github.com> AuthorDate: Tue Dec 12 14:33:23 2023 +0100 Merge pull request #29716:Add InfluxDB credentials as env variables .github/workflows/beam_Java_JMH.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow_V2.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Direct.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Flink.yml| 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Spark.yml| 2 ++ .github/workflows/beam_PostCommit_Java_Tpcds_Dataflow.yml | 4 +++- .github/workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 ++ 10 files changed, 21 insertions(+), 1 deletion(-)
(beam) branch master updated (eff6c203991 -> 8ecfaa64fa8)
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from eff6c203991 Bump actions/setup-java from 3 to 4 (#29573) add 4431479fd90 Add InfluxDB credentials as env variables where they are missing new 8ecfaa64fa8 Merge pull request #29716:Add InfluxDB credentials as env variables The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/beam_Java_JMH.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow_V2.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Direct.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Flink.yml| 2 ++ .github/workflows/beam_PostCommit_Java_Nexmark_Spark.yml| 2 ++ .github/workflows/beam_PostCommit_Java_Tpcds_Dataflow.yml | 4 +++- .github/workflows/beam_PostCommit_Java_Tpcds_Flink.yml | 2 ++ .github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 ++ 10 files changed, 21 insertions(+), 1 deletion(-)
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new c6bbcce3ccb Updating config from bot c6bbcce3ccb is described below commit c6bbcce3ccb8bc2b0aee9e8a1a40c67a2ea93713 Author: github-actions AuthorDate: Tue Dec 12 13:07:10 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-29716.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29716.json b/scripts/ci/pr-bot/state/pr-state/pr-29716.json index 8a92900947e..c82cdd128d1 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-29716.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-29716.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
(beam) branch dependabot/github_actions/actions/setup-java-4 deleted (was 2c482dad6ed)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-java-4 in repository https://gitbox.apache.org/repos/asf/beam.git was 2c482dad6ed Bump actions/setup-java from 3 to 4 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(beam) branch master updated: Bump actions/setup-java from 3 to 4 (#29573)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new eff6c203991 Bump actions/setup-java from 3 to 4 (#29573)
eff6c203991 is described below
commit eff6c203991909613f2871179c265bab095002a2
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AuthorDate: Tue Dec 12 07:54:46 2023 -0500
Bump actions/setup-java from 3 to 4 (#29573)
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 3 to
4.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/v3...v4)
---
updated-dependencies:
- dependency-name: actions/setup-java
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot]
Co-authored-by: dependabot[bot]
<49699333+dependabot[bot]@users.noreply.github.com>
---
.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml | 2 +-
.github/workflows/build_release_candidate.yml | 10 +-
.github/workflows/code_completion_plugin_tests.yml| 2 +-
.github/workflows/run_rc_validation.yml | 4 ++--
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
b/.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
index e6740017afa..2c918b35497 100644
--- a/.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
+++ b/.github/workflows/beam_PostCommit_Java_Jpms_Direct_Java21.yml
@@ -71,7 +71,7 @@ jobs:
github_token: ${{ secrets.GITHUB_TOKEN }}
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
- name: Set up Java
- uses: actions/setup-java@v3.11.0
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: |
diff --git a/.github/workflows/build_release_candidate.yml
b/.github/workflows/build_release_candidate.yml
index 6f1499a6376..abf583b4dda 100644
--- a/.github/workflows/build_release_candidate.yml
+++ b/.github/workflows/build_release_candidate.yml
@@ -59,7 +59,7 @@ jobs:
ref: "v${{ github.event.inputs.RELEASE }}-RC${{
github.event.inputs.RC }}"
repository: apache/beam
- name: Install Java 8
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '8'
@@ -107,7 +107,7 @@ jobs:
echo "Must provide an apache password to stage artifacts to
https://dist.apache.org/repos/dist/dev/beam/;
fi
- name: Install Java 8
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '8'
@@ -243,7 +243,7 @@ jobs:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@v1.3.0
- name: Install Java 11
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '11'
@@ -255,7 +255,7 @@ jobs:
- run: echo "JAVA11_HOME=${JAVA_HOME}" >> "$GITHUB_OUTPUT"
id: export-java11
- name: Install Java 8
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '8'
@@ -306,7 +306,7 @@ jobs:
with:
node-version: '16'
- name: Install Java 8
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '8'
diff --git a/.github/workflows/code_completion_plugin_tests.yml
b/.github/workflows/code_completion_plugin_tests.yml
index 38ffd2fbd3f..9dbb5f6452f 100644
--- a/.github/workflows/code_completion_plugin_tests.yml
+++ b/.github/workflows/code_completion_plugin_tests.yml
@@ -73,7 +73,7 @@ jobs:
# Setup Java environment for the next steps
- name: Setup Java
-uses: actions/setup-java@v3
+uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '11'
diff --git a/.github/workflows/run_rc_validation.yml
b/.github/workflows/run_rc_validation.yml
index 35fb3ce05ea..130a2ce25ee 100644
--- a/.github/workflows/run_rc_validation.yml
+++ b/.github/workflows/run_rc_validation.yml
@@ -120,7 +120,7 @@ jobs:
- name: Setup Java JDK
- uses: actions/setup-java@v3.8.0
+ uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11
@@ -187,7 +187,7 @@ jobs:
uses: azure/setup-kubectl@v3
- name: Setup Java JDK
- uses: actions/setup-java@v3.8.0
+ uses: actions/setup-java@v4
with:
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 94eb3b8e28a Updating config from bot
94eb3b8e28a is described below
commit 94eb3b8e28a2a81eccea8fc1fa05cae470d7ddf6
Author: github-actions
AuthorDate: Tue Dec 12 12:13:53 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index def6a2ac3f2..c8c0ba0f1a5 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1702382886854,
+"damccorm": 1702383230637,
"Abacn": 1702037652612
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5508c53ebbc Updating config from bot
5508c53ebbc is described below
commit 5508c53ebbc5d3516d026ad9f095affdbf6db295
Author: github-actions
AuthorDate: Tue Dec 12 12:13:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29573.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29573.json
b/scripts/ci/pr-bot/state/pr-state/pr-29573.json
index 9c51466b9ee..8a92900947e 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-29573.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29573.json
@@ -1,7 +1,7 @@
{
"commentedAboutFailingChecks": false,
"reviewersAssignedForLabels": {
-"build": "Abacn"
+"build": "damccorm"
},
"nextAction": "Reviewers",
"stopReviewerNotifications": false,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 48de4f7db14 Updating config from bot
48de4f7db14 is described below
commit 48de4f7db142e995be3f5d828e9f73d781b96e12
Author: github-actions
AuthorDate: Tue Dec 12 12:08:10 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 5e384fde8d7..def6a2ac3f2 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1701923731135,
+"damccorm": 1702382886854,
"Abacn": 1702037652612
}
}
\ No newline at end of file
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 45c5f8bb0f6 Updating config from bot
45c5f8bb0f6 is described below
commit 45c5f8bb0f6bd0e0fff558a596a892f53a968ea8
Author: github-actions
AuthorDate: Tue Dec 12 12:08:08 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29716.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29716.json
b/scripts/ci/pr-bot/state/pr-state/pr-29716.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29716.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
(beam) branch aromanenko-dev-patch-1 created (now 5542d5927c4)
This is an automated email from the ASF dual-hosted git repository. aromanenko pushed a change to branch aromanenko-dev-patch-1 in repository https://gitbox.apache.org/repos/asf/beam.git at 5542d5927c4 Update beam_PostCommit_Java_Tpcds_Spark.yml This branch includes the following new commits: new 5542d5927c4 Update beam_PostCommit_Java_Tpcds_Spark.yml The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(beam) 01/01: Update beam_PostCommit_Java_Tpcds_Spark.yml
This is an automated email from the ASF dual-hosted git repository.
aromanenko pushed a commit to branch aromanenko-dev-patch-1
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 5542d5927c48716ae6af70778e12167f834ad497
Author: Alexey Romanenko <33895511+aromanenko-...@users.noreply.github.com>
AuthorDate: Tue Dec 12 12:46:04 2023 +0100
Update beam_PostCommit_Java_Tpcds_Spark.yml
---
.github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml
b/.github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml
index 370c9c406d4..580cfc83132 100644
--- a/.github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml
+++ b/.github/workflows/beam_PostCommit_Java_Tpcds_Spark.yml
@@ -54,6 +54,8 @@ env:
--dataDirectory=gs://beam-tpcds/datasets/parquet/nonpartitioned
--resultsDirectory=gs://beam-tpcds/results/spark3-rdd/
--tpcParallel=1
+ INFLUXDB_USER: ${{ secrets.INFLUXDB_USER }}
+ INFLUXDB_USER_PASSWORD: ${{ secrets.INFLUXDB_USER_PASSWORD }}
tpcdsBigQueryArgs: |
--bigQueryTable=tpcds
--bigQueryDataset=tpcds
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 344d0bf961f Updating config from bot
344d0bf961f is described below
commit 344d0bf961f89c310e3515b2556338c87666b6c5
Author: github-actions
AuthorDate: Tue Dec 12 09:34:42 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index 1b0c1a10f20..1cd26f3d827 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
"kennknowles": 1701967055102,
-"robertwb": 1701912589475,
+"robertwb": 1702373679889,
"kileys": 1674428678843,
"apilloud": 1678822446183,
"Abacn": 1702037652272,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 95b08c44fd8 Updating config from bot
95b08c44fd8 is described below
commit 95b08c44fd8f8f9070cac518176b939be4391ef7
Author: github-actions
AuthorDate: Tue Dec 12 09:34:43 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
index e831582aa6e..e64821d43ab 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json
@@ -1,7 +1,7 @@
{
"label": "io",
"dateOfLastReviewAssignment": {
-"chamikaramj": 1701459290723,
+"chamikaramj": 1702373679889,
"johnjcasey": 1701967055102,
"pabloem": 1691787951165,
"Abacn": 1701761214632,
(beam) branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 5f3b40bb262 Updating config from bot
5f3b40bb262 is described below
commit 5f3b40bb262cbe3368d59d273008832c301b040b
Author: github-actions
AuthorDate: Tue Dec 12 09:34:41 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-29715.json | 11 +++
1 file changed, 11 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-29715.json
b/scripts/ci/pr-bot/state/pr-state/pr-29715.json
new file mode 100644
index 000..063ce8593a5
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-29715.json
@@ -0,0 +1,11 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "robertwb",
+"io": "chamikaramj"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
