[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9fc88142dff Updating config from bot
9fc88142dff is described below
commit 9fc88142dff06935e27e79af99067c8340ba08e2
Author: github-actions
AuthorDate: Wed Sep 20 06:06:58 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index bf4dfd10ece..90043692c34 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1695159288720,
-"Abacn": 1695134103573
+"Abacn": 1695190015080
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 52208a4f1cc Updating config from bot
52208a4f1cc is described below
commit 52208a4f1cc6a9ac66361b0a4388bc25e23e4634
Author: github-actions
AuthorDate: Wed Sep 20 06:06:56 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28552.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28552.json
b/scripts/ci/pr-bot/state/pr-state/pr-28552.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28552.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 19ffd6ef62a Updating config from bot
19ffd6ef62a is described below
commit 19ffd6ef62ac18cc45bb25d1b03c6dda0f132e61
Author: github-actions
AuthorDate: Wed Sep 20 05:56:15 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28553.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28553.json
b/scripts/ci/pr-bot/state/pr-state/pr-28553.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28553.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch dependabot/github_actions/actions/checkout-4 created (now 2e058f18cbb)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/checkout-4 in repository https://gitbox.apache.org/repos/asf/beam.git at 2e058f18cbb Bump actions/checkout from 3 to 4 No new revisions were added by this update.
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 2e06082fc51 Updating config from bot
2e06082fc51 is described below
commit 2e06082fc51539285a1aace1da16bc4059ebccc5
Author: github-actions
AuthorDate: Wed Sep 20 05:06:10 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 2b81e121a12..13256dd9b91 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1695186357128,
-"jrmccluskey": 1695166481364,
+"jrmccluskey": 1695186367677,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1695186362698
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 6f33a10836b Updating config from bot
6f33a10836b is described below
commit 6f33a10836bfe43a5b256433708214df5a4a2586
Author: github-actions
AuthorDate: Wed Sep 20 05:06:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28550.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28550.json
b/scripts/ci/pr-bot/state/pr-state/pr-28550.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28550.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 7bf85189430 Updating config from bot
7bf85189430 is described below
commit 7bf85189430560ecc0b2b9b02c496b521b46c36d
Author: github-actions
AuthorDate: Wed Sep 20 05:06:09 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28549.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28549.json
b/scripts/ci/pr-bot/state/pr-state/pr-28549.json
new file mode 100644
index 000..1c2c2e31975
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28549.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new ea0352df2f2 Updating config from bot ea0352df2f2 is described below commit ea0352df2f2b21d212eef38fe0866009e4255ad1 Author: github-actions AuthorDate: Wed Sep 20 05:06:06 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 65169f3fc03..2b81e121a12 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1695166481364, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1695150352743 +"riteshghorse": 1695186362698 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bd3c7d26fd6 Updating config from bot
bd3c7d26fd6 is described below
commit bd3c7d26fd64ca1569cba3f8bfd07d0987b4fa65
Author: github-actions
AuthorDate: Wed Sep 20 05:06:00 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 27873af70de..65169f3fc03 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -1,7 +1,7 @@
{
"label": "go",
"dateOfLastReviewAssignment": {
-"lostluck": 1695099965508,
+"lostluck": 1695186357128,
"jrmccluskey": 1695166481364,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new aa08b8f8bc9 Updating config from bot
aa08b8f8bc9 is described below
commit aa08b8f8bc9668fce94c8707e413c4e035e4fa68
Author: github-actions
AuthorDate: Wed Sep 20 05:05:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28551.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28551.json
b/scripts/ci/pr-bot/state/pr-state/pr-28551.json
new file mode 100644
index 000..8ea94810368
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28551.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "lostluck"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated (4184f5ea521 -> 79d0a8d1109)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 4184f5ea521 Update HuggingFace api doc and add text2audio pipeline task (#28474) add 79d0a8d1109 tooltips, footer, humanize (#28538) No new revisions were added by this update. Summary of changes: .../runners/prism/internal/web/assets/style.css| 58 +- .../pkg/beam/runners/prism/internal/web/debugz.go | 25 +++--- .../beam/runners/prism/internal/web/debugz.html| 10 ++-- .../pkg/beam/runners/prism/internal/web/index.html | 52 +-- 4 files changed, 108 insertions(+), 37 deletions(-)
[beam] branch dependabot/go_modules/sdks/github.com/docker/docker-24.0.6incompatible created (now 104f54f61e2)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/docker/docker-24.0.6incompatible in repository https://gitbox.apache.org/repos/asf/beam.git at 104f54f61e2 Bump github.com/docker/docker in /sdks No new revisions were added by this update.
[beam] branch dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.24.0 created (now 2234f19f4f8)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/testcontainers/testcontainers-go-0.24.0 in repository https://gitbox.apache.org/repos/asf/beam.git at 2234f19f4f8 Bump github.com/testcontainers/testcontainers-go in /sdks No new revisions were added by this update.
[beam] branch dependabot/go_modules/sdks/google.golang.org/api-0.142.0 created (now d3fdb6ab1d2)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/google.golang.org/api-0.142.0 in repository https://gitbox.apache.org/repos/asf/beam.git at d3fdb6ab1d2 Bump google.golang.org/api from 0.141.0 to 0.142.0 in /sdks No new revisions were added by this update.
[beam] branch nightly-refs/heads/master updated (52abb1452e6 -> b39089870e7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch nightly-refs/heads/master in repository https://gitbox.apache.org/repos/asf/beam.git from 52abb1452e6 Merge pull request #28485 [BEAM-28399] Ensure dataflow experiments are always set when dataflowflow runner is used. add 011efacee3f Add GitHub Workflow Replacement for Jenkins job_PostCommit_SQL (#28469) add f087afad003 Remove extra registrations for createFn. (#28488) add fa144f45b14 Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_Examples_Dataflow_V2 (#28352) add 3f56f6578b7 Comment results to PR only when issue_comments (#28480) add 8c7e8b0949a Retrieve element type from input PCollection in parquetio.Write (#28491) add a0e9775ac25 Support legacy DATE and TIME logical types in xlang JdbcIO (#28382) add 1fc54fdfb91 Add Load Tests CoGBK Dataflow Streaming Java github action (#28372) add ffde6888ee0 Add GitHub Workflow Replacement for Jenkins job_PostCommit_Python (#28301) add de10fbd5e5b Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_PVR_* (#28412) add 7e830593e61 Remove TableSchema to JSON conversion. (#28274) add 99743347470 migrate CancelStaleDataflowJobs to GitHub (#28408) add e7e140ec216 Implement unittest add f37d86014ae Implement fix add 4a5a0076f35 Update translations_test.py add b3a8e1531ca yapf add 881c5979e96 Merge pull request #28489 Propagate annotations of CombinePerKey transforms. add b6f432dbaad Bump google.golang.org/api from 0.140.0 to 0.141.0 in /sdks (#28463) add db01dbe5f3c create run_portable_pipeline test add 7642b60385c implement change - required seemingly unrelated changes as well add 718fb77722b Move confib validation to runner execution time add ee603b7d279 Move dot requiring tests out into separate testcase add e42f4fb6708 Use module-specific logger add 36038fcad4c use tempdir for unit test add 7f709e712c3 remove unuseful test comments add 90059bdbc57 mypy fix add 91842e632b2 Merge pull request #28496 RenderRunner cleanup and fixes. add 0d37c6f45d8 [#28187] Add standalone prism validates runner precommit (#28487) add d6068ad66b0 Updating Storage API Autosharding documentation to include that it doesn't work on Runner V2 (#28233) add 3024ec24006 Better errors when inputs are omitted. (#28289) add cd65aac2be7 Fix broken link (#28510) add 6d7c6999a01 Fix building prism in action. (#28509) add 8871a4e8d2c Remove internal series.append call from frames.py (#28499) add 71c68ca041d Fix numeric_only logic in frames_test for Pandas 2 (#28422) add a722740e2cc Support kw only arguments for frame methods for Pandas 2 (#28454) add d79f537927c Suppress FutureWarnings for Pandas < 2 (#28481) add 08a97671516 Sets recommended max Xmx 32G when set_recommended_max_xmx experiment is enabled (#28442) add 5ffddb82f72 Add PerformanceTests BigQueryIO actions (#28325) add 49283e31ece Bump archive from 3.3.7 to 3.3.8 in /playground/frontend (#28351) add 79f46e00184 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend (#28350) add 34024902746 Bump actions/setup-python from 2 to 4 (#28275) add 8f3a38e7d09 kubectl install direct and use only auth plugin from gcloud (#28501) add 2ad06b3cdcd Bump actions/checkout from 3 to 4 (#28302) add c7274e9b151 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28519) add 9c4e8bff7c9 Support writes to Postgres with pgx driver (#28521) add 346541b1b82 Update build.gradle -- remove duplicate deps (#28377) add 82228873f41 Fix numpy install when building wheels (#28532) add 3915c85b842 [Document]: `HealthcareApiClient` methods. (#28482) add e6ecface49a Add GitHub Workflow Replacements for Jenkins job_PostCommit_Java_Nexmark (#28468) add 6f91ef83cad Migrate "beam_PostCommit_Python_Nexmark_Direct" Jenkins jobs to GitHub Actions (#28483) add 6c4bdf10556 added beam_PostCommit_Javadoc job to GitHub Actions (#28522) add a90879af543 Revert "Remove TableSchema to JSON conversion. (#28274)" (#28533) add f2a0c76703b Remove inplace argument from set_axis for Pandas 2 (#28536) add 0f2f3b12ac6 Only value_counts.drop_na for Pandas 2 (#28500) add 0f727927c01 Return errors for insert_rows_json exceptions (#21080) (#28091) add 272da41a1cc Updating access modifiers for utility methods (#28511) add 6261a007a9e Log user code exceptions in Java (#28514) add 612bfc4b486 Log user code exceptions in Python (#28515) add 27ada99648b Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#28517) add a1ec0552a34 Add extra marks to VertexAI IT tests (#28534) add 845dfc51e07 Create py.typed for use with mypy (#27942) add b39089870e7 Remove unused ToRow, GroupByKey, mark WithSchema as experimental. (#28504) No n
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 530341d5e46 Updating config from bot
530341d5e46 is described below
commit 530341d5e469a4b361512924dc55b9df63ef097a
Author: github-actions
AuthorDate: Wed Sep 20 03:34:48 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28548.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28548.json
b/scripts/ci/pr-bot/state/pr-state/pr-28548.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28548.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new de1226ce280 Updating config from bot de1226ce280 is described below commit de1226ce2806eff07df04789f3647d9702fb46a1 Author: github-actions AuthorDate: Wed Sep 20 03:06:09 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28538.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28538.json b/scripts/ci/pr-bot/state/pr-state/pr-28538.json index aa3638a74ec..1e9dcd816d5 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28538.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28538.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch master updated: Update HuggingFace api doc and add text2audio pipeline task (#28474)
This is an automated email from the ASF dual-hosted git repository.
riteshghorse pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 4184f5ea521 Update HuggingFace api doc and add text2audio pipeline
task (#28474)
4184f5ea521 is described below
commit 4184f5ea521941fcba6b90769b2d7f35b7262471
Author: Ritesh Ghorse
AuthorDate: Tue Sep 19 22:40:15 2023 -0400
Update HuggingFace api doc and add text2audio pipeline task (#28474)
* update api doc and add text2audio pipeline task
* update doc
* update doc
* fix indent
* correct example snippet
---
.../apache_beam/ml/inference/huggingface_inference.py | 18 ++
1 file changed, 14 insertions(+), 4 deletions(-)
diff --git a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
index aee61336378..3ec063808ae 100644
--- a/sdks/python/apache_beam/ml/inference/huggingface_inference.py
+++ b/sdks/python/apache_beam/ml/inference/huggingface_inference.py
@@ -98,6 +98,7 @@ class PipelineTask(str, Enum):
TextClassification = 'text-classification'
TextGeneration = 'text-generation'
Text2TextGeneration = 'text2text-generation'
+ TextToAudio = 'text-to-audio'
TokenClassification = 'token-classification'
Translation = 'translation'
VideoClassification = 'video-classification'
@@ -570,7 +571,7 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
def __init__(
self,
task: Union[str, PipelineTask] = "",
- model=None,
+ model: str = "",
*,
inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
load_pipeline_args: Optional[Dict[str, Any]] = None,
@@ -593,9 +594,18 @@ class HuggingFacePipelineModelHandler(ModelHandler[str,
Args:
task (str or enum.Enum): task supported by HuggingFace Pipelines.
Accepts a string task or an enum.Enum from PipelineTask.
- model : path to pretrained model on Hugging Face Models Hub to use custom
-model for the chosen task. If the model already defines the task then
-no need to specify the task parameter.
+ model (str): path to the pretrained *model-id* on Hugging Face Models Hub
+to use custom model for the chosen task. If the `model` already defines
+the task then no need to specify the `task` parameter.
+Use the *model-id* string instead of an actual model here.
+Model-specific kwargs for `from_pretrained(..., **model_kwargs)` can be
+specified with `model_kwargs` using `load_pipeline_args`.
+
+Example Usage::
+ model_handler = HuggingFacePipelineModelHandler(
+task="text-generation", model="meta-llama/Llama-2-7b-hf",
+load_pipeline_args={'model_kwargs':{'quantization_map':config}})
+
inference_fn: the inference function to use during RunInference.
Default is _default_pipeline_inference_fn.
load_pipeline_args (Dict[str, Any]): keyword arguments to provide load
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 18408304be0 Updating config from bot
18408304be0 is described below
commit 18408304be097944368c0ea7bcdcf98f2156e1cd
Author: github-actions
AuthorDate: Wed Sep 20 01:49:38 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28546.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28546.json
b/scripts/ci/pr-bot/state/pr-state/pr-28546.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28546.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Remove unused ToRow, GroupByKey, mark WithSchema as experimental. (#28504)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new b39089870e7 Remove unused ToRow, GroupByKey, mark WithSchema as
experimental. (#28504)
b39089870e7 is described below
commit b39089870e72c5b901fbd9dbd75f72a55927fff4
Author: Robert Bradshaw
AuthorDate: Tue Sep 19 17:04:25 2023 -0700
Remove unused ToRow, GroupByKey, mark WithSchema as experimental. (#28504)
---
sdks/python/apache_beam/yaml/yaml_provider.py | 8 +---
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/sdks/python/apache_beam/yaml/yaml_provider.py
b/sdks/python/apache_beam/yaml/yaml_provider.py
index 382dcfd97dc..d01852a69c3 100644
--- a/sdks/python/apache_beam/yaml/yaml_provider.py
+++ b/sdks/python/apache_beam/yaml/yaml_provider.py
@@ -549,15 +549,9 @@ def create_builtin_provider():
'PyFilter': lambda keep: beam.Filter(
python_callable.PythonCallableWithSource(keep)),
'PyTransform': fully_qualified_named_transform,
- 'PyToRow': lambda fields: beam.Select(
- **{
- name: python_callable.PythonCallableWithSource(fn)
- for (name, fn) in fields.items()
- }),
- 'WithSchema': with_schema,
+ 'WithSchemaExperimental': with_schema,
'Flatten': Flatten,
'WindowInto': WindowInto,
- 'GroupByKey': beam.GroupByKey,
},
no_input_transforms=('Create', ))
[beam] branch master updated: Create py.typed for use with mypy (#27942)
This is an automated email from the ASF dual-hosted git repository. tvalentyn pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 845dfc51e07 Create py.typed for use with mypy (#27942) 845dfc51e07 is described below commit 845dfc51e070b16ee2d8831ba268547ff7bffd0e Author: David Cavazos AuthorDate: Tue Sep 19 16:43:02 2023 -0700 Create py.typed for use with mypy (#27942) Co-authored-by: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com> --- CHANGES.md | 1 + sdks/python/apache_beam/py.typed | 19 +++ 2 files changed, 20 insertions(+) diff --git a/CHANGES.md b/CHANGES.md index cdf93909cb6..bbe9d539531 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -68,6 +68,7 @@ * In Python, [RunInference](https://beam.apache.org/documentation/sdks/python-machine-learning/#why-use-the-runinference-api) now supports loading many models in the same transform using a [KeyedModelHandler](https://beam.apache.org/documentation/sdks/python-machine-learning/#use-a-keyed-modelhandler) ([#27628](https://github.com/apache/beam/issues/27628)). * In Python, the [VertexAIModelHandlerJSON](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.vertex_ai_inference.html#apache_beam.ml.inference.vertex_ai_inference.VertexAIModelHandlerJSON) now supports passing in inference_args. These will be passed through to the Vertex endpoint as parameters. +* Added support to run `mypy` on user pipelines ([#27906](https://github.com/apache/beam/issues/27906)) ## Breaking Changes diff --git a/sdks/python/apache_beam/py.typed b/sdks/python/apache_beam/py.typed new file mode 100644 index 000..1aea47b9e7d --- /dev/null +++ b/sdks/python/apache_beam/py.typed @@ -0,0 +1,19 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# Marker file for PEP 561. +# The apache-beam package uses inline types.
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 9b2b08b6866 Updating config from bot
9b2b08b6866 is described below
commit 9b2b08b6866d9bba7a73a314baa597274da2abec
Author: github-actions
AuthorDate: Tue Sep 19 23:34:42 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28545.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28545.json
b/scripts/ci/pr-bot/state/pr-state/pr-28545.json
new file mode 100644
index 000..1c2c2e31975
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28545.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 28cc333ed09 Updating config from bot
28cc333ed09 is described below
commit 28cc333ed099f4edf47bab927cf6bdf0257d650c
Author: github-actions
AuthorDate: Tue Sep 19 23:34:44 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
index 28150547ba6..27873af70de 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json
@@ -2,7 +2,7 @@
"label": "go",
"dateOfLastReviewAssignment": {
"lostluck": 1695099965508,
-"jrmccluskey": 1695103578648,
+"jrmccluskey": 1695166481364,
"youngoli": 1657688896155,
"damccorm": 1680501930289,
"riteshghorse": 1695150352743
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new e8b2a737437 Updating config from bot e8b2a737437 is described below commit e8b2a737437082519e398c0b45db3f87606dc2cf Author: github-actions AuthorDate: Tue Sep 19 22:34:38 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28474.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28474.json b/scripts/ci/pr-bot/state/pr-state/pr-28474.json index 321b31c0f05..7d1dfe55f2f 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28474.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28474.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch master updated: Add extra marks to VertexAI IT tests (#28534)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new a1ec0552a34 Add extra marks to VertexAI IT tests (#28534)
a1ec0552a34 is described below
commit a1ec0552a340aa99a6d0aadc77f49caf8acab110
Author: Jack McCluskey <34928439+jrmcclus...@users.noreply.github.com>
AuthorDate: Tue Sep 19 18:18:12 2023 -0400
Add extra marks to VertexAI IT tests (#28534)
---
sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py | 1 +
sdks/python/test-suites/dataflow/common.gradle | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git
a/sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py
b/sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py
index 4ef42fb10a7..8ba315ba00c 100644
--- a/sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py
+++ b/sdks/python/apache_beam/ml/inference/vertex_ai_inference_it_test.py
@@ -26,6 +26,7 @@ import pytest
from apache_beam.io.filesystems import FileSystems
from apache_beam.testing.test_pipeline import TestPipeline
+# pylint: disable=ungrouped-imports
try:
from apache_beam.examples.inference import vertex_ai_image_classification
from apache_beam.examples.inference import vertex_ai_llm_text_classification
diff --git a/sdks/python/test-suites/dataflow/common.gradle
b/sdks/python/test-suites/dataflow/common.gradle
index eee6e9d2188..8b8a5680899 100644
--- a/sdks/python/test-suites/dataflow/common.gradle
+++ b/sdks/python/test-suites/dataflow/common.gradle
@@ -418,7 +418,8 @@ task vertexAIInferenceTest {
"test_opts": testOpts,
"suite": "VertexAITests-df-py${pythonVersionSuffix}",
"collect": "uses_vertex_ai and it_postcommit" ,
- "runner": "TestDataflowRunner"
+ "runner": "TestDataflowRunner",
+ "requirements_file": "$requirementsFile"
]
def cmdArgs = mapToArgString(argMap)
exec {
[beam] branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 deleted (was 2e6cbfff538)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 in repository https://gitbox.apache.org/repos/asf/beam.git was 2e6cbfff538 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated: Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#28517)
This is an automated email from the ASF dual-hosted git repository. lostluck pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 27ada99648b Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#28517) 27ada99648b is described below commit 27ada99648b07e6e32047df26193326d16e5cffc Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Tue Sep 19 15:00:26 2023 -0700 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#28517) Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.11.83 to 1.11.84. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/feature/s3/manager/v1.11.83...feature/s3/manager/v1.11.84) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/feature/s3/manager dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- sdks/go.mod | 2 +- sdks/go.sum | 9 ++--- 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/sdks/go.mod b/sdks/go.mod index 30ac185b816..169f882b1ea 100644 --- a/sdks/go.mod +++ b/sdks/go.mod @@ -33,7 +33,7 @@ require ( github.com/aws/aws-sdk-go-v2 v1.21.0 github.com/aws/aws-sdk-go-v2/config v1.18.40 github.com/aws/aws-sdk-go-v2/credentials v1.13.38 - github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.83 + github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.84 github.com/aws/aws-sdk-go-v2/service/s3 v1.38.5 github.com/aws/smithy-go v1.14.2 github.com/docker/go-connections v0.4.0 diff --git a/sdks/go.sum b/sdks/go.sum index 15fa89e26c3..13d194fa019 100644 --- a/sdks/go.sum +++ b/sdks/go.sum @@ -85,19 +85,17 @@ github.com/aws/aws-sdk-go-v2 v1.21.0/go.mod h1:/RfNgGmRxI+iFOB1OeJUyxiU+9s88k3pf github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13 h1:OPLEkmhXf6xFPiz0bLeDArZIDx1NNS4oJyG4nv3Gct0= github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13/go.mod h1:gpAbvyDGQFozTEmlTFO8XcQKHzubdq0LzRyJpG6MiXM= github.com/aws/aws-sdk-go-v2/config v1.5.0/go.mod h1:RWlPOAW3E3tbtNAqTwvSW54Of/yP3oiZXMI0xfUdjyA= -github.com/aws/aws-sdk-go-v2/config v1.18.39/go.mod h1:+NH/ZigdPckFpgB1TRcRuWCB/Kbbvkxc/iNAKTq5RhE= github.com/aws/aws-sdk-go-v2/config v1.18.40 h1:dbu1llI/nTIL+r6sYHMeVLl99DM8J8/o1I4EPurnhLg= github.com/aws/aws-sdk-go-v2/config v1.18.40/go.mod h1:JjrCZQwSPGCoZRQzKHyZNNueaKO+kFaEy2sR6mCzd90= github.com/aws/aws-sdk-go-v2/credentials v1.3.1/go.mod h1:r0n73xwsIVagq8RsxmZbGSRQFj9As3je72C2WzUIToc= -github.com/aws/aws-sdk-go-v2/credentials v1.13.37/go.mod h1:ACLrdkd4CLZyXOghZ8IYumQbcooAcp2jo/s2xsFH8IM= github.com/aws/aws-sdk-go-v2/credentials v1.13.38 h1:gDAuCdVlA4lmmgQhvpZlscwicloCqH44vkxLklGkQLA= github.com/aws/aws-sdk-go-v2/credentials v1.13.38/go.mod h1:sD4G/Ybgp6s89mWIES3Xn97CsRLpxvz9uVSdv0UxY8I= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.3.0/go.mod h1:2LAuqPx1I6jNfaGDucWfA2zqQCYCOMCDHiCOciALyNw= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11 h1:uDZJF1hu0EVT/4bogChk8DyjSF6fof6uL/0Y26Ma7Fg= github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.11/go.mod h1:TEPP4tENqBGO99KwVpV9MlOX4NSrSLP8u3KRy2CDwA8= github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.3.2/go.mod h1:qaqQiHSrOUVOfKe6fhgQ6UzhxjwqVW8aHNegd6Ws4w4= -github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.83 h1:wcluDLIQ0uYaxv0fCWQRimbXkPdTgWHUD21j1CzXEwc= -github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.83/go.mod h1:nGCBuon134gW67yAtxHKV73x+tAcY/xG4ZPNPDB1h/I= +github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.84 h1:LENrVcqnWTyI8fbIUCvxAMe+fXbREIaXzcR8WPwco1U= +github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.84/go.mod h1:LHxCiYAStsgps4srke7HujyADd504MSkNXjLpOtICTc= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.41 h1:22dGT7PneFMx4+b3pz7lMTRyN8ZKH7M2cW4GP9yUS2g= github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.41/go.mod h1:CrObHAuPneJBlfEJ5T3szXOUkLEThaGfvnhTf33buas= github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.35 h1:SijA0mgjV8E+8G45ltVHs0fvKpTj8xmZJ3VwhGKtUSI= @@ -122,14 +120,11 @@ github.com/aws/aws-sdk-go-v2/service/s3 v1.11.1/go.mod h1:XLAGFrEjbvMCLvAtWLLP32 github.com/aws/aws-sdk-go-v2/service/s3 v1.38.5 h1:A42xdtStObqy7NGvzZKpnyNXvoOmm+FENobZ0/ssHWk= github.com/aws/aws-sdk-go-v2/service/s3 v1.38.5/go.mod h1:rDGMZA7f4pbmTtPOk5v5UM2lmX6UAbRnMDJeDvnH7AM= github.com/aws/aws-sdk-go-v2/service/sso v1.3.1/go.mod h1:J3A3RGUvuCZjvSuZEcOpHDnzZP/sKbhDWV2T1EOzFIM= -github.com/aws/aws-sdk-go-v2/service/sso v1.13.6/go.mod h1:fIAwKQKBFu90pBxx07BFOMJLpRUGu8VOzLJakeY+0K4= github.com/aws/aws-sdk-go-v2/service/sso v1.14.0 h1:AR/hlT
[beam] branch master updated (6261a007a9e -> 612bfc4b486)
This is an automated email from the ASF dual-hosted git repository. udim pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 6261a007a9e Log user code exceptions in Java (#28514) add 612bfc4b486 Log user code exceptions in Python (#28515) No new revisions were added by this update. Summary of changes: sdks/python/apache_beam/runners/common.py | 4 + .../apache_beam/runners/worker/log_handler_test.py | 151 + 2 files changed, 155 insertions(+)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f19d3e31a4a Updating config from bot
f19d3e31a4a is described below
commit f19d3e31a4a5ed9fbbc0577f8a278be85e0451bc
Author: github-actions
AuthorDate: Tue Sep 19 21:34:47 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
index 3d043fb44c0..dc8ffe427f9 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
@@ -1,7 +1,7 @@
{
"label": "no-matching-label",
"dateOfLastReviewAssignment": {
-"Abacn": 1693514093629,
+"Abacn": 1695159284514,
"AnandInguva": 1695152095424,
"chamikaramj": 1693649245641,
"damccorm": 1693946094959,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bf4c78669be Updating config from bot
bf4c78669be is described below
commit bf4c78669bee7ba5c629e5fd6b6d932a72da867e
Author: github-actions
AuthorDate: Tue Sep 19 21:34:50 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28539.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28539.json
b/scripts/ci/pr-bot/state/pr-state/pr-28539.json
index 242a48d7d3b..8a92900947e 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-28539.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28539.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 09269821efe Updating config from bot
09269821efe is described below
commit 09269821efe7aad521876b9756942334a3cdeb69
Author: github-actions
AuthorDate: Tue Sep 19 21:34:51 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 7a8812eee9c..bf4dfd10ece 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1695127361892,
+"damccorm": 1695159288720,
"Abacn": 1695134103573
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f4d10a88490 Updating config from bot
f4d10a88490 is described below
commit f4d10a8849077e47ee5d0cb5f32cf0864a402528
Author: github-actions
AuthorDate: Tue Sep 19 21:34:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28540.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28540.json
b/scripts/ci/pr-bot/state/pr-state/pr-28540.json
new file mode 100644
index 000..d1ca4634459
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28540.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 66d405e50b5 Updating config from bot
66d405e50b5 is described below
commit 66d405e50b55162e984990c634679319440fb3ed
Author: github-actions
AuthorDate: Tue Sep 19 21:34:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28542.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28542.json
b/scripts/ci/pr-bot/state/pr-state/pr-28542.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28542.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1f39f436cb8 Updating config from bot
1f39f436cb8 is described below
commit 1f39f436cb81fca0ec5d34943d5ed4395d7e9705
Author: github-actions
AuthorDate: Tue Sep 19 21:05:46 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28539.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28539.json
b/scripts/ci/pr-bot/state/pr-state/pr-28539.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28539.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated (272da41a1cc -> 6261a007a9e)
This is an automated email from the ASF dual-hosted git repository. udim pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 272da41a1cc Updating access modifiers for utility methods (#28511) add 6261a007a9e Log user code exceptions in Java (#28514) No new revisions were added by this update. Summary of changes: .../java/org/apache/beam/fn/harness/FnHarness.java | 1 - .../fn/harness/control/BeamFnControlClient.java| 4 +- .../fn/harness/control/ExecutionStateSampler.java | 23 .../harness/data/PCollectionConsumerRegistry.java | 47 +--- .../fn/harness/logging/BeamFnLoggingClient.java| 23 ++-- .../beam/fn/harness/logging/BeamFnLoggingMDC.java | 20 +++ .../harness/control/ExecutionStateSamplerTest.java | 25 .../data/PCollectionConsumerRegistryTest.java | 134 + .../harness/logging/BeamFnLoggingClientTest.java | 20 +++ 9 files changed, 263 insertions(+), 34 deletions(-)
[beam] branch master updated (0f727927c01 -> 272da41a1cc)
This is an automated email from the ASF dual-hosted git repository. johncasey pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 0f727927c01 Return errors for insert_rows_json exceptions (#21080) (#28091) add 272da41a1cc Updating access modifiers for utility methods (#28511) No new revisions were added by this update. Summary of changes: sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 2 +- .../io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/SchemaUtil.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new db330fde3f8 Updating config from bot
db330fde3f8 is described below
commit db330fde3f8b498906e2a82bb30c6ff7f21212c8
Author: github-actions
AuthorDate: Tue Sep 19 19:34:59 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
index 624e5c3901f..3d043fb44c0 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json
@@ -2,7 +2,7 @@
"label": "no-matching-label",
"dateOfLastReviewAssignment": {
"Abacn": 1693514093629,
-"AnandInguva": 1693179286936,
+"AnandInguva": 1695152095424,
"chamikaramj": 1693649245641,
"damccorm": 1693946094959,
"johnjcasey": 1694099166373,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new f5db8fa1eaa Updating config from bot
f5db8fa1eaa is described below
commit f5db8fa1eaad2fd7a3e8cda16174bc7a72fe6739
Author: github-actions
AuthorDate: Tue Sep 19 19:34:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28537.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28537.json
b/scripts/ci/pr-bot/state/pr-state/pr-28537.json
new file mode 100644
index 000..e6d8e3d0a02
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28537.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "AnandInguva"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 96eac8f91af Updating config from bot 96eac8f91af is described below commit 96eac8f91af5f8b27a966e3ab33c111de47a5601 Author: github-actions AuthorDate: Tue Sep 19 19:09:46 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28506.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28506.json b/scripts/ci/pr-bot/state/pr-state/pr-28506.json index c9972b0d659..08ee9b0fe49 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28506.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28506.json @@ -4,7 +4,7 @@ "no-matching-label": "tvalentyn" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 6b3a175625c Updating config from bot 6b3a175625c is described below commit 6b3a175625cc7fa541898e5812f49650353f9016 Author: github-actions AuthorDate: Tue Sep 19 19:05:57 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-go.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-go.json b/scripts/ci/pr-bot/state/reviewers-for-label-go.json index 4780d3d918c..28150547ba6 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-go.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-go.json @@ -5,6 +5,6 @@ "jrmccluskey": 1695103578648, "youngoli": 1657688896155, "damccorm": 1680501930289, -"riteshghorse": 1695099961281 +"riteshghorse": 1695150352743 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new cc1185516c6 Updating config from bot cc1185516c6 is described below commit cc1185516c6d0d500ac7fb7747a78a29c20478f7 Author: github-actions AuthorDate: Tue Sep 19 19:06:06 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json index d499d73ac54..624e5c3901f 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-no-matching-label.json @@ -11,6 +11,6 @@ "lostluck": 1694703966579, "riteshghorse": 1694453703953, "robertwb": 1694734481490, -"tvalentyn": 1693092898557 +"tvalentyn": 1695150362753 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 053dc035fc9 Updating config from bot
053dc035fc9 is described below
commit 053dc035fc9757004c95aee80f25dc034caa9d63
Author: github-actions
AuthorDate: Tue Sep 19 19:06:04 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28506.json | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28506.json
b/scripts/ci/pr-bot/state/pr-state/pr-28506.json
index 242a48d7d3b..c9972b0d659 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-28506.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28506.json
@@ -1,7 +1,9 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"no-matching-label": "tvalentyn"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 797c813d6e4 Updating config from bot
797c813d6e4 is described below
commit 797c813d6e417de68f604bc494e2894b631125ed
Author: github-actions
AuthorDate: Tue Sep 19 19:05:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28538.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28538.json
b/scripts/ci/pr-bot/state/pr-state/pr-28538.json
new file mode 100644
index 000..aa3638a74ec
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28538.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"go": "riteshghorse"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated: Return errors for insert_rows_json exceptions (#21080) (#28091)
This is an automated email from the ASF dual-hosted git repository.
ahmedabualsaud pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 0f727927c01 Return errors for insert_rows_json exceptions (#21080)
(#28091)
0f727927c01 is described below
commit 0f727927c0136c63e8127ab5e9d3279fd5748dc4
Author: Adam Whitmore <61937566+ajdub...@users.noreply.github.com>
AuthorDate: Tue Sep 19 15:05:46 2023 -0400
Return errors for insert_rows_json exceptions (#21080) (#28091)
---
CHANGES.md | 1 +
sdks/python/apache_beam/io/gcp/bigquery_test.py| 853 +++--
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 22 +-
.../apache_beam/io/gcp/bigquery_write_it_test.py | 51 +-
4 files changed, 681 insertions(+), 246 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index 47c35fe3491..cdf93909cb6 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -82,6 +82,7 @@
## Bugfixes
* Fixed exception chaining issue in GCS connector (Python)
([#26769](https://github.com/apache/beam/issues/26769#issuecomment-1700422615)).
+* Fixed streaming inserts exception handling, GoogleAPICallErrors are now
retried according to retry strategy and routed to failed rows where appropriate
rather than causing a pipeline error (Python)
([#21080](https://github.com/apache/beam/issues/21080)).
## Security Fixes
diff --git a/sdks/python/apache_beam/io/gcp/bigquery_test.py
b/sdks/python/apache_beam/io/gcp/bigquery_test.py
index 3a3033dfcaf..7e9c1e63474 100644
--- a/sdks/python/apache_beam/io/gcp/bigquery_test.py
+++ b/sdks/python/apache_beam/io/gcp/bigquery_test.py
@@ -77,7 +77,6 @@ from apache_beam.testing.util import assert_that
from apache_beam.testing.util import equal_to
from apache_beam.transforms.display import DisplayData
from apache_beam.transforms.display_test import DisplayDataItemMatcher
-from apache_beam.utils import retry
# Protect against environments where bigquery library is not available.
# pylint: disable=wrong-import-order, wrong-import-position
@@ -931,77 +930,269 @@ class TestWriteToBigQuery(unittest.TestCase):
'GCP dependencies are not installed')
class BigQueryStreamingInsertsErrorHandling(unittest.TestCase):
- # Using https://cloud.google.com/bigquery/docs/error-messages and
- # https://googleapis.dev/python/google-api-core/latest/_modules/google
- #/api_core/exceptions.html
- # to determine error types and messages to try for retriables.
+ # Running tests with a variety of exceptions from https://googleapis.dev
+ #
/python/google-api-core/latest/_modules/google/api_core/exceptions.html.
+ # Choosing some exceptions that produce reasons included in
+ # bigquery_tools._NON_TRANSIENT_ERRORS and some that are not
@parameterized.expand([
+ # reason not in _NON_TRANSIENT_ERRORS for row 1 on first attempt
+ # transient error retried and succeeds on second attempt, 0 rows sent to
+ # failed rows
param(
- exception_type=exceptions.Forbidden if exceptions else None,
- error_reason='rateLimitExceeded'),
+ insert_response=[
+exceptions.TooManyRequests if exceptions else None,
+None],
+ error_reason='Too Many Requests', # not in _NON_TRANSIENT_ERRORS
+ failed_rows=[]),
+ # reason not in _NON_TRANSIENT_ERRORS for row 1 on both attempts, sent to
+ # failed rows after hitting max_retries
+ param(
+ insert_response=[
+exceptions.InternalServerError if exceptions else None,
+exceptions.InternalServerError if exceptions else None],
+ error_reason='Internal Server Error', # not in _NON_TRANSIENT_ERRORS
+ failed_rows=['value1', 'value3', 'value5']),
+ # reason in _NON_TRANSIENT_ERRORS for row 1 on both attempts, sent to
+ # failed_rows after hitting max_retries
+ param(
+ insert_response=[
+exceptions.Forbidden if exceptions else None,
+exceptions.Forbidden if exceptions else None],
+ error_reason='Forbidden', # in _NON_TRANSIENT_ERRORS
+ failed_rows=['value1', 'value3', 'value5']),
+ ])
+ def test_insert_rows_json_exception_retry_always(
+ self, insert_response, error_reason, failed_rows):
+# In this test, a pipeline will always retry all caught exception types
+# since RetryStrategy is not set and defaults to RETRY_ALWAYS
+with mock.patch('time.sleep'):
+ call_counter = 0
+ mock_response = mock.Mock()
+ mock_response.reason = error_reason
+
+ def store_callback(table, **kwargs):
+nonlocal call_counter
+# raise exception if insert_response element is an exception
+if insert_response[call_counter]:
+ exception_type = insert_response[call_counter]
+ call_counter += 1
+ raise exception_type('some exception', respons
[beam] branch master updated: Only value_counts.drop_na for Pandas 2 (#28500)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 0f2f3b12ac6 Only value_counts.drop_na for Pandas 2 (#28500)
0f2f3b12ac6 is described below
commit 0f2f3b12ac6f13dd068e95c22146f7aa636c6df4
Author: caneff
AuthorDate: Tue Sep 19 15:03:10 2023 -0400
Only value_counts.drop_na for Pandas 2 (#28500)
---
sdks/python/apache_beam/dataframe/frames.py | 17 ++---
sdks/python/apache_beam/dataframe/frames_test.py | 2 ++
2 files changed, 16 insertions(+), 3 deletions(-)
diff --git a/sdks/python/apache_beam/dataframe/frames.py
b/sdks/python/apache_beam/dataframe/frames.py
index b864e952e19..7929c879bdd 100644
--- a/sdks/python/apache_beam/dataframe/frames.py
+++ b/sdks/python/apache_beam/dataframe/frames.py
@@ -2348,8 +2348,13 @@ class DeferredSeries(DeferredDataFrameOrSeries):
result = column.groupby(column, dropna=dropna).size()
-# groupby.size() names the index, which we don't need
-result.index.name = None
+# Pandas 2 introduces new naming for the results.
+if PD_VERSION >= (2, 0):
+ result.index.name = getattr(self, "name", None)
+ result.name = "proportion" if normalize else "count"
+else:
+ # groupby.size() names the index, which we don't need
+ result.index.name = None
if normalize:
return result / column.length()
@@ -4007,12 +4012,18 @@ class DeferredDataFrame(DeferredDataFrameOrSeries):
columns = subset or list(self.columns)
if dropna:
-dropped = self.dropna()
+# Must include subset here because otherwise we spuriously drop NAs due
+# to columns outside our subset.
+dropped = self.dropna(subset=subset)
else:
dropped = self
result = dropped.groupby(columns, dropna=dropna).size()
+ # Pandas 2 introduces new naming for the results.
+ if PD_VERSION >= (2,0):
+result.name = "proportion" if normalize else "count"
+
if normalize:
return result/dropped.length()
else:
diff --git a/sdks/python/apache_beam/dataframe/frames_test.py
b/sdks/python/apache_beam/dataframe/frames_test.py
index 5d904855f87..257d77e0a6b 100644
--- a/sdks/python/apache_beam/dataframe/frames_test.py
+++ b/sdks/python/apache_beam/dataframe/frames_test.py
@@ -694,6 +694,8 @@ class DeferredFrameTest(_AbstractFrameTest):
self._run_test(lambda df: df.value_counts(), df)
self._run_test(lambda df: df.value_counts(normalize=True), df)
+# Ensure we don't drop rows due to nan values in unused columns.
+self._run_test(lambda df: df.value_counts('num_wings'), df)
if PD_VERSION >= (1, 3):
# dropna=False is new in pandas 1.3
[beam] branch master updated: Remove inplace argument from set_axis for Pandas 2 (#28536)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new f2a0c76703b Remove inplace argument from set_axis for Pandas 2 (#28536)
f2a0c76703b is described below
commit f2a0c76703bb78cb46b5e67f86d4b353cf0d6a3d
Author: caneff
AuthorDate: Tue Sep 19 14:52:56 2023 -0400
Remove inplace argument from set_axis for Pandas 2 (#28536)
---
sdks/python/apache_beam/dataframe/frames.py | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/sdks/python/apache_beam/dataframe/frames.py
b/sdks/python/apache_beam/dataframe/frames.py
index 34fc17553b0..b864e952e19 100644
--- a/sdks/python/apache_beam/dataframe/frames.py
+++ b/sdks/python/apache_beam/dataframe/frames.py
@@ -1901,7 +1901,8 @@ class DeferredSeries(DeferredDataFrameOrSeries):
@frame_base.with_docs_from(pd.Series)
@frame_base.args_to_kwargs(pd.Series)
- @frame_base.populate_defaults(pd.Series)
+ @frame_base.populate_defaults(
+ pd.Series, removed_args=['inplace'] if PD_VERSION >= (2, 0) else None)
@frame_base.maybe_inplace
def set_axis(self, labels, **kwargs):
# TODO: assigning the index is generally order-sensitive, but we could
@@ -2675,7 +2676,9 @@ class DeferredDataFrame(DeferredDataFrameOrSeries):
@frame_base.with_docs_from(pd.DataFrame)
@frame_base.args_to_kwargs(pd.DataFrame)
- @frame_base.populate_defaults(pd.DataFrame)
+ @frame_base.populate_defaults(
+ pd.DataFrame,
+ removed_args=['inplace'] if PD_VERSION >= (2, 0) else None)
@frame_base.maybe_inplace
def set_axis(self, labels, axis, **kwargs):
if axis in ('index', 0):
[beam] branch revert-28274-i-28080/java/BigQueryIO/datumReaderFactory deleted (was cdfffd80097)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch revert-28274-i-28080/java/BigQueryIO/datumReaderFactory in repository https://gitbox.apache.org/repos/asf/beam.git was cdfffd80097 Revert "Remove TableSchema to JSON conversion. (#28274)" The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (6c4bdf10556 -> a90879af543)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 6c4bdf10556 added beam_PostCommit_Javadoc job to GitHub Actions (#28522) add a90879af543 Revert "Remove TableSchema to JSON conversion. (#28274)" (#28533) No new revisions were added by this update. Summary of changes: .../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 40 +- .../sdk/io/gcp/bigquery/BigQueryIOReadTest.java| 18 ++ 2 files changed, 43 insertions(+), 15 deletions(-)
[beam] branch master updated (6f91ef83cad -> 6c4bdf10556)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 6f91ef83cad Migrate "beam_PostCommit_Python_Nexmark_Direct" Jenkins
jobs to GitHub Actions (#28483)
add 6c4bdf10556 added beam_PostCommit_Javadoc job to GitHub Actions
(#28522)
No new revisions were added by this update.
Summary of changes:
.github/workflows/README.md| 3 ++-
...k_Streaming.yml => beam_PostCommit_Javadoc.yml} | 31 +-
2 files changed, 20 insertions(+), 14 deletions(-)
copy .github/workflows/{beam_PostCommit_Java_PVR_Flink_Streaming.yml =>
beam_PostCommit_Javadoc.yml} (79%)
[beam] branch master updated (e6ecface49a -> 6f91ef83cad)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from e6ecface49a Add GitHub Workflow Replacements for Jenkins job_PostCommit_Java_Nexmark (#28468) add 6f91ef83cad Migrate "beam_PostCommit_Python_Nexmark_Direct" Jenkins jobs to GitHub Actions (#28483) No new revisions were added by this update. Summary of changes: .github/workflows/README.md| 1 + .../beam_PostCommit_Python_Nexmark_Direct.yml | 138 + ..._Python_ValidatesContainer_Dataflow_With_RC.yml | 2 +- 3 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/beam_PostCommit_Python_Nexmark_Direct.yml
[beam] branch master updated (3915c85b842 -> e6ecface49a)
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 3915c85b842 [Document]: `HealthcareApiClient` methods. (#28482)
add e6ecface49a Add GitHub Workflow Replacements for Jenkins
job_PostCommit_Java_Nexmark (#28468)
No new revisions were added by this update.
Summary of changes:
.github/workflows/README.md| 6 ++
...l => beam_PostCommit_Java_Nexmark_Dataflow.yml} | 87 +++---
...> beam_PostCommit_Java_Nexmark_Dataflow_V2.yml} | 58 +--
...m_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml} | 67 ++---
...yml => beam_PostCommit_Java_Nexmark_Direct.yml} | 82 ++--
yml => beam_PostCommit_Java_Nexmark_Flink.yml} | 81 ++--
yml => beam_PostCommit_Java_Nexmark_Spark.yml} | 81 ++--
7 files changed, 242 insertions(+), 220 deletions(-)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink_Java11.yml
=> beam_PostCommit_Java_Nexmark_Dataflow.yml} (55%)
copy .github/workflows/{beam_PostCommit_Java_DataflowV2.yml =>
beam_PostCommit_Java_Nexmark_Dataflow_V2.yml} (61%)
copy .github/workflows/{beam_PostCommit_Java_Examples_Dataflow_V2_Java.yml =>
beam_PostCommit_Java_Nexmark_Dataflow_V2_Java.yml} (61%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Spark_Java11.yml
=> beam_PostCommit_Java_Nexmark_Direct.yml} (58%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Spark_Java11.yml
=> beam_PostCommit_Java_Nexmark_Flink.yml} (58%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Spark_Java11.yml
=> beam_PostCommit_Java_Nexmark_Spark.yml} (58%)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 61524eb674f Updating config from bot
61524eb674f is described below
commit 61524eb674f5ea8d443699e50668b713d69e0f4e
Author: github-actions
AuthorDate: Tue Sep 19 18:15:35 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28264.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28264.json
b/scripts/ci/pr-bot/state/pr-state/pr-28264.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28264.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 6ebbf78f1c0 Updating config from bot
6ebbf78f1c0 is described below
commit 6ebbf78f1c0b638c39167a936d1da42e91a09508
Author: github-actions
AuthorDate: Tue Sep 19 18:06:52 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28533.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28533.json
b/scripts/ci/pr-bot/state/pr-state/pr-28533.json
new file mode 100644
index 000..242a48d7d3b
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28533.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": true,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 8d047cf373a Updating config from bot 8d047cf373a is described below commit 8d047cf373a5135f839731d3954ff6a43ca9bead Author: github-actions AuthorDate: Tue Sep 19 18:06:44 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28534.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28534.json b/scripts/ci/pr-bot/state/pr-state/pr-28534.json index 88d1920a8bb..24d0e10a64c 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28534.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28534.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new e1e4ea106f6 Updating config from bot e1e4ea106f6 is described below commit e1e4ea106f69428a9874fa8b6ce6ecb55585f469 Author: github-actions AuthorDate: Tue Sep 19 17:59:32 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28536.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28536.json b/scripts/ci/pr-bot/state/pr-state/pr-28536.json index 959c289c017..30d25f1f3dd 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28536.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28536.json @@ -4,7 +4,7 @@ "python": "jrmccluskey" }, "nextAction": "Reviewers", - "stopReviewerNotifications": false, + "stopReviewerNotifications": true, "remindAfterTestsPass": [], "committerAssigned": false } \ No newline at end of file
[beam] branch master updated: [Document]: `HealthcareApiClient` methods. (#28482)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 3915c85b842 [Document]: `HealthcareApiClient` methods. (#28482)
3915c85b842 is described below
commit 3915c85b84207f5f2c1e9dc4dd441769f3c586a5
Author: Svetak Sundhar
AuthorDate: Tue Sep 19 17:47:57 2023 +
[Document]: `HealthcareApiClient` methods. (#28482)
* Create HealthcareUtils file with shared resources
* revert
* Document methods in HealthcareApiClient
* fix spotless
* caps
---
.../sdk/io/gcp/healthcare/HealthcareApiClient.java | 242 +++--
1 file changed, 178 insertions(+), 64 deletions(-)
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareApiClient.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareApiClient.java
index a72b55bd720..39c30b94942 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareApiClient.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HealthcareApiClient.java
@@ -36,60 +36,68 @@ import
org.apache.beam.sdk.io.gcp.healthcare.HttpHealthcareApiClient.HealthcareH
import org.checkerframework.checker.nullness.qual.Nullable;
import org.joda.time.Instant;
-/** Defines a client that talks to the Cloud Healthcare API (version v1). */
+/** Defines a client to communicate with the GCP HCLS API (version v1). */
public interface HealthcareApiClient {
/**
- * Fetches an Hl7v2 message by its name from a Hl7v2 store.
+ * Gets a Hl7v2 message by its name from a Hl7v2 store.
*
- * @param msgName the msg name
- * @return HL7v2 message
- * @throws IOException the io exception
- * @throws ParseException the parse exception
+ * @param msgName The message name to be retrieved.
+ * @return The HL7v2 message.
+ * @throws IOException The IO Exception.
+ * @throws ParseException The Parse Exception.
*/
Message getHL7v2Message(String msgName) throws IOException, ParseException;
/**
- * Delete hl 7 v 2 message empty.
+ * Deletes an HL7v2 message.
*
- * @param msgName the msg name
- * @return the empty
- * @throws IOException the io exception
+ * @param msgName The message name to be deleted.
+ * @return Empty.
+ * @throws IOException The IO Exception.
*/
Empty deleteHL7v2Message(String msgName) throws IOException;
/**
- * Gets HL7v2 store.
+ * Gets an HL7v2 store.
*
- * @param storeName the store name
- * @return the HL7v2 store
- * @throws IOException the io exception
+ * @param storeName The store name to be retrieved.
+ * @return The HL7v2 store.
+ * @throws IOException The IO Exception.
*/
Hl7V2Store getHL7v2Store(String storeName) throws IOException;
/**
- * Gets earliest hl 7 v 2 send time.
+ * Gets the earliest HL7v2 send time.
*
- * @param hl7v2Store the hl 7 v 2 store
- * @param filter the filter
- * @return the earliest hl 7 v 2 send time
- * @throws IOException the io exception
+ * @param hl7v2Store The HL7v2 store.
+ * @param filter the filter to be matched on.
+ * @return The earliest HL7v2 send time.
+ * @throws IOException The IO Exception.
*/
Instant getEarliestHL7v2SendTime(String hl7v2Store, @Nullable String filter)
throws IOException;
+ /**
+ * Gets the latest HL7v2 send time.
+ *
+ * @param hl7v2Store The HL7v2 store.
+ * @param filter The filter to be matched on.
+ * @return The latest HL7v2 send time.
+ * @throws IOException The IO Exception.
+ */
Instant getLatestHL7v2SendTime(String hl7v2Store, @Nullable String filter)
throws IOException;
/**
- * Make send time bound hl 7 v 2 list request.
+ * Time Bound HL7v2 list request.
*
- * @param hl7v2Store the hl 7 v 2 store
- * @param start the start
- * @param end the end
- * @param otherFilter the other filter
- * @param orderBy the order by
- * @param pageToken the page token
- * @return the list messages response
- * @throws IOException the io exception
+ * @param hl7v2Store The HL7v2 store.
+ * @param start Start time.
+ * @param end End time.
+ * @param otherFilter The filter outside of the sendTime.
+ * @param orderBy Order by.
+ * @param pageToken The page token.
+ * @return HTTP List response.
+ * @throws IOException The IO Exception.
*/
ListMessagesResponse makeSendTimeBoundHL7v2ListRequest(
String hl7v2Store,
@@ -103,12 +111,12 @@ public interface HealthcareApiClient {
/**
* Make hl 7 v 2 list request list messages response.
*
- * @param hl7v2Store the hl 7 v 2 store
- * @param filter the filter
- * @param orderBy the order by
- * @param pageToken the page token
-
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 2ccef57b213 Updating config from bot 2ccef57b213 is described below commit 2ccef57b213469d6d1b358e80015515a9a4c0382 Author: github-actions AuthorDate: Tue Sep 19 17:34:47 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index de04019d4dc..028d2d20bc9 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -9,7 +9,7 @@ "pabloem": 1681281324703, "y1chi": 1667002607045, "damccorm": 1695143152600, -"jrmccluskey": 1694779650767, +"jrmccluskey": 1695144883894, "riteshghorse": 1694813680161, "liferoad": 1694802882751 }
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 84e747e811a Updating config from bot
84e747e811a is described below
commit 84e747e811ae9a8391c1dc4e7c5924e9d4d61ef2
Author: github-actions
AuthorDate: Tue Sep 19 17:34:45 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28536.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28536.json
b/scripts/ci/pr-bot/state/pr-state/pr-28536.json
new file mode 100644
index 000..959c289c017
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28536.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "jrmccluskey"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 0a5ec93696f Updating config from bot 0a5ec93696f is described below commit 0a5ec93696fbcbdec35244226450660f88bb0559 Author: github-actions AuthorDate: Tue Sep 19 17:05:56 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-python.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-python.json b/scripts/ci/pr-bot/state/reviewers-for-label-python.json index fa265709ab2..de04019d4dc 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-python.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-python.json @@ -8,7 +8,7 @@ "tvalentyn": 1694790366018, "pabloem": 1681281324703, "y1chi": 1667002607045, -"damccorm": 1694780040541, +"damccorm": 1695143152600, "jrmccluskey": 1694779650767, "riteshghorse": 1694813680161, "liferoad": 1694802882751
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new fcf6c1fab68 Updating config from bot
fcf6c1fab68 is described below
commit fcf6c1fab685d679d339071c6c18e72db0dbeb94
Author: github-actions
AuthorDate: Tue Sep 19 17:05:54 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28534.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28534.json
b/scripts/ci/pr-bot/state/pr-state/pr-28534.json
new file mode 100644
index 000..88d1920a8bb
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28534.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"python": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch users/damccorm/wheels deleted (was ae079980307)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/wheels in repository https://gitbox.apache.org/repos/asf/beam.git was ae079980307 Syntax fix The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (346541b1b82 -> 82228873f41)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 346541b1b82 Update build.gradle -- remove duplicate deps (#28377) add 82228873f41 Fix numpy install when building wheels (#28532) No new revisions were added by this update. Summary of changes: .github/workflows/build_wheels.yml | 4 ++-- sdks/python/build.gradle | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
[beam] branch master updated (9c4e8bff7c9 -> 346541b1b82)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 9c4e8bff7c9 Support writes to Postgres with pgx driver (#28521) add 346541b1b82 Update build.gradle -- remove duplicate deps (#28377) No new revisions were added by this update. Summary of changes: sdks/java/io/google-ads/build.gradle | 1 - 1 file changed, 1 deletion(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new bba41943b73 Updating config from bot
bba41943b73 is described below
commit bba41943b73c5e12daa18c600d85eeb8847c41ac
Author: github-actions
AuthorDate: Tue Sep 19 15:48:30 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28532.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28532.json
b/scripts/ci/pr-bot/state/pr-state/pr-28532.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28532.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 7cf348ceb75 Updating config from bot
7cf348ceb75 is described below
commit 7cf348ceb75a49ed4a12a787e197cb6c8db22828
Author: github-actions
AuthorDate: Tue Sep 19 15:35:22 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-java.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
index c02d61823d2..c80504e3b02 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-java.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-java.json
@@ -3,7 +3,7 @@
"dateOfLastReviewAssignment": {
"lukecwik": 1680701852535,
"kennknowles": 1695080261877,
-"robertwb": 1694520852280,
+"robertwb": 1695137718373,
"kileys": 1674428678843,
"apilloud": 1678822446183,
"Abacn": 1694797564637,
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new 33f3c7c6da2 Updating config from bot 33f3c7c6da2 is described below commit 33f3c7c6da2c920794caac64a8ff304f3446c4e2 Author: github-actions AuthorDate: Tue Sep 19 15:35:24 2023 + Updating config from bot --- scripts/ci/pr-bot/state/reviewers-for-label-io.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-io.json b/scripts/ci/pr-bot/state/reviewers-for-label-io.json index 79681422a10..a428b11cbb6 100644 --- a/scripts/ci/pr-bot/state/reviewers-for-label-io.json +++ b/scripts/ci/pr-bot/state/reviewers-for-label-io.json @@ -6,7 +6,7 @@ "pabloem": 1691787951165, "Abacn": 1694779650767, "ahmedabu98": 1695105814268, -"bvolpato": 1694289944058, +"bvolpato": 1695137718373, "manavgarg": 1690826779210 } } \ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 41f93cd4b66 Updating config from bot
41f93cd4b66 is described below
commit 41f93cd4b66ceffae259715bd183633060c4a83d
Author: github-actions
AuthorDate: Tue Sep 19 15:35:20 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28377.json | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28377.json
b/scripts/ci/pr-bot/state/pr-state/pr-28377.json
index 242a48d7d3b..aa172f7e71a 100644
--- a/scripts/ci/pr-bot/state/pr-state/pr-28377.json
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28377.json
@@ -1,7 +1,10 @@
{
- "commentedAboutFailingChecks": true,
- "reviewersAssignedForLabels": {},
- "nextAction": "Author",
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"java": "robertwb",
+"io": "bvolpato"
+ },
+ "nextAction": "Reviewers",
"stopReviewerNotifications": false,
"remindAfterTestsPass": [],
"committerAssigned": false
[beam] 01/01: Revert "Remove TableSchema to JSON conversion. (#28274)"
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a commit to branch
revert-28274-i-28080/java/BigQueryIO/datumReaderFactory
in repository https://gitbox.apache.org/repos/asf/beam.git
commit cdfffd80097fd022fafd9512c2ed27ac14a15621
Author: Yi Hu
AuthorDate: Tue Sep 19 11:12:12 2023 -0400
Revert "Remove TableSchema to JSON conversion. (#28274)"
This reverts commit 7e830593e61ba1fbff16411b5825bfb4aea53ba2.
---
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 40 +-
.../sdk/io/gcp/bigquery/BigQueryIOReadTest.java| 18 ++
2 files changed, 43 insertions(+), 15 deletions(-)
diff --git
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
index 3c006d24d03..58d76931244 100644
---
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
+++
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java
@@ -19,7 +19,6 @@ package org.apache.beam.sdk.io.gcp.bigquery;
import static
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers.resolveTempLocation;
import static
org.apache.beam.sdk.io.gcp.bigquery.BigQueryResourceNaming.createTempTableReference;
-import static org.apache.beam.sdk.util.Preconditions.checkStateNotNull;
import static
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkArgument;
import static
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkState;
@@ -50,6 +49,7 @@ import com.google.protobuf.Descriptors;
import com.google.protobuf.DynamicMessage;
import com.google.protobuf.Message;
import java.io.IOException;
+import java.io.Serializable;
import java.lang.reflect.InvocationTargetException;
import java.util.Collections;
import java.util.List;
@@ -132,10 +132,13 @@ import org.apache.beam.sdk.values.TypeDescriptor;
import org.apache.beam.sdk.values.TypeDescriptors;
import org.apache.beam.sdk.values.ValueInSingleWindow;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.annotations.VisibleForTesting;
+import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Function;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.MoreObjects;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Predicates;
import org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Strings;
+import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Supplier;
+import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Suppliers;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.ImmutableList;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.Iterables;
import
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.collect.Lists;
@@ -646,19 +649,29 @@ public class BigQueryIO {
BigQueryUtils.tableRowFromBeamRow());
}
+ private static class TableSchemaFunction
+ implements Serializable, Function<@Nullable String, @Nullable
TableSchema> {
+@Override
+public @Nullable TableSchema apply(@Nullable String input) {
+ return BigQueryHelpers.fromJsonString(input, TableSchema.class);
+}
+ }
+
@VisibleForTesting
static class GenericDatumTransformer implements DatumReader {
private final SerializableFunction parseFn;
-private final TableSchema tableSchema;
+private final Supplier tableSchema;
private GenericDatumReader reader;
private org.apache.avro.Schema writerSchema;
public GenericDatumTransformer(
SerializableFunction parseFn,
-TableSchema tableSchema,
+String tableSchema,
org.apache.avro.Schema writer) {
this.parseFn = parseFn;
- this.tableSchema = tableSchema;
+ this.tableSchema =
+ Suppliers.memoize(
+ Suppliers.compose(new TableSchemaFunction(),
Suppliers.ofInstance(tableSchema)));
this.writerSchema = writer;
this.reader = new GenericDatumReader<>(this.writerSchema);
}
@@ -676,7 +689,7 @@ public class BigQueryIO {
@Override
public T read(T reuse, Decoder in) throws IOException {
GenericRecord record = (GenericRecord) this.reader.read(reuse, in);
- return parseFn.apply(new SchemaAndRecord(record, this.tableSchema));
+ return parseFn.apply(new SchemaAndRecord(record,
this.tableSchema.get()));
}
}
@@ -708,9 +721,16 @@ public class BigQueryIO {
.setDatumReaderFactory(
(SerializableFunction>)
input -> {
- TableSchema safeInput = checkStateNotNull(input);
- return (AvroSource.DatumReaderFactory)
-
[beam] branch revert-28274-i-28080/java/BigQueryIO/datumReaderFactory created (now cdfffd80097)
This is an automated email from the ASF dual-hosted git repository. yhu pushed a change to branch revert-28274-i-28080/java/BigQueryIO/datumReaderFactory in repository https://gitbox.apache.org/repos/asf/beam.git at cdfffd80097 Revert "Remove TableSchema to JSON conversion. (#28274)" This branch includes the following new commits: new cdfffd80097 Revert "Remove TableSchema to JSON conversion. (#28274)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 updated (2c37fc13eeb -> 2e6cbfff538)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 in repository https://gitbox.apache.org/repos/asf/beam.git omit 2c37fc13eeb Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks add 2e6cbfff538 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2c37fc13eeb) \ N -- N -- N refs/heads/dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 (2e6cbfff538) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes:
[beam] branch master updated: Support writes to Postgres with pgx driver (#28521)
This is an automated email from the ASF dual-hosted git repository.
jrmccluskey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 9c4e8bff7c9 Support writes to Postgres with pgx driver (#28521)
9c4e8bff7c9 is described below
commit 9c4e8bff7c97f513f9df498debdbcc08698413d7
Author: Johanna Öjeling <51084516+johannaojel...@users.noreply.github.com>
AuthorDate: Tue Sep 19 17:03:11 2023 +0200
Support writes to Postgres with pgx driver (#28521)
---
sdks/go/pkg/beam/io/databaseio/writer.go | 2 +-
sdks/go/pkg/beam/io/databaseio/writer_test.go | 6 ++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/sdks/go/pkg/beam/io/databaseio/writer.go
b/sdks/go/pkg/beam/io/databaseio/writer.go
index 4719831581f..ebd805b080d 100644
--- a/sdks/go/pkg/beam/io/databaseio/writer.go
+++ b/sdks/go/pkg/beam/io/databaseio/writer.go
@@ -108,7 +108,7 @@ type valueTemplateGenerator struct {
func (v *valueTemplateGenerator) generate(rowCount int, columnColunt int)
string {
switch v.driver {
- case "postgres":
+ case "postgres", "pgx":
// the point is to generate ($1,$2),($3,$4)
valueTemplates := make([]string, rowCount)
for i := 0; i < rowCount; i++ {
diff --git a/sdks/go/pkg/beam/io/databaseio/writer_test.go
b/sdks/go/pkg/beam/io/databaseio/writer_test.go
index 24f617a9b85..91d7fb9246d 100644
--- a/sdks/go/pkg/beam/io/databaseio/writer_test.go
+++ b/sdks/go/pkg/beam/io/databaseio/writer_test.go
@@ -39,6 +39,12 @@ func TestValueTemplateGenerator_generate(t *testing.T) {
columnCount: 10,
expected:"",
},
+ {
+ generator: &valueTemplateGenerator{"pgx"},
+ rowCount:4,
+ columnCount: 3,
+ expected:
"($1,$2,$3),($4,$5,$6),($7,$8,$9),($10,$11,$12)",
+ },
{
generator: &valueTemplateGenerator{"mysql"},
rowCount:4,
[beam] branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 updated (77f25fd23b3 -> 2c37fc13eeb)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 in repository https://gitbox.apache.org/repos/asf/beam.git omit 77f25fd23b3 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks add 5ffddb82f72 Add PerformanceTests BigQueryIO actions (#28325) add 49283e31ece Bump archive from 3.3.7 to 3.3.8 in /playground/frontend (#28351) add 79f46e00184 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend (#28350) add 34024902746 Bump actions/setup-python from 2 to 4 (#28275) add 8f3a38e7d09 kubectl install direct and use only auth plugin from gcloud (#28501) add 2ad06b3cdcd Bump actions/checkout from 3 to 4 (#28302) add c7274e9b151 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28519) add 2c37fc13eeb Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (77f25fd23b3) \ N -- N -- N refs/heads/dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/feature/s3/manager-1.11.84 (2c37fc13eeb) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .../arc/images/Dockerfile | 8 +++- .github/workflows/README.md| 3 ++ .github/workflows/assign_milestone.yml | 2 +- .github/workflows/beam_CancelStaleDataflowJobs.yml | 2 +- ...eam_LoadTests_Java_CoGBK_Dataflow_Streaming.yml | 2 +- ...erformanceTests_BigQueryIO_Batch_Java_Avro.yml} | 54 +++--- ...erformanceTests_BigQueryIO_Batch_Java_Json.yml} | 54 +++--- ...PerformanceTests_BigQueryIO_Streaming_Java.yml} | 54 +++--- .../beam_PostCommit_BeamMetrics_Publish.yml| 2 +- .github/workflows/beam_PostCommit_Go.yml | 2 +- .../workflows/beam_PostCommit_Go_Dataflow_ARM.yml | 2 +- .github/workflows/beam_PostCommit_Go_VR_Flink.yml | 2 +- .github/workflows/beam_PostCommit_Go_VR_Samza.yml | 2 +- .github/workflows/beam_PostCommit_Go_VR_Spark.yml | 2 +- .github/workflows/beam_PostCommit_Java.yml | 2 +- .../beam_PostCommit_Java_Avro_Versions.yml | 2 +- .../workflows/beam_PostCommit_Java_DataflowV1.yml | 2 +- .../workflows/beam_PostCommit_Java_DataflowV2.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 2 +- ...beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_V2.yml | 2 +- ...m_PostCommit_Java_Examples_Dataflow_V2_Java.yml | 2 +- .../beam_PostCommit_Java_Examples_Direct.yml | 2 +- .../beam_PostCommit_Java_Examples_Flink.yml| 2 +- .../beam_PostCommit_Java_Examples_Spark.yml| 2 +- .../beam_PostCommit_Java_Hadoop_Versions.yml | 2 +- .../beam_PostCommit_Java_IO_Performance_Tests.yml | 4 +- .../beam_PostCommit_Java_Jpms_Dataflow_Java11.yml | 2 +- .../beam_PostCommit_Java_Jpms_Dataflow_Java17.yml | 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java11.yml| 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java17.yml| 2 +- .../beam_PostCommit_Java_Jpms_Flink_Java11.yml | 2 +- .../beam_PostCommit_Java_Jpms_Spark_Java11.yml | 2 +- .../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 +- .../workflows/beam_PostCommit_Java_PVR_Samza.yml | 2 +- .../beam_PostCommit_Java_PVR_Spark3_Streaming.yml | 2 +- .../beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 +- .github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 +- ...am_PostCommit_Java_ValidatesRunner_Dataflow.yml | 2 +- ..._Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 +- ...mit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 +- ...PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 +- ..._Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 +- ...beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 +- ...it_Java_ValidatesRunner_Direct_JavaVersions.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 2 +- ...ostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Samza.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Spark.yml | 2 +- ...va_ValidatesRunner_SparkStructuredStreaming.yml | 2 +- ...ostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 +- ...am_PostCommit_Java_ValidatesRunner_Twister2.yml |
[beam] branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.13.38 deleted (was bd927324463)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/credentials-1.13.38 in repository https://gitbox.apache.org/repos/asf/beam.git was bd927324463 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.18.40 deleted (was 7da730cd1fa)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/aws/aws-sdk-go-v2/config-1.18.40 in repository https://gitbox.apache.org/repos/asf/beam.git was 7da730cd1fa Bump github.com/aws/aws-sdk-go-v2/config in /sdks The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated: Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28519)
This is an automated email from the ASF dual-hosted git repository. jrmccluskey pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new c7274e9b151 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28519) c7274e9b151 is described below commit c7274e9b151f40ac99574e618a8bf66ee234716d Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Tue Sep 19 11:00:31 2023 -0400 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#28519) Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.39 to 1.18.40. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.18.39...config/v1.18.40) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- sdks/go.mod | 12 ++-- sdks/go.sum | 15 ++- 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/sdks/go.mod b/sdks/go.mod index e28e54896b8..30ac185b816 100644 --- a/sdks/go.mod +++ b/sdks/go.mod @@ -31,8 +31,8 @@ require ( cloud.google.com/go/spanner v1.49.0 cloud.google.com/go/storage v1.33.0 github.com/aws/aws-sdk-go-v2 v1.21.0 - github.com/aws/aws-sdk-go-v2/config v1.18.39 - github.com/aws/aws-sdk-go-v2/credentials v1.13.37 + github.com/aws/aws-sdk-go-v2/config v1.18.40 + github.com/aws/aws-sdk-go-v2/credentials v1.13.38 github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.11.83 github.com/aws/aws-sdk-go-v2/service/s3 v1.38.5 github.com/aws/smithy-go v1.14.2 @@ -96,9 +96,9 @@ require ( github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.1.36 // indirect github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.35 // indirect github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.15.4 // indirect - github.com/aws/aws-sdk-go-v2/service/sso v1.13.6 // indirect - github.com/aws/aws-sdk-go-v2/service/ssooidc v1.15.6 // indirect - github.com/aws/aws-sdk-go-v2/service/sts v1.21.5 // indirect + github.com/aws/aws-sdk-go-v2/service/sso v1.14.0 // indirect + github.com/aws/aws-sdk-go-v2/service/ssooidc v1.16.0 // indirect + github.com/aws/aws-sdk-go-v2/service/sts v1.22.0 // indirect github.com/cenkalti/backoff/v4 v4.2.1 // indirect github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect @@ -107,7 +107,7 @@ require ( github.com/containerd/containerd v1.7.3 // indirect github.com/cpuguy83/dockercfg v0.3.1 // indirect github.com/docker/distribution v2.8.2+incompatible // indirect - github.com/docker/docker v24.0.5+incompatible // indirect; but required to resolve issue docker has with go1.20 + github.com/docker/docker v24.0.5+incompatible // but required to resolve issue docker has with go1.20 github.com/docker/go-units v0.5.0 // indirect github.com/envoyproxy/go-control-plane v0.11.1 // indirect github.com/envoyproxy/protoc-gen-validate v1.0.2 // indirect diff --git a/sdks/go.sum b/sdks/go.sum index f60c5c87fb1..15fa89e26c3 100644 --- a/sdks/go.sum +++ b/sdks/go.sum @@ -85,11 +85,13 @@ github.com/aws/aws-sdk-go-v2 v1.21.0/go.mod h1:/RfNgGmRxI+iFOB1OeJUyxiU+9s88k3pf github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13 h1:OPLEkmhXf6xFPiz0bLeDArZIDx1NNS4oJyG4nv3Gct0= github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.4.13/go.mod h1:gpAbvyDGQFozTEmlTFO8XcQKHzubdq0LzRyJpG6MiXM= github.com/aws/aws-sdk-go-v2/config v1.5.0/go.mod h1:RWlPOAW3E3tbtNAqTwvSW54Of/yP3oiZXMI0xfUdjyA= -github.com/aws/aws-sdk-go-v2/config v1.18.39 h1:oPVyh6fuu/u4OiW4qcuQyEtk7U7uuNBmHmJSLg1AJsQ= github.com/aws/aws-sdk-go-v2/config v1.18.39/go.mod h1:+NH/ZigdPckFpgB1TRcRuWCB/Kbbvkxc/iNAKTq5RhE= +github.com/aws/aws-sdk-go-v2/config v1.18.40 h1:dbu1llI/nTIL+r6sYHMeVLl99DM8J8/o1I4EPurnhLg= +github.com/aws/aws-sdk-go-v2/config v1.18.40/go.mod h1:JjrCZQwSPGCoZRQzKHyZNNueaKO+kFaEy2sR6mCzd90= github.com/aws/aws-sdk-go-v2/credentials v1.3.1/go.mod h1:r0n73xwsIVagq8RsxmZbGSRQFj9As3je72C2WzUIToc= -github.com/aws/aws-sdk-go-v2/credentials v1.13.37 h1:BvEdm09+ZEh2XtN+PVHPcYwKY3wIeB6pw7vPRM4M9/U= github.com/aws/aws-sdk-go-v2/credentials v1.13.37/go.mod h1:ACLrdkd4CLZyXOghZ8IYumQbcooAcp2jo/s2xsFH8IM= +github.com/aws/aws-sdk-go-v2/credentials v1.13.38 h1:gDAuCdVlA4lmmgQhvpZlscwicloCqH44vkxLklGkQLA= +github.com/aws/aws-sdk-go-v2/credentials v1.13.38/go.mod h1:sD4G/Ybgp6s89mWIES3Xn97CsRLpxvz9uVSdv0UxY8I= github.com/aws/aws-sdk-g
[beam] branch dependabot/github_actions/actions/checkout-4 deleted (was 19169c21d2c)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch dependabot/github_actions/actions/checkout-4 in repository https://gitbox.apache.org/repos/asf/beam.git was 19169c21d2c Bump actions/checkout from 3 to 4 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (8f3a38e7d09 -> 2ad06b3cdcd)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 8f3a38e7d09 kubectl install direct and use only auth plugin from gcloud (#28501) add 2ad06b3cdcd Bump actions/checkout from 3 to 4 (#28302) No new revisions were added by this update. Summary of changes: .github/workflows/assign_milestone.yml | 2 +- .github/workflows/beam_CancelStaleDataflowJobs.yml | 2 +- .../beam_LoadTests_Java_CoGBK_Dataflow_Streaming.yml | 2 +- .../beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml | 2 +- .../beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml | 2 +- .../beam_PerformanceTests_BigQueryIO_Streaming_Java.yml | 2 +- .../workflows/beam_PostCommit_BeamMetrics_Publish.yml| 2 +- .github/workflows/beam_PostCommit_Go.yml | 2 +- .github/workflows/beam_PostCommit_Go_Dataflow_ARM.yml| 2 +- .github/workflows/beam_PostCommit_Go_VR_Flink.yml| 2 +- .github/workflows/beam_PostCommit_Go_VR_Samza.yml| 2 +- .github/workflows/beam_PostCommit_Go_VR_Spark.yml| 2 +- .github/workflows/beam_PostCommit_Java.yml | 2 +- .github/workflows/beam_PostCommit_Java_Avro_Versions.yml | 2 +- .github/workflows/beam_PostCommit_Java_DataflowV1.yml| 2 +- .github/workflows/beam_PostCommit_Java_DataflowV2.yml| 2 +- .../beam_PostCommit_Java_Examples_Dataflow_ARM.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_Java.yml | 2 +- .../beam_PostCommit_Java_Examples_Dataflow_V2.yml| 2 +- .../beam_PostCommit_Java_Examples_Dataflow_V2_Java.yml | 2 +- .../workflows/beam_PostCommit_Java_Examples_Direct.yml | 2 +- .../workflows/beam_PostCommit_Java_Examples_Flink.yml| 2 +- .../workflows/beam_PostCommit_Java_Examples_Spark.yml| 2 +- .../workflows/beam_PostCommit_Java_Hadoop_Versions.yml | 2 +- .../beam_PostCommit_Java_IO_Performance_Tests.yml| 4 ++-- .../beam_PostCommit_Java_Jpms_Dataflow_Java11.yml| 2 +- .../beam_PostCommit_Java_Jpms_Dataflow_Java17.yml| 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java11.yml | 2 +- .../beam_PostCommit_Java_Jpms_Direct_Java17.yml | 2 +- .../workflows/beam_PostCommit_Java_Jpms_Flink_Java11.yml | 2 +- .../workflows/beam_PostCommit_Java_Jpms_Spark_Java11.yml | 2 +- .../beam_PostCommit_Java_PVR_Flink_Streaming.yml | 2 +- .github/workflows/beam_PostCommit_Java_PVR_Samza.yml | 2 +- .../beam_PostCommit_Java_PVR_Spark3_Streaming.yml| 2 +- .../workflows/beam_PostCommit_Java_PVR_Spark_Batch.yml | 2 +- .github/workflows/beam_PostCommit_Java_Sickbay.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Dataflow.yml| 2 +- ...Commit_Java_ValidatesRunner_Dataflow_JavaVersions.yml | 2 +- ...ostCommit_Java_ValidatesRunner_Dataflow_Streaming.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.yml | 2 +- ...Commit_Java_ValidatesRunner_Dataflow_V2_Streaming.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Direct.yml | 2 +- ...stCommit_Java_ValidatesRunner_Direct_JavaVersions.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Flink.yml | 2 +- ...beam_PostCommit_Java_ValidatesRunner_Flink_Java11.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Samza.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Spark.yml | 2 +- ...mit_Java_ValidatesRunner_SparkStructuredStreaming.yml | 2 +- ...beam_PostCommit_Java_ValidatesRunner_Spark_Java11.yml | 2 +- .../beam_PostCommit_Java_ValidatesRunner_Twister2.yml| 2 +- .../beam_PostCommit_Java_ValidatesRunner_ULR.yml | 2 +- .github/workflows/beam_PostCommit_PortableJar_Flink.yml | 2 +- .github/workflows/beam_PostCommit_PortableJar_Spark.yml | 2 +- .github/workflows/beam_PostCommit_Python.yml | 2 +- .../beam_PostCommit_Python_Examples_Dataflow.yml | 2 +- .../workflows/beam_PostCommit_Python_Examples_Direct.yml | 2 +- .../workflows/beam_PostCommit_Python_Examples_Flink.yml | 2 +- .../workflows/beam_PostCommit_Python_Examples_Spark.yml | 2 +- .../workflows/beam_PostCommit_Python_MongoDBIO_IT.yml| 2 +- ...eam_PostCommit_Python_ValidatesContainer_Dataflow.yml | 2 +- ...Commit_Python_ValidatesContainer_Dataflow_With_RC.yml | 2 +- .../beam_PostCommit_Python_ValidatesRunner_Dataflow.yml | 2 +- .../beam_PostCommit_Python_ValidatesRunner_Flink.yml | 2 +- .../beam_PostCommit_Python_ValidatesRunner_Samza.yml | 2 +- .../beam_PostCommit_Python_ValidatesRunner_Spark.yml | 2 +- .../beam_PostCommit_Python_Xlang_Gcp_Dataflow.yml| 2 +- .../beam_PostCommit_Python_Xlang_Gcp_Direct.yml | 2 +- .../beam_PostCommit_Python_Xlang_IO_Dataflow.yml | 2 +- .github/workflows/beam_PostCommit_SQL.yml| 2 +- .github/workflows/beam_PostCommit_Sickbay_Python.
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 456286220b6 Updating config from bot
456286220b6 is described below
commit 456286220b6b9280286885792ac01758dee3eff1
Author: github-actions
AuthorDate: Tue Sep 19 14:35:06 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 20d2a7c6d0b..7a8812eee9c 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1695127361892,
-"Abacn": 1695123292421
+"Abacn": 1695134103573
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 0f61a855e97 Updating config from bot
0f61a855e97 is described below
commit 0f61a855e97e73e8b357af5e249d2cb2c34a00c4
Author: github-actions
AuthorDate: Tue Sep 19 14:35:05 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28530.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28530.json
b/scripts/ci/pr-bot/state/pr-state/pr-28530.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28530.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch master updated (34024902746 -> 8f3a38e7d09)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 34024902746 Bump actions/setup-python from 2 to 4 (#28275) add 8f3a38e7d09 kubectl install direct and use only auth plugin from gcloud (#28501) No new revisions were added by this update. Summary of changes: .github/gh-actions-self-hosted-runners/arc/images/Dockerfile | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-)
[beam] branch dependabot/github_actions/actions/setup-python-4 deleted (was 7b06f05becd)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/setup-python-4 in repository https://gitbox.apache.org/repos/asf/beam.git was 7b06f05becd Bump actions/setup-python from 2 to 4 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch dependabot/github_actions/actions/checkout-4 updated (b23f633efc4 -> 19169c21d2c)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/github_actions/actions/checkout-4 in repository https://gitbox.apache.org/repos/asf/beam.git discard b23f633efc4 Bump actions/checkout from 3 to 4 add 49283e31ece Bump archive from 3.3.7 to 3.3.8 in /playground/frontend (#28351) add 79f46e00184 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend (#28350) add 19169c21d2c Bump actions/checkout from 3 to 4 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (b23f633efc4) \ N -- N -- N refs/heads/dependabot/github_actions/actions/checkout-4 (19169c21d2c) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: learning/tour-of-beam/frontend/pubspec.lock | 48 + playground/frontend/pubspec.lock| 48 + 2 files changed, 56 insertions(+), 40 deletions(-)
[beam] branch master updated (79f46e00184 -> 34024902746)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 79f46e00184 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend (#28350) add 34024902746 Bump actions/setup-python from 2 to 4 (#28275) No new revisions were added by this update. Summary of changes: .github/workflows/beam_Release_Python_NightlySnapshot.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[beam] branch dependabot/github_actions/actions/checkout-4 updated (4a9bcfe1d9b -> b23f633efc4)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch
dependabot/github_actions/actions/checkout-4
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 4a9bcfe1d9b Bump actions/checkout from 3 to 4
add d6068ad66b0 Updating Storage API Autosharding documentation to include
that it doesn't work on Runner V2 (#28233)
add 3024ec24006 Better errors when inputs are omitted. (#28289)
add cd65aac2be7 Fix broken link (#28510)
add 6d7c6999a01 Fix building prism in action. (#28509)
add 8871a4e8d2c Remove internal series.append call from frames.py (#28499)
add 71c68ca041d Fix numeric_only logic in frames_test for Pandas 2 (#28422)
add a722740e2cc Support kw only arguments for frame methods for Pandas 2
(#28454)
add d79f537927c Suppress FutureWarnings for Pandas < 2 (#28481)
add 08a97671516 Sets recommended max Xmx 32G when set_recommended_max_xmx
experiment is enabled (#28442)
add 5ffddb82f72 Add PerformanceTests BigQueryIO actions (#28325)
add b23f633efc4 Bump actions/checkout from 3 to 4
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (4a9bcfe1d9b)
\
N -- N -- N
refs/heads/dependabot/github_actions/actions/checkout-4 (b23f633efc4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/README.md| 3 +
...erformanceTests_BigQueryIO_Batch_Java_Avro.yml} | 52 ++---
...erformanceTests_BigQueryIO_Batch_Java_Json.yml} | 52 ++---
...PerformanceTests_BigQueryIO_Streaming_Java.yml} | 52 ++---
.../config_BigQueryIO_Batch_Java_Avro.txt} | 38 -
.../config_BigQueryIO_Batch_Java_Json.txt} | 38 -
.../config_BigQueryIO_Streaming_Java.txt} | 39 +-
sdks/go/test/run_validatesrunner_tests.sh | 8 +-
sdks/java/container/boot.go| 14 +++-
sdks/python/apache_beam/dataframe/frame_base.py| 22 --
.../apache_beam/dataframe/frame_base_test.py | 47 +++-
sdks/python/apache_beam/dataframe/frames.py| 10 ++-
sdks/python/apache_beam/dataframe/frames_test.py | 85 -
sdks/python/apache_beam/yaml/yaml_provider.py | 89 +++---
sdks/python/apache_beam/yaml/yaml_transform.py | 84 +++-
.../apache_beam/yaml/yaml_transform_scope_test.py | 31 +---
.../python/apache_beam/yaml/yaml_transform_test.py | 47
.../apache_beam/yaml/yaml_transform_unit_test.py | 4 +
.../documentation/io/built-in/google-bigquery.md | 4 +
.../en/documentation/sdks/python-dependencies.md | 2 +-
20 files changed, 478 insertions(+), 243 deletions(-)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml} (66%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml} (66%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Streaming_Java.yml} (66%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Batch_Java_Avro.txt} (56%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Batch_Java_Json.txt} (56%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Streaming_Java.txt} (55%)
[beam] branch dependabot/pub/learning/tour-of-beam/frontend/archive-3.3.8 deleted (was 79639afcab6)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pub/learning/tour-of-beam/frontend/archive-3.3.8 in repository https://gitbox.apache.org/repos/asf/beam.git was 79639afcab6 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch dependabot/pub/playground/frontend/archive-3.3.8 deleted (was 18c66ea228a)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pub/playground/frontend/archive-3.3.8 in repository https://gitbox.apache.org/repos/asf/beam.git was 18c66ea228a Bump archive from 3.3.7 to 3.3.8 in /playground/frontend The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[beam] branch master updated (49283e31ece -> 79f46e00184)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 49283e31ece Bump archive from 3.3.7 to 3.3.8 in /playground/frontend (#28351) add 79f46e00184 Bump archive from 3.3.7 to 3.3.8 in /learning/tour-of-beam/frontend (#28350) No new revisions were added by this update. Summary of changes: learning/tour-of-beam/frontend/pubspec.lock | 48 + 1 file changed, 28 insertions(+), 20 deletions(-)
[beam] branch master updated (5ffddb82f72 -> 49283e31ece)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git from 5ffddb82f72 Add PerformanceTests BigQueryIO actions (#28325) add 49283e31ece Bump archive from 3.3.7 to 3.3.8 in /playground/frontend (#28351) No new revisions were added by this update. Summary of changes: playground/frontend/pubspec.lock | 48 +++- 1 file changed, 28 insertions(+), 20 deletions(-)
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch pr-bot-state in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/pr-bot-state by this push: new dd28c14ff13 Updating config from bot dd28c14ff13 is described below commit dd28c14ff1388cac647ab66a362d2a134da700bd Author: github-actions AuthorDate: Tue Sep 19 14:06:25 2023 + Updating config from bot --- scripts/ci/pr-bot/state/pr-state/pr-28325.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28325.json b/scripts/ci/pr-bot/state/pr-state/pr-28325.json index 9c51466b9ee..ebb27d68fb5 100644 --- a/scripts/ci/pr-bot/state/pr-state/pr-28325.json +++ b/scripts/ci/pr-bot/state/pr-state/pr-28325.json @@ -6,5 +6,5 @@ "nextAction": "Reviewers", "stopReviewerNotifications": false, "remindAfterTestsPass": [], - "committerAssigned": false + "committerAssigned": true } \ No newline at end of file
[beam] branch master updated (08a97671516 -> 5ffddb82f72)
This is an automated email from the ASF dual-hosted git repository.
yhu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 08a97671516 Sets recommended max Xmx 32G when set_recommended_max_xmx
experiment is enabled (#28442)
add 5ffddb82f72 Add PerformanceTests BigQueryIO actions (#28325)
No new revisions were added by this update.
Summary of changes:
.github/workflows/README.md| 3 ++
...erformanceTests_BigQueryIO_Batch_Java_Avro.yml} | 52 +++---
...erformanceTests_BigQueryIO_Batch_Java_Json.yml} | 52 +++---
...PerformanceTests_BigQueryIO_Streaming_Java.yml} | 52 +++---
.../config_BigQueryIO_Batch_Java_Avro.txt} | 38 +---
.../config_BigQueryIO_Batch_Java_Json.txt} | 38 +---
.../config_BigQueryIO_Streaming_Java.txt} | 39 +---
7 files changed, 145 insertions(+), 129 deletions(-)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Batch_Java_Avro.yml} (66%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Batch_Java_Json.yml} (66%)
copy .github/workflows/{beam_PostCommit_Java_ValidatesRunner_Flink.yml =>
beam_PerformanceTests_BigQueryIO_Streaming_Java.yml} (66%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Batch_Java_Avro.txt} (56%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Batch_Java_Json.txt} (56%)
copy
.github/workflows/{load-tests-job-configs/config_CoGBK_Java_Streaming_2GB_MultipleKey.txt
=> performance-tests-job-configs/config_BigQueryIO_Streaming_Java.txt} (55%)
[beam] branch users/damccorm/wheels updated: Syntax fix
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/wheels
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/users/damccorm/wheels by this
push:
new ae079980307 Syntax fix
ae079980307 is described below
commit ae0799803075bb01cd59e0641ef5063cb23446ce
Author: Danny McCormick
AuthorDate: Tue Sep 19 10:01:31 2023 -0400
Syntax fix
---
sdks/python/build.gradle | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sdks/python/build.gradle b/sdks/python/build.gradle
index ea4c6c662c0..762bed268d6 100644
--- a/sdks/python/build.gradle
+++ b/sdks/python/build.gradle
@@ -96,7 +96,7 @@ platform_identifiers_map.each { platform, idsuffix ->
exec {
environment CIBW_BUILD: "cp${pyversion}-${idsuffix}"
environment CIBW_ENVIRONMENT: "SETUPTOOLS_USE_DISTUTILS=stdlib"
- environment CIBW_BEFORE_BUILD: "pip install cython==0.29.36 numpy
--config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade
setuptools"
+ environment CIBW_BEFORE_BUILD: "pip install cython==0.29.36 numpy
--config-settings=setup-args='-Dallow-noblas=true' && pip install --upgrade
setuptools"
// note: sync cibuildwheel version with GitHub Action
// .github/workflow/build_wheel.yml:build_wheels "Install
cibuildwheel" step
executable 'sh'
[beam] branch users/damccorm/wheels created (now 47c7f85dbbf)
This is an automated email from the ASF dual-hosted git repository. damccorm pushed a change to branch users/damccorm/wheels in repository https://gitbox.apache.org/repos/asf/beam.git at 47c7f85dbbf Fix numpy install when building wheels This branch includes the following new commits: new 47c7f85dbbf Fix numpy install when building wheels The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[beam] 01/01: Fix numpy install when building wheels
This is an automated email from the ASF dual-hosted git repository.
damccorm pushed a commit to branch users/damccorm/wheels
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 47c7f85dbbf4a907939519e093773ae80be55c98
Author: Danny McCormick
AuthorDate: Tue Sep 19 09:59:19 2023 -0400
Fix numpy install when building wheels
---
.github/workflows/build_wheels.yml | 4 ++--
sdks/python/build.gradle | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/build_wheels.yml
b/.github/workflows/build_wheels.yml
index 1dd58608233..e019e5c261a 100644
--- a/.github/workflows/build_wheels.yml
+++ b/.github/workflows/build_wheels.yml
@@ -269,7 +269,7 @@ jobs:
# TODO: https://github.com/apache/beam/issues/23048
CIBW_SKIP: "*-musllinux_*"
CIBW_ENVIRONMENT: "SETUPTOOLS_USE_DISTUTILS=stdlib"
-CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy && pip install
--upgrade setuptools
+CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy
--config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade
setuptools
run: cibuildwheel --print-build-identifiers && cibuildwheel --output-dir
wheelhouse
shell: bash
- name: install sha512sum on MacOS
@@ -295,7 +295,7 @@ jobs:
# TODO: https://github.com/apache/beam/issues/23048
CIBW_SKIP: "*-musllinux_*"
CIBW_ENVIRONMENT: "SETUPTOOLS_USE_DISTUTILS=stdlib"
-CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy && pip install
--upgrade setuptools
+CIBW_BEFORE_BUILD: pip install cython==0.29.36 numpy
--config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade
setuptools
run: cibuildwheel --print-build-identifiers && cibuildwheel --output-dir
wheelhouse
shell: bash
- name: Add RC checksums
diff --git a/sdks/python/build.gradle b/sdks/python/build.gradle
index 88d0abae83b..ea4c6c662c0 100644
--- a/sdks/python/build.gradle
+++ b/sdks/python/build.gradle
@@ -96,7 +96,7 @@ platform_identifiers_map.each { platform, idsuffix ->
exec {
environment CIBW_BUILD: "cp${pyversion}-${idsuffix}"
environment CIBW_ENVIRONMENT: "SETUPTOOLS_USE_DISTUTILS=stdlib"
- environment CIBW_BEFORE_BUILD: "pip install cython==0.29.36 numpy &&
pip install --upgrade setuptools"
+ environment CIBW_BEFORE_BUILD: "pip install cython==0.29.36 numpy
--config-settings=setup-args="-Dallow-noblas=true" && pip install --upgrade
setuptools"
// note: sync cibuildwheel version with GitHub Action
// .github/workflow/build_wheel.yml:build_wheels "Install
cibuildwheel" step
executable 'sh'
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 4d9d4ce1f20 Updating config from bot
4d9d4ce1f20 is described below
commit 4d9d4ce1f203769aa12675313f76fe28a4ecd55c
Author: github-actions
AuthorDate: Tue Sep 19 12:42:45 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index badfce3c083..20d2a7c6d0b 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1695117975432,
+"damccorm": 1695127361892,
"Abacn": 1695123292421
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 75c0a8bf0f4 Updating config from bot
75c0a8bf0f4 is described below
commit 75c0a8bf0f4c4d4cd7c832553da2b40fe1bb76d7
Author: github-actions
AuthorDate: Tue Sep 19 12:42:43 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28497.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28497.json
b/scripts/ci/pr-bot/state/pr-state/pr-28497.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28497.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new a9a7fbcf743 Updating config from bot
a9a7fbcf743 is described below
commit a9a7fbcf743c131d4455e2321cda94576ba8f847
Author: github-actions
AuthorDate: Tue Sep 19 11:34:55 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 023d8a17bf1..badfce3c083 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -2,6 +2,6 @@
"label": "build",
"dateOfLastReviewAssignment": {
"damccorm": 1695117975432,
-"Abacn": 1695101689246
+"Abacn": 1695123292421
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 606c769d776 Updating config from bot
606c769d776 is described below
commit 606c769d77677b1e08ba4415c3618772170e80fd
Author: github-actions
AuthorDate: Tue Sep 19 11:34:53 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28494.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28494.json
b/scripts/ci/pr-bot/state/pr-state/pr-28494.json
new file mode 100644
index 000..9c51466b9ee
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28494.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "Abacn"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new b766cdef3e2 Updating config from bot
b766cdef3e2 is described below
commit b766cdef3e249eda10ede258e61be7a6d6740949
Author: github-actions
AuthorDate: Tue Sep 19 10:11:57 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28524.json | 8
1 file changed, 8 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28524.json
b/scripts/ci/pr-bot/state/pr-state/pr-28524.json
new file mode 100644
index 000..9c2aa5aa212
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28524.json
@@ -0,0 +1,8 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {},
+ "nextAction": "Author",
+ "stopReviewerNotifications": true,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 1cf8541703a Updating config from bot
1cf8541703a is described below
commit 1cf8541703a4f041b0beba2f8dac08c4ea4f000f
Author: github-actions
AuthorDate: Tue Sep 19 10:06:19 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/reviewers-for-label-build.json | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
index 793dfdd3320..023d8a17bf1 100644
--- a/scripts/ci/pr-bot/state/reviewers-for-label-build.json
+++ b/scripts/ci/pr-bot/state/reviewers-for-label-build.json
@@ -1,7 +1,7 @@
{
"label": "build",
"dateOfLastReviewAssignment": {
-"damccorm": 1695042438148,
+"damccorm": 1695117975432,
"Abacn": 1695101689246
}
}
\ No newline at end of file
[beam] branch pr-bot-state updated: Updating config from bot
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch pr-bot-state
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/pr-bot-state by this push:
new 24bb6a2825d Updating config from bot
24bb6a2825d is described below
commit 24bb6a2825de4c0daf00f796930187d09419acc4
Author: github-actions
AuthorDate: Tue Sep 19 10:06:17 2023 +
Updating config from bot
---
scripts/ci/pr-bot/state/pr-state/pr-28522.json | 10 ++
1 file changed, 10 insertions(+)
diff --git a/scripts/ci/pr-bot/state/pr-state/pr-28522.json
b/scripts/ci/pr-bot/state/pr-state/pr-28522.json
new file mode 100644
index 000..8a92900947e
--- /dev/null
+++ b/scripts/ci/pr-bot/state/pr-state/pr-28522.json
@@ -0,0 +1,10 @@
+{
+ "commentedAboutFailingChecks": false,
+ "reviewersAssignedForLabels": {
+"build": "damccorm"
+ },
+ "nextAction": "Reviewers",
+ "stopReviewerNotifications": false,
+ "remindAfterTestsPass": [],
+ "committerAssigned": false
+}
\ No newline at end of file
