[I] [Feature Request]: [flink] Remove Java Serializaton of CoderTypeSerializer from Flink savepoints [beam]

2024-02-21 Thread via GitHub
je-ik opened a new issue, #30385: URL: https://github.com/apache/beam/issues/30385 ### What would you like to happen? See discussion [on the mailing list][1]. [1]: https://lists.apache.org/thread/s8o8jc2k2kb41q5g0v0xmoyszg1gdcst ### Issue Priority Priority: 3 (nic

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.13.8 to 1.16.4 in /sdks [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30382: URL: https://github.com/apache/beam/pull/30382#issuecomment-1958736265 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.2 in /sdks [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30383: URL: https://github.com/apache/beam/pull/30383#issuecomment-1958736243 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.42.2 to 1.50.3 in /sdks [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30384: URL: https://github.com/apache/beam/pull/30384#issuecomment-1958736220 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.2 in /sdks [beam]

2024-02-21 Thread via GitHub
codecov[bot] commented on PR #30383: URL: https://github.com/apache/beam/pull/30383#issuecomment-1958697187 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/30383?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.42.2 to 1.50.2 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] commented on PR #30366: URL: https://github.com/apache/beam/pull/30366#issuecomment-1958694557 Superseded by #30384. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.42.2 to 1.50.3 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30384: URL: https://github.com/apache/beam/pull/30384 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.42.2 to 1.50.3. Commits https://github.com/aws/aws-sdk-go-v2/commit/7888f0e10eb7ef5db0

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.42.2 to 1.50.2 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] closed pull request #30366: Bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.42.2 to 1.50.2 in /sdks URL: https://github.com/apache/beam/pull/30366 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.1 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] closed pull request #30364: Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.1 in /sdks URL: https://github.com/apache/beam/pull/30364 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.1 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] commented on PR #30364: URL: https://github.com/apache/beam/pull/30364#issuecomment-1958693311 Superseded by #30383. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.26.2 to 1.27.2 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30383: URL: https://github.com/apache/beam/pull/30383 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.26.2 to 1.27.2. Commits https://github.com/aws/aws-sdk-go-v2/commit/7888f0e10eb7ef5db0b67a

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.13.8 to 1.16.3 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] closed pull request #30365: Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.13.8 to 1.16.3 in /sdks URL: https://github.com/apache/beam/pull/30365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.13.8 to 1.16.4 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30382: URL: https://github.com/apache/beam/pull/30382 Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.13.8 to 1.16.4. Commits https://github.com/aws/aws-sdk-go-v2/commit/5f14e2a8eb

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.13.8 to 1.16.3 in /sdks [beam]

2024-02-21 Thread via GitHub
dependabot[bot] commented on PR #30365: URL: https://github.com/apache/beam/pull/30365#issuecomment-1958692309 Superseded by #30382. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Fix updating of annotations for explicitly named transforms. [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30380: URL: https://github.com/apache/beam/pull/30380#issuecomment-1958513954 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @AnandInguva for label python. Available commands: - `st

Re: [PR] [YAML] Add note about running yaml pipelines directly via gcloud. [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30381: URL: https://github.com/apache/beam/pull/30381#issuecomment-1958513883 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @rszper for label website. Available commands: - `stop r

[PR] [YAML] Add note about running yaml pipelines directly via gcloud. [beam]

2024-02-21 Thread via GitHub
robertwb opened a new pull request, #30381: URL: https://github.com/apache/beam/pull/30381 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in yo

[PR] Fix updating of annotations for explicitly named transforms. [beam]

2024-02-21 Thread via GitHub
robertwb opened a new pull request, #30380: URL: https://github.com/apache/beam/pull/30380 It would be good to have a better way to do this, but this is compatible with how things work for now. Thank you for your contribution! Follow this checklist to

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py310 [beam]

2024-02-21 Thread via GitHub
tvalentyn closed pull request #30375: Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py310 URL: https://github.com/apache/beam/pull/30375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py310 [beam]

2024-02-21 Thread via GitHub
dependabot[bot] commented on PR #30375: URL: https://github.com/apache/beam/pull/30375#issuecomment-1958451040 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

Re: [PR] [YAML] Fix MapToFields error output type inference [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30378: URL: https://github.com/apache/beam/pull/30378#issuecomment-1958320919 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

Re: [PR] [YAML] Fix MapToFields error output type inference [beam]

2024-02-21 Thread via GitHub
Polber commented on PR #30378: URL: https://github.com/apache/beam/pull/30378#issuecomment-1958317426 R: @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[PR] [YAML] Fix MapToFields error output type inference [beam]

2024-02-21 Thread via GitHub
Polber opened a new pull request, #30378: URL: https://github.com/apache/beam/pull/30378 MapToFields error_handling output would infer the original payload "element" to be of type `Any` which would break use-cases that redirect error output to a Java transform using Xlang. This PR pa

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498429490 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498426456 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498425150 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498424681 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

Re: [PR] Default max Parallelism should not apply all UnBoundedSource. [beam]

2024-02-21 Thread via GitHub
talatuyarer commented on PR #29792: URL: https://github.com/apache/beam/pull/29792#issuecomment-1958163408 @je-ik Could you review this ? Do you see any issue with this PR ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[I] [Failing Test]: 404 opensource.org/licenses cause java container build fail [beam]

2024-02-21 Thread via GitHub
Abacn opened a new issue, #30377: URL: https://github.com/apache/beam/issues/30377 ### What happened? e.g. https://opensource.org/licenses/cpl1.0.txt no longer exist ### Issue Failure Failure: Test is continually failing ### Issue Priority Priority: 0 (outag

Re: [PR] Grafana flaky test alert [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30344: URL: https://github.com/apache/beam/pull/30344#discussion_r1498385140 ## .test-infra/metrics/grafana/provisioning/alerts/flaky_test.yaml: ## @@ -0,0 +1,149 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] Grafana flaky test alert [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30344: URL: https://github.com/apache/beam/pull/30344#discussion_r1498384039 ## .test-infra/metrics/grafana/provisioning/alerts/flaky_test.yaml: ## @@ -0,0 +1,149 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] Grafana flaky test alert [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30344: URL: https://github.com/apache/beam/pull/30344#discussion_r1498383461 ## .test-infra/metrics/grafana/provisioning/alerts/flaky_test.yaml: ## @@ -0,0 +1,149 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-02-21 Thread via GitHub
m-trieu commented on PR #30312: URL: https://github.com/apache/beam/pull/30312#issuecomment-1958106493 @scwhittle ready for a look! thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] Metrics prefetcher script [beam]

2024-02-21 Thread via GitHub
volatilemolotov commented on code in PR #30327: URL: https://github.com/apache/beam/pull/30327#discussion_r1498368587 ## .test-infra/metrics/grafana/dashboards/GA-Post-Commits_status_dashboard.json: ## @@ -935,7 +935,7 @@ "hide": false, "metricColumn": "non

Re: [PR] start to refactor persistence layer to prepare for direct path [beam]

2024-02-21 Thread via GitHub
m-trieu commented on PR #30265: URL: https://github.com/apache/beam/pull/30265#issuecomment-1958071032 Run Java Precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Add DirectCommitWorkStream for direct path [beam]

2024-02-21 Thread via GitHub
m-trieu commented on PR #30255: URL: https://github.com/apache/beam/pull/30255#issuecomment-1958073827 @scwhittle ready for another look thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] start to refactor persistence layer to prepare for direct path [beam]

2024-02-21 Thread via GitHub
m-trieu commented on PR #30265: URL: https://github.com/apache/beam/pull/30265#issuecomment-1958070719 ready for another look @scwhittle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Support passing credentials from pipeline options into SpannerIO.readChangeStream [beam]

2024-02-21 Thread via GitHub
dengwe1 commented on PR #30361: URL: https://github.com/apache/beam/pull/30361#issuecomment-1958048280 assign set of reviewers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] [YAML] Add split transform. [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30368: URL: https://github.com/apache/beam/pull/30368#issuecomment-1958041627 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

Re: [PR] [YAML] Add split transform. [beam]

2024-02-21 Thread via GitHub
liferoad commented on PR #30368: URL: https://github.com/apache/beam/pull/30368#issuecomment-1958038207 R: @polber -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
damccorm commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498343490 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See t

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py39 [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30373: URL: https://github.com/apache/beam/pull/30373#issuecomment-1958002085 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @riteshghorse for label python. Available commands: - `s

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py311 [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30374: URL: https://github.com/apache/beam/pull/30374#issuecomment-1958001993 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey for label python. Available commands: - `st

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py310 [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30375: URL: https://github.com/apache/beam/pull/30375#issuecomment-1958001903 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @tvalentyn for label python. Available commands: - `stop

Re: [PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py38 [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30376: URL: https://github.com/apache/beam/pull/30376#issuecomment-1958001784 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label python. Available commands: - `stop

Re: [PR] Wire error handling into PubSubIO and add initial tests [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30372: URL: https://github.com/apache/beam/pull/30372#issuecomment-1957930996 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py38 [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30376: URL: https://github.com/apache/beam/pull/30376 Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.3 to 42.0.4. Changelog Sourced from https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst";>cryptography's

[PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py311 [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30374: URL: https://github.com/apache/beam/pull/30374 Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.3 to 42.0.4. Changelog Sourced from https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst";>cryptography's

[PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py310 [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30375: URL: https://github.com/apache/beam/pull/30375 Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.3 to 42.0.4. Changelog Sourced from https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst";>cryptography's

Re: [PR] Support passing credentials from pipeline options into SpannerIO.readChangeStream [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30361: URL: https://github.com/apache/beam/pull/30361#issuecomment-1957887856 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[PR] Bump cryptography from 42.0.3 to 42.0.4 in /sdks/python/container/py39 [beam]

2024-02-21 Thread via GitHub
dependabot[bot] opened a new pull request, #30373: URL: https://github.com/apache/beam/pull/30373 Bumps [cryptography](https://github.com/pyca/cryptography) from 42.0.3 to 42.0.4. Changelog Sourced from https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst";>cryptography's

Re: [PR] Support passing credentials from pipeline options into SpannerIO.readChangeStream [beam]

2024-02-21 Thread via GitHub
dengwe1 commented on PR #30361: URL: https://github.com/apache/beam/pull/30361#issuecomment-1957886057 R: @nancyxu123 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] [Failing Test]: XVR tests running on VR test suites causing test failure [beam]

2024-02-21 Thread via GitHub
Abacn commented on issue #30353: URL: https://github.com/apache/beam/issues/30353#issuecomment-1957844717 It's a configuration problem. XVR test didn't run tests in core (that moved from runners-core-construction) because the change here: https://github.com/apache/beam/pull/29924/files#r149

Re: [I] [Failing Test]: XVR tests running on VR test suites causing test failure [beam]

2024-02-21 Thread via GitHub
kennknowles commented on issue #30353: URL: https://github.com/apache/beam/issues/30353#issuecomment-1957825932 Do we need to roll back? Is it a configuration problem or a functionality problem? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
andreydevyatkin commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1498229338 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

[PR] Wire error handling into PubSubIO and add initial tests [beam]

2024-02-21 Thread via GitHub
johnjcasey opened a new pull request, #30372: URL: https://github.com/apache/beam/pull/30372 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution qui

Re: [PR] add bqio changes to CHANGES.md [beam]

2024-02-21 Thread via GitHub
johnjcasey merged PR #30369: URL: https://github.com/apache/beam/pull/30369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apac

Re: [PR] [YAML] Delete redundant (and now broken) links at bottom of main yaml page. [beam]

2024-02-21 Thread via GitHub
shunping commented on PR #30371: URL: https://github.com/apache/beam/pull/30371#issuecomment-1957770454 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [I] [Bug]: Performance regression in Python pipelines using side inputs in 2.52.0-2.54.0 SDKs [beam]

2024-02-21 Thread via GitHub
tvalentyn closed issue #30360: [Bug]: Performance regression in Python pipelines using side inputs in 2.52.0-2.54.0 SDKs URL: https://github.com/apache/beam/issues/30360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Make state cache optional due to a potential performance regression for some users. [beam]

2024-02-21 Thread via GitHub
tvalentyn merged PR #30362: URL: https://github.com/apache/beam/pull/30362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apach

Re: [PR] Also export AsMultiMap view. [beam]

2024-02-21 Thread via GitHub
tvalentyn merged PR #30370: URL: https://github.com/apache/beam/pull/30370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apach

Re: [PR] [YAML] Delete redundant (and now broken) links at bottom of main yaml page. [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30371: URL: https://github.com/apache/beam/pull/30371#issuecomment-1957766564 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

Re: [PR] [YAML] Delete redundant (and now broken) links at bottom of main yaml page. [beam]

2024-02-21 Thread via GitHub
robertwb commented on PR #30371: URL: https://github.com/apache/beam/pull/30371#issuecomment-1957764804 R: @shunping -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[PR] [YAML] Delete redundant (and now broken) links at bottom of main yaml page. [beam]

2024-02-21 Thread via GitHub
robertwb opened a new pull request, #30371: URL: https://github.com/apache/beam/pull/30371 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in yo

Re: [PR] add bqio changes to CHANGES.md [beam]

2024-02-21 Thread via GitHub
johnjcasey commented on code in PR #30369: URL: https://github.com/apache/beam/pull/30369#discussion_r1498189429 ## CHANGES.md: ## @@ -61,6 +61,7 @@ ## I/Os +* Added support for handling bad records to BigQueryIO ([#30081](https://github.com/apache/beam/pull/30081)). Revi

[PR] Also export AsMultiMap view. [beam]

2024-02-21 Thread via GitHub
tvalentyn opened a new pull request, #30370: URL: https://github.com/apache/beam/pull/30370 https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html#apache_beam.pvalue.AsDict does not mention `AsMultiMap`. I suspect it is because it is not part of `__all__`. -

Re: [PR] add bqio changes to CHANGES.md [beam]

2024-02-21 Thread via GitHub
ahmedabu98 commented on code in PR #30369: URL: https://github.com/apache/beam/pull/30369#discussion_r1498175026 ## CHANGES.md: ## @@ -61,6 +61,7 @@ ## I/Os +* Added support for handling bad records to BigQueryIO ([#30081](https://github.com/apache/beam/pull/30081)). Revi

[PR] add bqio changes to CHANGES.md [beam]

2024-02-21 Thread via GitHub
johnjcasey opened a new pull request, #30369: URL: https://github.com/apache/beam/pull/30369 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution qui

Re: [PR] [YAML] Documentation redirects. [beam]

2024-02-21 Thread via GitHub
robertwb merged PR #30268: URL: https://github.com/apache/beam/pull/30268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Add the ability to test examples from yaml docs. [beam]

2024-02-21 Thread via GitHub
robertwb merged PR #30198: URL: https://github.com/apache/beam/pull/30198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Add a test to read and write GCS with gRPC enabled. [beam]

2024-02-21 Thread via GitHub
shunping commented on PR #30352: URL: https://github.com/apache/beam/pull/30352#issuecomment-1957728409 Run Java_Examples_Dataflow PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Add some tolerance for skippped images in cleaner [beam]

2024-02-21 Thread via GitHub
damccorm merged PR #30294: URL: https://github.com/apache/beam/pull/30294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Add some tolerance for skippped images in cleaner [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30294: URL: https://github.com/apache/beam/pull/30294#issuecomment-1957710913 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498139008 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1075,6 +,42 @@ message StateClearRequest {} // A response to c

Re: [PR] Add some tolerance for skippped images in cleaner [beam]

2024-02-21 Thread via GitHub
damccorm commented on PR #30294: URL: https://github.com/apache/beam/pull/30294#issuecomment-1957707831 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Add some tolerance for skippped images in cleaner [beam]

2024-02-21 Thread via GitHub
damccorm commented on PR #30294: URL: https://github.com/apache/beam/pull/30294#issuecomment-1957706024 > Probably just remove the last check. I think keeping the check is still worthwhile because if there's an issue stale containers could pile up. I went with a slightly different app

Re: [PR] Add a test to read and write GCS with gRPC enabled. [beam]

2024-02-21 Thread via GitHub
shunping commented on code in PR #30352: URL: https://github.com/apache/beam/pull/30352#discussion_r1498133856 ## sdks/java/extensions/google-cloud-platform-core/src/test/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilIT.java: ## @@ -76,4 +86,67 @@ public void testRewriteMu

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
shunping commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498124213 ## sdks/java/harness/src/test/java/org/apache/beam/fn/harness/state/FakeBeamFnStateClient.java: ## @@ -174,6 +215,124 @@ public CompletableFuture handle(StateRequest.B

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498123563 ## sdks/java/harness/src/test/java/org/apache/beam/fn/harness/state/FakeBeamFnStateClient.java: ## @@ -174,6 +215,124 @@ public CompletableFuture handle(StateRequest.B

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498119280 ## sdks/java/harness/src/test/java/org/apache/beam/fn/harness/state/FakeBeamFnStateClient.java: ## @@ -174,6 +215,124 @@ public CompletableFuture handle(StateRequest.B

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
shunping commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498118154 ## sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/FnApiStateAccessor.java: ## @@ -169,6 +172,7 @@ public ResultT get() { } @Override + @Suppr

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498116158 ## sdks/java/harness/src/main/java/org/apache/beam/fn/harness/state/FnApiStateAccessor.java: ## @@ -169,6 +172,7 @@ public ResultT get() { } @Override + @Suppr

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498107366 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1021,6 +1033,29 @@ message StateKey { bytes map_key = 5; }

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-02-21 Thread via GitHub
robertwb commented on code in PR #30317: URL: https://github.com/apache/beam/pull/30317#discussion_r1498106140 ## model/fn-execution/src/main/proto/org/apache/beam/model/fn_execution/v1/beam_fn_api.proto: ## @@ -1075,6 +,42 @@ message StateClearRequest {} // A response to c

Re: [PR] [YAML] Add split transform. [beam]

2024-02-21 Thread via GitHub
github-actions[bot] commented on PR #30368: URL: https://github.com/apache/beam/pull/30368#issuecomment-1957658196 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @liferoad for label python. Available commands: - `stop

Re: [PR] Implementing lull reporting at bundle level processing [beam]

2024-02-21 Thread via GitHub
arvindram03 commented on PR #29882: URL: https://github.com/apache/beam/pull/29882#issuecomment-1957574408 Failures are due to https://opensource.org/licenses/mit planned downtime -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Add TensorflowHub embeddings to MLTransform [beam]

2024-02-21 Thread via GitHub
AnandInguva merged PR #30289: URL: https://github.com/apache/beam/pull/30289 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apa

Re: [PR] Add TensorflowHub embeddings to MLTransform [beam]

2024-02-21 Thread via GitHub
AnandInguva commented on PR #30289: URL: https://github.com/apache/beam/pull/30289#issuecomment-1957479693 Merging this. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[PR] [YAML] Add split transform. [beam]

2024-02-21 Thread via GitHub
robertwb opened a new pull request, #30368: URL: https://github.com/apache/beam/pull/30368 This allows one to split a single PCollection into multiple distinct PCollections based on some property of the elements. It performs the role of (numeric) partition and side outputs from other

Re: [PR] Implementing lull reporting at bundle level processing [beam]

2024-02-21 Thread via GitHub
arvindram03 commented on code in PR #29882: URL: https://github.com/apache/beam/pull/29882#discussion_r1497976473 ## runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/DataflowExecutionStateTrackerTest.java: ## @@ -123,12 +157,112 @@

Re: [PR] [DRAFT] Duet AI documentation lookup prompts for Beam ML [beam]

2024-02-21 Thread via GitHub
dariabezkorovaina commented on code in PR #30351: URL: https://github.com/apache/beam/pull/30351#discussion_r1497801704 ## learning/prompts/documentation-lookup/33_ai_ml_pipelines.md: ## @@ -0,0 +1,20 @@ +Prompt: +How can I use Apache Beam in AI/ML pipeline? +Response: +You can

Re: [PR] Bump timeout of PostCommit Java PVR Spark3 Streaming to 3h [beam]

2024-02-21 Thread via GitHub
damccorm merged PR #30356: URL: https://github.com/apache/beam/pull/30356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Grafana flaky test alert [beam]

2024-02-21 Thread via GitHub
damccorm commented on code in PR #30344: URL: https://github.com/apache/beam/pull/30344#discussion_r1497920735 ## .test-infra/metrics/grafana/provisioning/alerts/flaky_test.yaml: ## @@ -0,0 +1,149 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contri

Re: [PR] Flaky test issue script [beam]

2024-02-21 Thread via GitHub
damccorm commented on code in PR #30345: URL: https://github.com/apache/beam/pull/30345#discussion_r1497848395 ## .github/flaky_test_detection.py: ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See t

Re: [I] [Failing Test]: XVR tests running on VR test suites causing test failure [beam]

2024-02-21 Thread via GitHub
Abacn closed issue #30353: [Failing Test]: XVR tests running on VR test suites causing test failure URL: https://github.com/apache/beam/issues/30353 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Fix XVR test exclusions [beam]

2024-02-21 Thread via GitHub
Abacn merged PR #30354: URL: https://github.com/apache/beam/pull/30354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

Re: [PR] Fix XVR test exclusions [beam]

2024-02-21 Thread via GitHub
Abacn commented on PR #30354: URL: https://github.com/apache/beam/pull/30354#issuecomment-1957135408 Let me merge this to get the postcommits back work then investigating unrelated test failures -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Fix XVR test exclusions [beam]

2024-02-21 Thread via GitHub
Abacn commented on PR #30354: URL: https://github.com/apache/beam/pull/30354#issuecomment-1957124739 Failed tests: - beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming - run1: org.apache.beam.sdk.transforms.CombineTest$CombineWithContextTests > testHotKeyCombineWithSid

Re: [PR] Metrics prefetcher script [beam]

2024-02-21 Thread via GitHub
damccorm commented on code in PR #30327: URL: https://github.com/apache/beam/pull/30327#discussion_r1497738540 ## .test-infra/metrics/grafana/dashboards/GA-Post-Commits_status_dashboard.json: ## @@ -935,7 +935,7 @@ "hide": false, "metricColumn": "none",

Re: [PR] Fix XVR test exclusions [beam]

2024-02-21 Thread via GitHub
chamikaramj commented on code in PR #30354: URL: https://github.com/apache/beam/pull/30354#discussion_r1497842314 ## runners/google-cloud-dataflow-java/build.gradle: ## @@ -192,6 +191,7 @@ def commonLegacyExcludeCategories = [ ] def commonRunnerV2ExcludeCategories = [ +

  1   2   >