Re: [PR] add a way for channels to be closed manually [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30425: URL: https://github.com/apache/beam/pull/30425#discussion_r1515691626 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcDispatcherClient.java: ## @@ -97,6 +97,14 @@

Re: [PR] add a way for channels to be closed manually [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30425: URL: https://github.com/apache/beam/pull/30425#discussion_r1515690054 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/stubs/ChannelCache.java: ## @@ -0,0 +1,120 @@ +/*

Re: [PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.29.1 in /sdks [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30557: URL: https://github.com/apache/beam/pull/30557#issuecomment-1982403037 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[PR] Bump github.com/testcontainers/testcontainers-go from 0.26.0 to 0.29.1 in /sdks [beam]

2024-03-06 Thread via GitHub
dependabot[bot] opened a new pull request, #30557: URL: https://github.com/apache/beam/pull/30557 Bumps [github.com/testcontainers/testcontainers-go](https://github.com/testcontainers/testcontainers-go) from 0.26.0 to 0.29.1. Release notes Sourced from https://github.com/testconta

Re: [PR] add a way for channels to be closed manually [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30425: URL: https://github.com/apache/beam/pull/30425#discussion_r1515523277 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/stubs/ChannelCache.java: ## @@ -0,0 +1,120 @@ +/*

Re: [PR] Jimmytobin2425 patch 1 [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30550: URL: https://github.com/apache/beam/pull/30550#issuecomment-1982330769 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @liferoad for label python. Available commands: - `stop

[PR] try new protobuf [beam]

2024-03-06 Thread via GitHub
riteshghorse opened a new pull request, #30556: URL: https://github.com/apache/beam/pull/30556 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution q

Re: [PR] [RRIO] Begin adding RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30555: URL: https://github.com/apache/beam/pull/30555#issuecomment-1982288013 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label website. Available commands: - `stop

Re: [PR] [RRIO] Stage WebApis examples module [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30554: URL: https://github.com/apache/beam/pull/30554#issuecomment-1982252454 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Add vertex AI dependency [beam]

2024-03-06 Thread via GitHub
damondouglas merged PR #30553: URL: https://github.com/apache/beam/pull/30553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.ap

[PR] [RRIO] Begin adding RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas opened a new pull request, #30555: URL: https://github.com/apache/beam/pull/30555 This PR addresses #30379 by adding the beginnings of RequestResponseIO examples and documentation on website for the Java SDK. Thank you for your contribution! Fo

Re: [PR] Add vertex AI dependency [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30553: URL: https://github.com/apache/beam/pull/30553#issuecomment-1982228545 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[PR] [RRIO] Stage WebApis examples module [beam]

2024-03-06 Thread via GitHub
damondouglas opened a new pull request, #30554: URL: https://github.com/apache/beam/pull/30554 This PR addresses #30379 staging for examples that make WebApis requests pulling images using RequestResponseIO. Thank you for your contribution! Follow this

Re: [PR] [BEAM-30531] Automatically execute unbounded pipelines in streaming mode. [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30533: URL: https://github.com/apache/beam/pull/30533#issuecomment-198873 waiting on author -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Do not pre-install beam in tensorrt container [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30552: URL: https://github.com/apache/beam/pull/30552#issuecomment-1982216770 Run Python PreCommit 3.8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] [TESTING] Update protobuf timestamp limit for protobuf 4.26 [beam]

2024-03-06 Thread via GitHub
riteshghorse commented on PR #29873: URL: https://github.com/apache/beam/pull/29873#issuecomment-1982216462 Closing this in favor of adding our own wrapper for the timestamp method without checks in short-term . -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] [TESTING] Update protobuf timestamp limit for protobuf 4.26 [beam]

2024-03-06 Thread via GitHub
riteshghorse closed pull request #29873: [TESTING] Update protobuf timestamp limit for protobuf 4.26 URL: https://github.com/apache/beam/pull/29873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] Add vertex AI dependency [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30553: URL: https://github.com/apache/beam/pull/30553#issuecomment-1982206061 cc: @kennknowles -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Do not pre-install beam in tensorrt container [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30552: URL: https://github.com/apache/beam/pull/30552#issuecomment-1982205371 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Add vertex AI dependency [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30553: URL: https://github.com/apache/beam/pull/30553#issuecomment-1982205023 Actually, do we need to add Vertex AI as a Beam dependency if we don't directly depend on it? can the dependency be only added for the project that defines the relevant example? -- T

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30312: URL: https://github.com/apache/beam/pull/30312#discussion_r1515381512 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/commits/StreamingEngineWorkCommitter.java: ## @@ -0,0

[PR] Add vertex AI dependency [beam]

2024-03-06 Thread via GitHub
damondouglas opened a new pull request, #30553: URL: https://github.com/apache/beam/pull/30553 This PR addresses #30379 by adding the Vertex AI dependency. This is needed for future example. Thank you for your contribution! Follow this checklist to help

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30312: URL: https://github.com/apache/beam/pull/30312#discussion_r1515369962 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/commits/StreamingEngineWorkCommitter.java: ## @@ -0,0

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-03-06 Thread via GitHub
m-trieu commented on code in PR #30312: URL: https://github.com/apache/beam/pull/30312#discussion_r1515369507 ## runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/commits/StreamingEngineWorkCommitterTest.java: ## @@ -

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas closed pull request #30430: [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK URL: https://github.com/apache/beam/pull/30430 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas commented on PR #30430: URL: https://github.com/apache/beam/pull/30430#issuecomment-1982182212 I'll break up the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-1982175603 alternative fix at #30552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] Do not pre-install beam in tensorrt container [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30552: URL: https://github.com/apache/beam/pull/30552#issuecomment-1982174944 The change already applied to gcr. WIll wait to see if Python PostCommit passes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[PR] Do not pre-install beam in tensorrt container [beam]

2024-03-06 Thread via GitHub
Abacn opened a new pull request, #30552: URL: https://github.com/apache/beam/pull/30552 Fix Python PostCommit tensorrttest **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help

Re: [PR] Fixes an upgrade imcompatiblity of BQ read/write transforms [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30551: URL: https://github.com/apache/beam/pull/30551#issuecomment-1982163394 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [PR] Duet AI prompts: containers, hints, external calls (no links) [beam]

2024-03-06 Thread via GitHub
olehborysevych commented on code in PR #30435: URL: https://github.com/apache/beam/pull/30435#discussion_r1515349817 ## learning/prompts/documentation-lookup-nolinks/47_batching_for_external_calls.md: ## @@ -0,0 +1,70 @@ +Prompt: +How to reduce payload when calling external serv

Re: [PR] Reduce nullness checks in flink adapters. [beam]

2024-03-06 Thread via GitHub
robertwb merged PR #30488: URL: https://github.com/apache/beam/pull/30488 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Reduce nullness checks in flink adapters. [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30488: URL: https://github.com/apache/beam/pull/30488#issuecomment-1982086997 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @lostluck added as fallback since no labels match configuration

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-1982075343 rebuilding the image sounds like a more appropriate fix here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-1982074688 I'd rather keep it as is unless Beam requires some functionality from a newer version. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [I] [Bug]: Upgrade compatibility broken when upgrading BQ read/write transforms to 2.55.0.dev [beam]

2024-03-06 Thread via GitHub
chamikaramj commented on issue #30534: URL: https://github.com/apache/beam/issues/30534#issuecomment-1982068658 Should be fixed by https://github.com/apache/beam/issues/30534. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[PR] Fixes an upgrade imcompatiblity of BQ read/write transforms [beam]

2024-03-06 Thread via GitHub
chamikaramj opened a new pull request, #30551: URL: https://github.com/apache/beam/pull/30551 This fixes https://github.com/apache/beam/issues/30534. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly

Re: [PR] Reduce nullness checks in flink adapters. [beam]

2024-03-06 Thread via GitHub
robertwb commented on code in PR #30488: URL: https://github.com/apache/beam/pull/30488#discussion_r1515270765 ## runners/flink/src/main/java/org/apache/beam/runners/flink/adapter/BeamFlinkDataSetAdapter.java: ## @@ -213,7 +216,8 @@ private FlinkBatchPortablePipelineTranslator

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-1982001500 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-198284 R: @tvalentyn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Reduce nullness checks in flink adapters. [beam]

2024-03-06 Thread via GitHub
robertwb commented on code in PR #30488: URL: https://github.com/apache/beam/pull/30488#discussion_r1515262267 ## runners/flink/src/main/java/org/apache/beam/runners/flink/adapter/FlinkInput.java: ## @@ -71,7 +71,7 @@ public String getUrn() { } @Override -@Suppre

Re: [PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30548: URL: https://github.com/apache/beam/pull/30548#issuecomment-1981998857 tensorrt test passed: https://github.com/apache/beam/actions/runs/8179296900/job/22365079715?pr=30548 Revert test only change and mark as ready for review -- This is an automated me

[PR] Jimmytobin2425 patch 1 [beam]

2024-03-06 Thread via GitHub
jimmytobin2425 opened a new pull request, #30550: URL: https://github.com/apache/beam/pull/30550 The issue this warning message points to (https://github.com/apache/beam/issues/22969) has been resolved showing it was an issue with Python 2 and not with beam. I propose to remove this warning

Re: [PR] Remove warning for mixing yield and return in a DoFn [beam]

2024-03-06 Thread via GitHub
jimmytobin2425 closed pull request #30549: Remove warning for mixing yield and return in a DoFn URL: https://github.com/apache/beam/pull/30549 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] [BEAM-30531] Automatically execute unbounded pipelines in streaming mode. [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30533: URL: https://github.com/apache/beam/pull/30533#issuecomment-1981972184 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @riteshghorse for label python. Available commands: - `s

Re: [PR] Fix internal test failure from PR-30455 [beam]

2024-03-06 Thread via GitHub
Abacn commented on code in PR #30546: URL: https://github.com/apache/beam/pull/30546#discussion_r1515232397 ## sdks/python/apache_beam/io/gcp/bigquery_tools_test.py: ## @@ -224,9 +224,15 @@ def test_delete_dataset_retries_for_timeouts(self, patched_time_sleep): self.assert

Re: [PR] Fix internal test failure from PR-30455 [beam]

2024-03-06 Thread via GitHub
Abacn commented on code in PR #30546: URL: https://github.com/apache/beam/pull/30546#discussion_r1515232397 ## sdks/python/apache_beam/io/gcp/bigquery_tools_test.py: ## @@ -224,9 +224,15 @@ def test_delete_dataset_retries_for_timeouts(self, patched_time_sleep): self.assert

Re: [I] [Bug]: Upgrade compatibility broken when upgrading BQ read/write transforms to 2.55.0.dev [beam]

2024-03-06 Thread via GitHub
chamikaramj commented on issue #30534: URL: https://github.com/apache/beam/issues/30534#issuecomment-1981934491 I'm working on a fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] ActiveWorkRefresh [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30390: URL: https://github.com/apache/beam/pull/30390#issuecomment-1981922464 There is a flaky test added: testInvalidateStuckCommits: https://github.com/apache/beam/runs/22276370706 ``` Wanted but not invoked: forComputation.invalidate( ,

Re: [I] [Failing Test]: dataflow runner worker project test stuck causing Java PreCommit time out [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #28957: URL: https://github.com/apache/beam/issues/28957#issuecomment-1981917789 Other flaky test: testLatencyAttributionToQueuedState: https://github.com/apache/beam/runs/22270690743 ``` java.lang.AssertionError: expected: but was: at org.ju

Re: [PR] Add KafkaIO Stress test [beam]

2024-03-06 Thread via GitHub
Abacn merged PR #30467: URL: https://github.com/apache/beam/pull/30467 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

[PR] Remove warning for mixing yield and return in a DoFn [beam]

2024-03-06 Thread via GitHub
jimmytobin2425 opened a new pull request, #30549: URL: https://github.com/apache/beam/pull/30549 The issue this warning message points to (https://github.com/apache/beam/issues/22969) has been resolved showing it was an issue with Python 2 and not with beam. I propose to remove this warning

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damccorm commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515217941 ## website/www/site/content/en/documentation/io/built-in/webapis.md: ## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- + + +[Built-in I/O Transforms](/doc

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damccorm commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515216455 ## website/www/site/content/en/documentation/io/built-in/webapis.md: ## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- + + +[Built-in I/O Transforms](/doc

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damccorm commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515215407 ## examples/java/webapis/src/main/java/org/apache/beam/examples/webapis/UsingHttpClientExample.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundat

Re: [I] [Bug]: Upgrade compatibility broken when upgrading BQ read/write transforms to 2.55.0.dev [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #30534: URL: https://github.com/apache/beam/issues/30534#issuecomment-1981879397 What is the plan as for 2.55.0 release ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[PR] Bump google auth lower bound [beam]

2024-03-06 Thread via GitHub
Abacn opened a new pull request, #30548: URL: https://github.com/apache/beam/pull/30548 Fix tensorRT test in Python PostCommit. Caused by https://github.com/googleapis/google-cloud-python/issues/12254 but only surfaced to Beam until March 4th, likely transient dependency upgrade **Pl

Re: [I] [Failing Test]: Python PostCommit failing hdfsIntegrationTest in generate_external_transform_wrappers [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #30459: URL: https://github.com/apache/beam/issues/30459#issuecomment-1981848636 There is another new failure: ``` :sdks:python:test-suites:dataflow:py38:tensorRTtests ``` ``` ERROR:apache_beam.runners.dataflow.dataflow_runner:: JOB_MESSAGE_ERR

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515187165 ## website/www/site/content/en/documentation/io/built-in/webapis.md: ## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- + + +[Built-in I/O Transforms](

Re: [PR] [BEAM-30531] Automatically execute unbounded pipelines in streaming mode. [beam]

2024-03-06 Thread via GitHub
robertwb commented on PR #30533: URL: https://github.com/apache/beam/pull/30533#issuecomment-1981829484 Yes, happy to let this change bake. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Fix internal test failure from PR-30455 [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30546: URL: https://github.com/apache/beam/pull/30546#issuecomment-1981828119 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @AnandInguva for label python. R: @ahmedabu98 for label io.

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515170769 ## website/www/site/content/en/documentation/io/built-in/webapis.md: ## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- + + +[Built-in I/O Transforms](

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515178887 ## examples/java/webapis/src/main/java/org/apache/beam/examples/webapis/UsingHttpClientExample.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Fou

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damondouglas commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515170769 ## website/www/site/content/en/documentation/io/built-in/webapis.md: ## @@ -0,0 +1,441 @@ +--- +title: "Web Apis I/O connector" +--- + + +[Built-in I/O Transforms](

Re: [PR] Bump github.com/cloudevents/sdk-go/v2 from 2.6.1 to 2.15.2 in /playground/backend [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30547: URL: https://github.com/apache/beam/pull/30547#issuecomment-1981785135 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [I] [Failing Test]: beam_PostCommit_XVR_Direct perma-red. [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #28972: URL: https://github.com/apache/beam/issues/28972#issuecomment-1981769577 For some reason the window in WindowedValue decoded here https://github.com/apache/beam/blob/1a05f39883fca49f8b8068a68a358dfe973055c0/sdks/python/apache_beam/runners/portability/f

Re: [PR] [RRIO]: Add RequestResponseIO examples and documentation on website for the Java SDK [beam]

2024-03-06 Thread via GitHub
damccorm commented on code in PR #30430: URL: https://github.com/apache/beam/pull/30430#discussion_r1515134717 ## examples/java/webapis/src/main/java/org/apache/beam/examples/webapis/UsingHttpClientExample.java: ## @@ -0,0 +1,68 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-03-06 Thread via GitHub
shunping commented on PR #30317: URL: https://github.com/apache/beam/pull/30317#issuecomment-1981758955 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] fix playground backend container builds [beam]

2024-03-06 Thread via GitHub
damccorm merged PR #30497: URL: https://github.com/apache/beam/pull/30497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] fix playground backend container builds [beam]

2024-03-06 Thread via GitHub
damccorm commented on PR #30497: URL: https://github.com/apache/beam/pull/30497#issuecomment-1981719597 Agreed this is unrelated to precommit failure (looks like a go lint issue) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] Bump github.com/cloudevents/sdk-go/v2 from 2.6.1 to 2.15.2 in /playground/backend [beam]

2024-03-06 Thread via GitHub
dependabot[bot] opened a new pull request, #30547: URL: https://github.com/apache/beam/pull/30547 Bumps [github.com/cloudevents/sdk-go/v2](https://github.com/cloudevents/sdk-go) from 2.6.1 to 2.15.2. Release notes Sourced from https://github.com/cloudevents/sdk-go/releases";>githu

Re: [I] [Failing Test]: beam_PostCommit_XVR_Direct perma-red. [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #28972: URL: https://github.com/apache/beam/issues/28972#issuecomment-1981692067 CC: @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Allow local runners to execute arbitrary cross language pipelines without Docker. [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #29283: URL: https://github.com/apache/beam/pull/29283#issuecomment-1981680333 It appears Python XVR Direct is failing after this change. Specifically https://github.com/apache/beam/issues/28972#issuecomment-1930538626 any ideas? -- This is an automated message from

[PR] Fix pr 30455 [beam]

2024-03-06 Thread via GitHub
shunping opened a new pull request, #30546: URL: https://github.com/apache/beam/pull/30546 PR #30455 has caused some internal test failure. We are going to skip the test if BigQueryWrapper cannot be imported. (Internal bug id: 302004313) Thank yo

Re: [I] [Failing Test]: beam_PostCommit_XVR_Direct perma-red. [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #28972: URL: https://github.com/apache/beam/issues/28972#issuecomment-1981654497 This is a regression in Beam 2.53.0. Unfortunately GHA logs expires in 3 months. From now one only knows the regression happens between Nov 8, 2023 (last successful run and https://git

Re: [PR] fix playground backend container builds [beam]

2024-03-06 Thread via GitHub
volatilemolotov commented on PR #30497: URL: https://github.com/apache/beam/pull/30497#issuecomment-1981623256 No, seems like a test failure, that part of code was not touched -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Document requirements cache behavior differences. [beam]

2024-03-06 Thread via GitHub
tvalentyn commented on PR #30493: URL: https://github.com/apache/beam/pull/30493#issuecomment-1981593752 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Document requirements cache behavior differences. [beam]

2024-03-06 Thread via GitHub
rszper commented on code in PR #30493: URL: https://github.com/apache/beam/pull/30493#discussion_r1514988361 ## website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md: ## @@ -46,11 +46,21 @@ To supply a requirements.txt file: The runner will use th

Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]

2024-03-06 Thread via GitHub
riteshghorse commented on issue #25590: URL: https://github.com/apache/beam/issues/25590#issuecomment-1981553174 +1. I'm working with @tvalentyn on workaround for that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] Add Redistribute transform to Java SDK and Dataflow translation [beam]

2024-03-06 Thread via GitHub
kennknowles commented on PR #30545: URL: https://github.com/apache/beam/pull/30545#issuecomment-1981548748 One question which wasn't in the design docs was how to implement: wrap Reshuffle (aka build a composite that just invokes Reshuffle and relies on everything built around it) or fork.

Re: [PR] [Python] Vertex AI Feature Store enrichment handler [beam]

2024-03-06 Thread via GitHub
riteshghorse merged PR #30388: URL: https://github.com/apache/beam/pull/30388 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.ap

Re: [PR] [Python] Vertex AI Feature Store enrichment handler [beam]

2024-03-06 Thread via GitHub
riteshghorse commented on PR #30388: URL: https://github.com/apache/beam/pull/30388#issuecomment-1981522022 Unrelated failure/flake. Merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Fix hdfs integration test [beam]

2024-03-06 Thread via GitHub
Abacn merged PR #30458: URL: https://github.com/apache/beam/pull/30458 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

Re: [I] [Failing Test]: Python PostCommit failing hdfsIntegrationTest in generate_external_transform_wrappers [beam]

2024-03-06 Thread via GitHub
Abacn closed issue #30459: [Failing Test]: Python PostCommit failing hdfsIntegrationTest in generate_external_transform_wrappers URL: https://github.com/apache/beam/issues/30459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Fix hdfs integration test [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30458: URL: https://github.com/apache/beam/pull/30458#issuecomment-1981509771 Python Test need rebase to pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Add KafkaIO Stress test [beam]

2024-03-06 Thread via GitHub
Abacn commented on code in PR #30467: URL: https://github.com/apache/beam/pull/30467#discussion_r1514948281 ## it/kafka/src/test/java/org/apache/beam/it/kafka/KafkaIOST.java: ## @@ -0,0 +1,480 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] Implement ordered list state for FnApi. [beam]

2024-03-06 Thread via GitHub
github-actions[bot] commented on PR #30317: URL: https://github.com/apache/beam/pull/30317#issuecomment-1981431117 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @robertwb for label java. Available commands: - `stop re

Re: [I] [Bug]: Go SDK Dataflow jobs fail on DataSampling disabled [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #29760: URL: https://github.com/apache/beam/issues/29760#issuecomment-1981419452 It seems this issue was resolved? The linked workflow run https://github.com/apache/beam/issues/29760#issuecomment-1854793020 was successful -- This is an automated message from the

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-03-06 Thread via GitHub
scwhittle commented on PR #30312: URL: https://github.com/apache/beam/pull/30312#issuecomment-1981420874 There are still some open comments. This is part of work to support direct path which won't be ready for the cut. So I don't think this should hold up the cut. -- This is an automate

Re: [I] [Bug]: Go SDK Dataflow jobs fail on DataSampling disabled [beam]

2024-03-06 Thread via GitHub
Abacn closed issue #29760: [Bug]: Go SDK Dataflow jobs fail on DataSampling disabled URL: https://github.com/apache/beam/issues/29760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] [Task]: Update the minor version of protobuf library in the upper bound prior to Beam release. [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #25590: URL: https://github.com/apache/beam/issues/25590#issuecomment-1981406944 4.25.3 is the latest 4.x release as of 2.55.0 cut day. and it is `<4.26.0`. Next release needs to aware that the upcoming release bumped the major version, 4.25 -> 5.26, indicating brea

Re: [I] [Task]: Update the minor version of cloudpickle library prior to Beam release. [beam]

2024-03-06 Thread via GitHub
Abacn commented on issue #23119: URL: https://github.com/apache/beam/issues/23119#issuecomment-1981400099 2.2.1 is still the latest of 2.x as of 2.55.0 cut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Refactor commit logic out of StreamingDataflowWorker [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30312: URL: https://github.com/apache/beam/pull/30312#issuecomment-1981396920 fyi today is release cut. What is the status of this PR? I see the comments are all replied? CC: @scwhittle @m-trieu -- This is an automated message from the Apache Git Service. To respon

Re: [PR] fix playground backend container builds [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30497: URL: https://github.com/apache/beam/pull/30497#issuecomment-1981390739 Is Playground PreCommit failure related to this change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Bigtable: use InstanceBuilder to dynamic load override class [beam]

2024-03-06 Thread via GitHub
Abacn merged PR #30542: URL: https://github.com/apache/beam/pull/30542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

Re: [PR] Revert "Document yaml pipeline options" [beam]

2024-03-06 Thread via GitHub
Abacn closed pull request #30536: Revert "Document yaml pipeline options" URL: https://github.com/apache/beam/pull/30536 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Fix hdfs integration test [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30458: URL: https://github.com/apache/beam/pull/30458#issuecomment-1981375814 R: @riteshghorse -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] Fix hdfs integration test [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30458: URL: https://github.com/apache/beam/pull/30458#issuecomment-1981375569 both hdfsIntegration test and azureIntegrationTest passed, though there are new failures due to #30417 and reverted in #30535 -- This is an automated message from the Apache Git Service. T

Re: [PR] Revert "Add test code to overwrite SQL in Beam Python JDBC (#30417)" [beam]

2024-03-06 Thread via GitHub
Abacn commented on PR #30535: URL: https://github.com/apache/beam/pull/30535#issuecomment-1981370763 PreCommit Python failed unrelated change and fixed by #30540, merging for now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Revert "Add test code to overwrite SQL in Beam Python JDBC (#30417)" [beam]

2024-03-06 Thread via GitHub
Abacn merged PR #30535: URL: https://github.com/apache/beam/pull/30535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

  1   2   >