Re: [PR] Update pypi documentation 30145 [beam]

2025-03-24 Thread via GitHub
rakeshcusat commented on code in PR #34329: URL: https://github.com/apache/beam/pull/34329#discussion_r2011365432 ## sdks/python/README.md: ## @@ -0,0 +1,90 @@ +# Apache Beam + +[Apache Beam](http://beam.apache.org/) is a unified model for defining both batch and streaming data

Re: [PR] Update pypi documentation 30145 [beam]

2025-03-24 Thread via GitHub
rakeshcusat commented on code in PR #34329: URL: https://github.com/apache/beam/pull/34329#discussion_r2011364932 ## sdks/python/README.md: ## @@ -0,0 +1,90 @@ +# Apache Beam + Review Comment: Sure thing, I will add the details here. ## sdks/python/README.md:

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.10 in /sdks [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34415: URL: https://github.com/apache/beam/pull/34415#issuecomment-2750095693 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @lostluck for label go. Available commands: - `stop revi

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.67 in /sdks [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34416: URL: https://github.com/apache/beam/pull/34416#issuecomment-2750095637 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey for label go. Available commands: - `stop r

Re: [I] [Bug]: javaJar provider does not work with --yaml_pipeline flag: TypeError: a bytes-like object is required, not 'str' [beam]

2025-03-24 Thread via GitHub
robertwb commented on issue #34343: URL: https://github.com/apache/beam/issues/34343#issuecomment-2737931955 Thanks for catching this! Alternative fix at https://github.com/apache/beam/pull/34351 (turns out the bytes object was coming from `urllib.parse.urlparse`). We'll try to get this in

[PR] Enforce ordering to avoid svn conflicts [beam]

2025-03-24 Thread via GitHub
damccorm opened a new pull request, #34405: URL: https://github.com/apache/beam/pull/34405 Relies on https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/using-jobs-in-a-workflow#defining-prerequisite-jobs Since we have if conditions for these defined,

Re: [PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
damccorm commented on code in PR #34404: URL: https://github.com/apache/beam/pull/34404#discussion_r2010877814 ## sdks/python/apache_beam/yaml/yaml_provider.py: ## @@ -661,16 +661,20 @@ def create_transform(self, type, args, yaml_create_transform): return self._transform_f

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.10 in /sdks [beam]

2025-03-24 Thread via GitHub
codecov[bot] commented on PR #34415: URL: https://github.com/apache/beam/pull/34415#issuecomment-2750057072 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/34415?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

Re: [PR] track GetWork from WindmillWorker to UserWorker in proxyless path [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34413: URL: https://github.com/apache/beam/pull/34413#issuecomment-2750057341 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey added as fallback since no labels match configuration

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.67 in /sdks [beam]

2025-03-24 Thread via GitHub
codecov[bot] commented on PR #34416: URL: https://github.com/apache/beam/pull/34416#issuecomment-2750057167 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/34416?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term

[PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.67 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] opened a new pull request, #34416: URL: https://github.com/apache/beam/pull/34416 Bumps [github.com/aws/aws-sdk-go-v2/feature/s3/manager](https://github.com/aws/aws-sdk-go-v2) from 1.17.62 to 1.17.67. Commits https://github.com/aws/aws-sdk-go-v2/commit/1572db05

Re: [PR] Use list instead of set to make type hint serialization deterministic [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34414: URL: https://github.com/apache/beam/pull/34414#issuecomment-2750035330 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewer

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.66 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] closed pull request #34259: Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.66 in /sdks URL: https://github.com/apache/beam/pull/34259 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager from 1.17.62 to 1.17.66 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] commented on PR #34259: URL: https://github.com/apache/beam/pull/34259#issuecomment-2750035618 Superseded by #34416. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.9 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] commented on PR #34357: URL: https://github.com/apache/beam/pull/34357#issuecomment-2750034127 Superseded by #34415. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.10 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] opened a new pull request, #34415: URL: https://github.com/apache/beam/pull/34415 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.29.6 to 1.29.10. Commits https://github.com/aws/aws-sdk-go-v2/commit/1572db05596936d5bb6d7

Re: [PR] Use list instead of set to make type hint serialization deterministic [beam]

2025-03-24 Thread via GitHub
AdrS commented on PR #34414: URL: https://github.com/apache/beam/pull/34414#issuecomment-2750033618 R: @robertwb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

Re: [PR] Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.9 in /sdks [beam]

2025-03-24 Thread via GitHub
dependabot[bot] closed pull request #34357: Bump github.com/aws/aws-sdk-go-v2/config from 1.29.6 to 1.29.9 in /sdks URL: https://github.com/apache/beam/pull/34357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[PR] Use list instead of set to make type hint serialization deterministic [beam]

2025-03-24 Thread via GitHub
AdrS opened a new pull request, #34414: URL: https://github.com/apache/beam/pull/34414 See #34410 for context. Testing: - Unit tests: `pytest -v apache_beam/typehints/*_test.py` - I patched the change into the Google monorepo and ran a global presubmit -- This is an automated

Re: [PR] [Dataflow Streaming] Optimize serialization of small commits [beam]

2025-03-24 Thread via GitHub
scwhittle commented on code in PR #34355: URL: https://github.com/apache/beam/pull/34355#discussion_r2005190991 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/stubs/CloudWindmillServiceV1Alpha1CustomStub.java:

[PR] Add warning comment so that folks don't change this [beam]

2025-03-24 Thread via GitHub
damccorm opened a new pull request, #34409: URL: https://github.com/apache/beam/pull/34409 Follow up to https://github.com/apache/beam/pull/34404/files#r2010875137 Thank you for your contribution! Follow this checklist to help us incorporate your contri

[PR] track GetWork from WindmillWorker to UserWorker in proxyless path [beam]

2025-03-24 Thread via GitHub
m-trieu opened a new pull request, #34413: URL: https://github.com/apache/beam/pull/34413 We want to track the GetWorkTimings even when we don't proxy the calls through the dispatcher. R: @scwhittle Thank you for your contribution! Follow th

Re: [PR] add graceful restart mechanism for GetWorkStream to prevent DEADLINE_… [beam]

2025-03-24 Thread via GitHub
m-trieu commented on PR #34367: URL: https://github.com/apache/beam/pull/34367#issuecomment-2749923185 Run Java precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
damccorm commented on code in PR #34404: URL: https://github.com/apache/beam/pull/34404#discussion_r2010875137 ## sdks/python/apache_beam/yaml/yaml_provider.py: ## @@ -661,16 +661,20 @@ def create_transform(self, type, args, yaml_create_transform): return self._transform_f

Re: [I] [Bug]: Dataflow option validation fails in VSCode Jupyter notebook [beam]

2025-03-24 Thread via GitHub
shunping commented on issue #34101: URL: https://github.com/apache/beam/issues/34101#issuecomment-2749567109 Since this issue will continue to cause confusion in argument parsing, we can do a breaking change in the next release to disable allow_abbrev. WDYT? -- This is an automated messag

[PR] Update build.gradle [beam]

2025-03-24 Thread via GitHub
liferoad opened a new pull request, #34412: URL: https://github.com/apache/beam/pull/34412 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quick

Re: [PR] Fix PostCommit TransformService Direct job [beam]

2025-03-24 Thread via GitHub
robertwb commented on PR #34375: URL: https://github.com/apache/beam/pull/34375#issuecomment-2749571925 0.0.0.0 might assume/require ipv4 though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] add vendor to manually shutdown and restart GetWorkerMetadataStream to prevent DEADLINE_EXCEEDED errors [beam]

2025-03-24 Thread via GitHub
m-trieu commented on PR #34053: URL: https://github.com/apache/beam/pull/34053#issuecomment-2748458689 run Java precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Add warning comment so that folks don't change this [beam]

2025-03-24 Thread via GitHub
robertwb merged PR #34409: URL: https://github.com/apache/beam/pull/34409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Cherry pick 34401 for 2.64 [beam]

2025-03-24 Thread via GitHub
liferoad closed pull request #34403: Cherry pick 34401 for 2.64 URL: https://github.com/apache/beam/pull/34403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2749098821 > we can just change the JdbcIO.read transform (pointed above, JdbcIO.java#L1513) added by jdbcio.readfrompartition to explicitly set a schema (Long), this is a much smaller change than intr

[PR] Normalize tf record io [beam]

2025-03-24 Thread via GitHub
derrickaw opened a new pull request, #34411: URL: https://github.com/apache/beam/pull/34411 Add TFRecordIO normalization for java and python. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and eas

Re: [PR] Add to CHANGES.md follow up negative int32 fix [beam]

2025-03-24 Thread via GitHub
lostluck merged PR #34408: URL: https://github.com/apache/beam/pull/34408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Add to CHANGES.md follow up negative int32 fix [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34408: URL: https://github.com/apache/beam/pull/34408#issuecomment-2749333183 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @lostluck added as fallback since no labels match configuration

Re: [PR] Create a varint32 coder and used it for RowCoder [beam]

2025-03-24 Thread via GitHub
Abacn commented on code in PR #34354: URL: https://github.com/apache/beam/pull/34354#discussion_r2010837857 ## sdks/python/apache_beam/coders/coders.py: ## @@ -650,6 +650,25 @@ def __hash__(self): Coder.register_structured_urn(common_urns.coders.VARINT.urn, VarIntCoder) +cl

Re: [PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
robertwb merged PR #34404: URL: https://github.com/apache/beam/pull/34404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[PR] Add to CHANGES.md follow up negative int32 fix [beam]

2025-03-24 Thread via GitHub
Abacn opened a new pull request, #34408: URL: https://github.com/apache/beam/pull/34408 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly

Re: [I] The PostCommit Go Dataflow ARM job is flaky [beam]

2025-03-24 Thread via GitHub
Amar3tto commented on issue #34396: URL: https://github.com/apache/beam/issues/34396#issuecomment-2749098434 Failed Job ID: 2025-03-24_09_13_00-4907108703771792565 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Create a varint32 coder and used it for RowCoder [beam]

2025-03-24 Thread via GitHub
Abacn merged PR #34354: URL: https://github.com/apache/beam/pull/34354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

Re: [I] [Bug]: Cross-language JDBC (MSSQL) - incorrect negative Integer type conversion [beam]

2025-03-24 Thread via GitHub
Abacn closed issue #34089: [Bug]: Cross-language JDBC (MSSQL) - incorrect negative Integer type conversion URL: https://github.com/apache/beam/issues/34089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
damccorm commented on PR #34404: URL: https://github.com/apache/beam/pull/34404#issuecomment-2749234925 From a Beam perspective, this seems like a harmless change, but it seems like a fragile thing to depend on (and something we could easily accidentally (or intentionally need to) change

Re: [PR] adding google-crc32c [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34374: URL: https://github.com/apache/beam/pull/34374#issuecomment-2749223546 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewer

Re: [PR] adding google-crc32c [beam]

2025-03-24 Thread via GitHub
damccorm commented on PR #34374: URL: https://github.com/apache/beam/pull/34374#issuecomment-2749220869 R: @liferoad @tvalentyn adding you since you have more context here - thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] The PostCommit Go job is flaky [beam]

2025-03-24 Thread via GitHub
Amar3tto commented on issue #34397: URL: https://github.com/apache/beam/issues/34397#issuecomment-2749086715 Failed Job ID: 2025-03-23_15_07_30-2211162315298891746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2749147248 > But we might as well add it to give the ability to explicitly pass partition bounds to avoid extra database calls during execution. For now I am marginally fine with the change. For

Re: [PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34404: URL: https://github.com/apache/beam/pull/34404#issuecomment-2749142682 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label python. Available commands: - `stop

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
claudevdm commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2749120689 > > we can just change the JdbcIO.read transform (pointed above, JdbcIO.java#L1513) added by jdbcio.readfrompartition to explicitly set a schema (Long), this is a much smaller change tha

Re: [PR] [Java] Add parsedData to Hl7v2Message and Update HL7v2IO Docs [beam]

2025-03-24 Thread via GitHub
Suvrat1629 commented on PR #34213: URL: https://github.com/apache/beam/pull/34213#issuecomment-2749100046 > @svetakvsundhar I did fix the spotless checks and was working on the tests, I tried running the test file as it was and got the following error:`org.apache.beam.sdk.util.UserCodeExcep

Re: [I] The PostRelease Nightly Snapshot job is flaky [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on issue #30505: URL: https://github.com/apache/beam/issues/30505#issuecomment-2749083309 Reopening since the workflow is still flaky -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [Java]Add Map Type Support to JsonToRow Transformer [beam]

2025-03-24 Thread via GitHub
Suvrat1629 commented on PR #34347: URL: https://github.com/apache/beam/pull/34347#issuecomment-2749089908 @m-trieu Gentle ping, Please take a look. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[I] The PostRelease Nightly Snapshot job is flaky [beam]

2025-03-24 Thread via GitHub
github-actions[bot] opened a new issue, #30505: URL: https://github.com/apache/beam/issues/30505 The PostRelease Nightly Snapshot is failing over 50% of the time Please visit https://github.com/apache/beam/actions/workflows/beam_PostRelease_NightlySnapshot.yml?query=is%3Afailure+branch%3

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on code in PR #34394: URL: https://github.com/apache/beam/pull/34394#discussion_r2010736225 ## sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java: ## @@ -72,6 +72,8 @@ public Schema configurationSchema() { // readQuery

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
claudevdm commented on code in PR #34394: URL: https://github.com/apache/beam/pull/34394#discussion_r2010728937 ## sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java: ## @@ -72,6 +72,8 @@ public Schema configurationSchema() { // readQu

Re: [PR] [DO NOT MERGE] Removed < 1.66 for grpcio [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34196: URL: https://github.com/apache/beam/pull/34196#issuecomment-2747928097 Reminder, please take a look at this pr: @jrmccluskey -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [Feature Request]: Yaml JdbcIO partitioning [beam]

2025-03-24 Thread via GitHub
svetakvsundhar commented on issue #34349: URL: https://github.com/apache/beam/issues/34349#issuecomment-2749036330 thanks @claudevdm ! @shubham-j-sde , are you able to modify your PR to follow the above? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on code in PR #34394: URL: https://github.com/apache/beam/pull/34394#discussion_r2010704388 ## sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java: ## @@ -72,6 +72,8 @@ public Schema configurationSchema() { // readQuery

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2749017454 > there are two cases where the database is accessed during expansion time: > > 2. Get partition bounds This is because Get partition bounds is done by a JdbcIO.read transform

Re: [PR] Clickhouse IT RM JDBC fix + ClickHouseRowsCheck [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34407: URL: https://github.com/apache/beam/pull/34407#issuecomment-2749006919 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [I] [Feature Request]: Yaml JdbcIO partitioning [beam]

2025-03-24 Thread via GitHub
damccorm commented on issue #34349: URL: https://github.com/apache/beam/issues/34349#issuecomment-2748960497 Yeah, this sounds good to me - thanks Claude! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[PR] Factor out a get_default_sql_provider() function. [beam]

2025-03-24 Thread via GitHub
zilin opened a new pull request, #34404: URL: https://github.com/apache/beam/pull/34404 This will allow other users to replace the default sql provider used by 'Join' and 'Combine-sql'. **Please** add a meaningful description for your change here

[PR] Clickhouse IT RM JDBC fix + ClickHouseRowsCheck [beam]

2025-03-24 Thread via GitHub
BentsiLeviav opened a new pull request, #34407: URL: https://github.com/apache/beam/pull/34407 In this PR, the following is introduced: * A fix to the get JDBC function * Implement `ClickHouseRowsCheck` GitHub Actions Tests Status (on master branch) --

Re: [PR] Update build.gradle (#34401) [beam]

2025-03-24 Thread via GitHub
liferoad merged PR #34406: URL: https://github.com/apache/beam/pull/34406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2748832987 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @liferoad for label python. R: @kennknowles for label java.

Re: [PR] Fix PostCommit TransformService Direct job [beam]

2025-03-24 Thread via GitHub
Abacn commented on PR #34375: URL: https://github.com/apache/beam/pull/34375#issuecomment-2748806649 This was caused by #34073, specifically this line: https://github.com/apache/beam/pull/34073/files#diff-4d72b1d96a597dfd42528b37549162b5124462b5c8bf6655d9cef91d9b3c8ac7L58 cc: @robertw

Re: [PR] Update build.gradle [beam]

2025-03-24 Thread via GitHub
liferoad merged PR #34401: URL: https://github.com/apache/beam/pull/34401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

Re: [PR] Enable cloudpickle default [beam]

2025-03-24 Thread via GitHub
tvalentyn commented on code in PR #34223: URL: https://github.com/apache/beam/pull/34223#discussion_r2010565360 ## sdks/python/apache_beam/internal/pickler.py: ## @@ -33,9 +33,12 @@ USE_CLOUDPICKLE = 'cloudpickle' USE_DILL = 'dill' -DEFAULT_PICKLE_LIB = USE_DILL -desired_p

Re: [PR] Enforce ordering to avoid svn conflicts [beam]

2025-03-24 Thread via GitHub
damccorm merged PR #34405: URL: https://github.com/apache/beam/pull/34405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[PR] Cherry pick 34401 for 2.64 [beam]

2025-03-24 Thread via GitHub
liferoad opened a new pull request, #34403: URL: https://github.com/apache/beam/pull/34403 Cherry pick https://github.com/apache/beam/pull/34401/files Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickl

Re: [PR] Cherry-pick #34401 for Beam 2.64.0 [beam]

2025-03-24 Thread via GitHub
liferoad closed pull request #34402: Cherry-pick #34401 for Beam 2.64.0 URL: https://github.com/apache/beam/pull/34402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[PR] Cherry-pick #34401 for Beam 2.64.0 [beam]

2025-03-24 Thread via GitHub
liferoad opened a new pull request, #34402: URL: https://github.com/apache/beam/pull/34402 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quick

[PR] Update build.gradle [beam]

2025-03-24 Thread via GitHub
liferoad opened a new pull request, #34401: URL: https://github.com/apache/beam/pull/34401 added virtualenv-clone **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incor

Re: [PR] add vendor to manually shutdown and restart GetWorkerMetadataStream to prevent DEADLINE_EXCEEDED errors [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34053: URL: https://github.com/apache/beam/pull/34053#issuecomment-2748664153 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles added as fallback since no labels match configuration

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on PR #34394: URL: https://github.com/apache/beam/pull/34394#issuecomment-2748638867 also if the purpose is > bypassing the requirement to grant database access to machine where pipeline is built/submitted for partition bound inference. we should be able to chan

Re: [PR] Add ability to specify partition bounds via jdbc xlang. [beam]

2025-03-24 Thread via GitHub
Abacn commented on code in PR #34394: URL: https://github.com/apache/beam/pull/34394#discussion_r2010471432 ## sdks/java/io/jdbc/src/main/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProvider.java: ## @@ -72,6 +72,8 @@ public Schema configurationSchema() { // readQuery

Re: [PR] Issue warnings when dataSourceConfiguration and dataSourceProviderFn are set [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34395: URL: https://github.com/apache/beam/pull/34395#issuecomment-2748255400 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @m-trieu for label java. R: @damondouglas for label io.

Re: [PR] [Dataflow Streaming] Use concurrent queue for commit queue [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #33690: URL: https://github.com/apache/beam/pull/33690#issuecomment-2748016871 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull r

Re: [PR] add graceful restart mechanism for GetWorkStream to prevent DEADLINE_… [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34367: URL: https://github.com/apache/beam/pull/34367#issuecomment-2748483846 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @johnjcasey added as fallback since no labels match configuration

Re: [PR] add graceful restart mechanism for GetWorkStream to prevent DEADLINE_… [beam]

2025-03-24 Thread via GitHub
m-trieu commented on PR #34367: URL: https://github.com/apache/beam/pull/34367#issuecomment-2748479514 assign set of reviewers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [PR] cherry pick 34375 to 2.64.0 [beam]

2025-03-24 Thread via GitHub
liferoad commented on PR #34389: URL: https://github.com/apache/beam/pull/34389#issuecomment-2748130087 No need to cherry pick. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] Handle missing tables more gracefully. [beam]

2025-03-24 Thread via GitHub
github-actions[bot] closed pull request #33610: Handle missing tables more gracefully. URL: https://github.com/apache/beam/pull/33610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] CP: Perform correct release version validation (#34378) [beam]

2025-03-24 Thread via GitHub
liferoad merged PR #34400: URL: https://github.com/apache/beam/pull/34400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[PR] CP: Perform correct release version validation (#34378) [beam]

2025-03-24 Thread via GitHub
damccorm opened a new pull request, #34400: URL: https://github.com/apache/beam/pull/34400 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quick

Re: [PR] [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch [beam]

2025-03-24 Thread via GitHub
liferoad commented on PR #34381: URL: https://github.com/apache/beam/pull/34381#issuecomment-2748134290 https://github.com/apache/beam/blob/master/.github/workflows/beam_PostCommit_TransformService_Direct.yml fails with the release branch due to https://github.com/apache/beam/pull/34375. W

Re: [PR] cherry pick 34375 to 2.64.0 [beam]

2025-03-24 Thread via GitHub
liferoad closed pull request #34389: cherry pick 34375 to 2.64.0 URL: https://github.com/apache/beam/pull/34389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] Create a varint32 coder and used it for RowCoder [beam]

2025-03-24 Thread via GitHub
liferoad commented on code in PR #34354: URL: https://github.com/apache/beam/pull/34354#discussion_r2010175578 ## sdks/python/apache_beam/coders/coders.py: ## @@ -650,6 +650,25 @@ def __hash__(self): Coder.register_structured_urn(common_urns.coders.VARINT.urn, VarIntCoder)

Re: [PR] add graceful restart mechanism for GetWorkStream to prevent DEADLINE_… [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34367: URL: https://github.com/apache/beam/pull/34367#issuecomment-274646 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment `assign set of reviewer

Re: [I] Performance Regression or Improvement: sideinpts_python_batch_10gb_1kb_10workers_1000window_first_iterable:runtime [beam]

2025-03-24 Thread via GitHub
Amar3tto commented on issue #34198: URL: https://github.com/apache/beam/issues/34198#issuecomment-2748108364 https://github.com/user-attachments/assets/b006a679-d8a8-40fe-84bb-65ab8055b53a"; /> Looks ok -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [I] Performance Regression or Improvement: sideinpts_python_batch_10gb_1kb_10workers_1000window_first_iterable:runtime [beam]

2025-03-24 Thread via GitHub
Amar3tto closed issue #34198: Performance Regression or Improvement: sideinpts_python_batch_10gb_1kb_10workers_1000window_first_iterable:runtime URL: https://github.com/apache/beam/issues/34198 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] [Dataflow Streaming] Use concurrent queue for commit queue [beam]

2025-03-24 Thread via GitHub
arunpandianp commented on PR #33690: URL: https://github.com/apache/beam/pull/33690#issuecomment-2748080584 Have a different pending change to reduce contention on queue inserts. Closing this. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] [Dataflow Streaming] Use concurrent queue for commit queue [beam]

2025-03-24 Thread via GitHub
arunpandianp closed pull request #33690: [Dataflow Streaming] Use concurrent queue for commit queue URL: https://github.com/apache/beam/pull/33690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add support for collections.abc.Mapping [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34001: URL: https://github.com/apache/beam/pull/34001#issuecomment-2747928478 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluske

Re: [PR] Fix ProtoCoder NoSuchMethodException [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34194: URL: https://github.com/apache/beam/pull/34194#issuecomment-2747928178 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damondougl

Re: [PR] Interpolate console URL properly in error case [beam]

2025-03-24 Thread via GitHub
liferoad commented on PR #28066: URL: https://github.com/apache/beam/pull/28066#issuecomment-2746329744 @kennknowles let me know if you could get this PR ready. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] upgraded gcp-bom to 26.57.0 [beam]

2025-03-24 Thread via GitHub
github-actions[bot] commented on PR #34399: URL: https://github.com/apache/beam/pull/34399#issuecomment-2747662787 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

Re: [I] [Bug]: beam.Map function outputs no data after beam.GroupByKey using DataflowRunner [beam]

2025-03-24 Thread via GitHub
bgalvao commented on issue #34360: URL: https://github.com/apache/beam/issues/34360#issuecomment-2747727749 > Have you changed the Beam version? I'm using 2.60 at the moment. Can try and upgrade to latest (2.63). One thing I noticed is that only by commenting out the timestamp_a

[PR] upgraded gcp-bom to 26.57.0 [beam]

2025-03-24 Thread via GitHub
sarthakbhutani opened a new pull request, #34399: URL: https://github.com/apache/beam/pull/34399 ran ``` python3 scripts/tools/bomupgrader.py 26.57.0``` logs - ``` INFO:root:-Read BeamModulePlugin- INFO:root:-check dependency defs in BeamModulePlugin- INFO:

Re: [I] The PostCommit Java Jpms Dataflow Java17 job is flaky [beam]

2025-03-24 Thread via GitHub
scwhittle commented on issue #33786: URL: https://github.com/apache/beam/issues/33786#issuecomment-2747218900 The workers are failing with error: ``` java.lang.NoSuchMethodError: 'boolean org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements(org.apache.beam.model.fnex

Re: [PR] test commit [beam]

2025-03-24 Thread via GitHub
m-trieu commented on code in PR #34148: URL: https://github.com/apache/beam/pull/34148#discussion_r2006479893 ## runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/streaming/harness/WindmillStreamSender.java: ## @@ -17,152 +17,18 @@