[GitHub] [beam] AnandInguva commented on a diff in pull request #28384: [WIP] pyproject toml support.

2023-09-08 Thread via GitHub
AnandInguva commented on code in PR #28384: URL: https://github.com/apache/beam/pull/28384#discussion_r1320500820 ## sdks/python/tox.ini: ## @@ -124,8 +127,7 @@ commands = setenv = # keep the version of pylint in sync with the 'rev' in .pre-commit-config.yaml deps = - -r bui

[GitHub] [beam] AnandInguva opened a new pull request, #28384: Pyproject rebase

2023-09-08 Thread via GitHub
AnandInguva opened a new pull request, #28384: URL: https://github.com/apache/beam/pull/28384 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution qu

[GitHub] [beam] github-actions[bot] commented on pull request #28305: Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_Avro_Versions

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28305: URL: https://github.com/apache/beam/pull/28305#issuecomment-1712432715 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label build. Available commands: - `stop r

[GitHub] [beam] github-actions[bot] commented on pull request #28303: Add GitHub Workflow Replacement for Jenkins job_PostCommit_Sickbay_Python

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28303: URL: https://github.com/apache/beam/pull/28303#issuecomment-1712432732 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label build. Available commands: - `stop revi

[GitHub] [beam] aleksandr-dudko commented on a diff in pull request #28305: Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_Avro_Versions

2023-09-08 Thread via GitHub
aleksandr-dudko commented on code in PR #28305: URL: https://github.com/apache/beam/pull/28305#discussion_r1320496295 ## .github/workflows/beam_PostCommit_Java_Avro_Versions.yml: ## @@ -0,0 +1,79 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

[GitHub] [beam] tvalentyn commented on a diff in pull request #28243: Add docs for per key inference

2023-09-08 Thread via GitHub
tvalentyn commented on code in PR #28243: URL: https://github.com/apache/beam/pull/28243#discussion_r1320479937 ## website/www/site/content/en/documentation/sdks/python-machine-learning.md: ## @@ -215,6 +215,53 @@ with pipeline as p: If you are unsure if your data is keyed, y

[GitHub] [beam] Abacn commented on a diff in pull request #28173: Fix flaky test StreamingDataflowWorkerTest

2023-09-08 Thread via GitHub
Abacn commented on code in PR #28173: URL: https://github.com/apache/beam/pull/28173#discussion_r1320458889 ## runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorkerTest.java: ## @@ -33,11 +33,7 @@ import static o

[GitHub] [beam] Abacn commented on pull request #28173: Fix flaky test StreamingDataflowWorkerTest

2023-09-08 Thread via GitHub
Abacn commented on PR #28173: URL: https://github.com/apache/beam/pull/28173#issuecomment-1712386097 Interestingly, after merging this test worked well for a while became highly flaky since Sept 8th: #28383 -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [beam] github-actions[bot] commented on pull request #28382: Support legacy DATE and TIME logical types in xlang JdbcIO

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28382: URL: https://github.com/apache/beam/pull/28382#issuecomment-1712385864 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @liferoad for label python. R: @bvolpato for label java. R:

[GitHub] [beam] tvalentyn commented on issue #28246: [Bug]: [Python SDK] Memory leak in 2.47.0 - 2.50.0 SDKs.

2023-09-08 Thread via GitHub
tvalentyn commented on issue #28246: URL: https://github.com/apache/beam/issues/28246#issuecomment-1712377743 > I'm getting dependency conflict when trying to build a runtime image with > ... > is this a known issue? Must be side effects from poetry installation. See: https://be

[GitHub] [beam] tvalentyn commented on pull request #28365: Update protobuf dependency to the version that fixes memory leaks.

2023-09-08 Thread via GitHub
tvalentyn commented on PR #28365: URL: https://github.com/apache/beam/pull/28365#issuecomment-1712376942 Unfrotunately most recent version of protbuf is still leaking memory, will follow up with protobuf team. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] tvalentyn closed pull request #28365: Update protobuf dependency to the version that fixes memory leaks.

2023-09-08 Thread via GitHub
tvalentyn closed pull request #28365: Update protobuf dependency to the version that fixes memory leaks. URL: https://github.com/apache/beam/pull/28365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [beam] tvalentyn commented on issue #28246: [Bug]: [Python SDK] Memory leak in 2.47.0 - 2.50.0 SDKs.

2023-09-08 Thread via GitHub
tvalentyn commented on issue #28246: URL: https://github.com/apache/beam/issues/28246#issuecomment-1712376412 Unfortunately I am still seeing a leak on `protobuf==4.24.3`, asked in https://github.com/protocolbuffers/protobuf/issues/10088#issuecomment-1712375991 . -- This is an automated

[GitHub] [beam] codecov[bot] commented on pull request #28382: Support legacy DATE and TIME logical types in xlang JdbcIO

2023-09-08 Thread via GitHub
codecov[bot] commented on PR #28382: URL: https://github.com/apache/beam/pull/28382#issuecomment-1712364137 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/28382?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report

[GitHub] [beam] Abacn opened a new pull request, #28382: Support legacy DATE and TIME logical types in xlang JdbcIO

2023-09-08 Thread via GitHub
Abacn opened a new pull request, #28382: URL: https://github.com/apache/beam/pull/28382 Part of #28359 (date and time support without breaking change); fixes #19715 * Add identifier to URN for legacy Java logical types * Implement JdbcIO logical type javasdk_date and javasdk_ti

[GitHub] [beam] github-actions[bot] commented on pull request #28381: Fix typos, and clarify playground deploy.

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28381: URL: https://github.com/apache/beam/pull/28381#issuecomment-1712354893 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @jrmccluskey added as fallback since no labels match configuration

[GitHub] [beam] volatilemolotov commented on a diff in pull request #28305: Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_Avro_Versions

2023-09-08 Thread via GitHub
volatilemolotov commented on code in PR #28305: URL: https://github.com/apache/beam/pull/28305#discussion_r1320417980 ## .github/workflows/beam_PostCommit_Java_Avro_Versions.yml: ## @@ -0,0 +1,79 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

[GitHub] [beam] lostluck opened a new pull request, #28381: Fix typos, and clarify playground deploy.

2023-09-08 Thread via GitHub
lostluck opened a new pull request, #28381: URL: https://github.com/apache/beam/pull/28381 Fix a typo, and make some of the instructions more explicit. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quick

[GitHub] [beam] ioggstream commented on issue #23457: [Feature Request]: documentation how to override connector logic to be reflected when executing pipeline

2023-09-08 Thread via GitHub
ioggstream commented on issue #23457: URL: https://github.com/apache/beam/issues/23457#issuecomment-1712248583 Wouldn't it be OK to parametrize ReplaceOne with something like the following? It works for me, and everything remains quite isolated. We just need to abstract out the WriteOp

[GitHub] [beam] robertwb commented on a diff in pull request #28363: [yaml] Add UDF docs

2023-09-08 Thread via GitHub
robertwb commented on code in PR #28363: URL: https://github.com/apache/beam/pull/28363#discussion_r1320349719 ## sdks/python/apache_beam/yaml/readme_test.py: ## @@ -155,14 +161,27 @@ def __exit__(self, *args): self.tempdir.cleanup() +def replace_mapping_fields(spec, ar

[GitHub] [beam] lostluck merged pull request #28287: Post 2.50.0 Streamline release guide improvements

2023-09-08 Thread via GitHub
lostluck merged PR #28287: URL: https://github.com/apache/beam/pull/28287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] lostluck commented on pull request #28287: Post 2.50.0 Streamline release guide improvements

2023-09-08 Thread via GitHub
lostluck commented on PR #28287: URL: https://github.com/apache/beam/pull/28287#issuecomment-1712130007 > I hate hate HATE markdown that is only readable when rendered (aka using a sequence of 1. 1. 1. 1.). But you didn't start the fire, for the most part... I'm right there with you,

[GitHub] [beam] andreydevyatkin commented on a diff in pull request #28305: Add GitHub Workflow Replacement for Jenkins job_PostCommit_Java_Avro_Versions

2023-09-08 Thread via GitHub
andreydevyatkin commented on code in PR #28305: URL: https://github.com/apache/beam/pull/28305#discussion_r1320247539 ## .github/workflows/beam_PostCommit_Java_Avro_Versions.yml: ## @@ -0,0 +1,79 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contrib

[GitHub] [beam] damccorm commented on a diff in pull request #28327: Add notebook for per key models

2023-09-08 Thread via GitHub
damccorm commented on code in PR #28327: URL: https://github.com/apache/beam/pull/28327#discussion_r1320229732 ## examples/notebooks/beam-ml/per_key_models.ipynb: ## @@ -0,0 +1,597 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { +"colab": { + "provenanc

[GitHub] [beam] damccorm commented on a diff in pull request #28327: Add notebook for per key models

2023-09-08 Thread via GitHub
damccorm commented on code in PR #28327: URL: https://github.com/apache/beam/pull/28327#discussion_r1320229732 ## examples/notebooks/beam-ml/per_key_models.ipynb: ## @@ -0,0 +1,597 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { +"colab": { + "provenanc

[GitHub] [beam] damccorm commented on a diff in pull request #28327: Add notebook for per key models

2023-09-08 Thread via GitHub
damccorm commented on code in PR #28327: URL: https://github.com/apache/beam/pull/28327#discussion_r1320229431 ## examples/notebooks/beam-ml/per_key_models.ipynb: ## @@ -0,0 +1,597 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { +"colab": { + "provenanc

[GitHub] [beam] damccorm commented on pull request #28327: Add notebook for per key models

2023-09-08 Thread via GitHub
damccorm commented on PR #28327: URL: https://github.com/apache/beam/pull/28327#issuecomment-1712066475 > Should we also include this in the ReadMe list? https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/README.md Yes, done! -- This is an automated message from

[GitHub] [beam] lostluck commented on pull request #28287: Post 2.50.0 Streamline release guide improvments

2023-09-08 Thread via GitHub
lostluck commented on PR #28287: URL: https://github.com/apache/beam/pull/28287#issuecomment-1712057821 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [beam] github-actions[bot] commented on pull request #28380: Add Java variants of WriteTo{Csv,Json}.

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28380: URL: https://github.com/apache/beam/pull/28380#issuecomment-1712056531 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] lostluck commented on a diff in pull request #28287: Post 2.50.0 Streamline release guide improvments

2023-09-08 Thread via GitHub
lostluck commented on code in PR #28287: URL: https://github.com/apache/beam/pull/28287#discussion_r1320197180 ## website/www/site/content/en/contribute/release-guide.md: ## @@ -167,101 +228,103 @@ Configure access to the [Apache Nexus repository](https://repository.apache.org/

[GitHub] [beam] robertwb commented on pull request #28380: Add Java variants of WriteTo{Csv,Json}.

2023-09-08 Thread via GitHub
robertwb commented on PR #28380: URL: https://github.com/apache/beam/pull/28380#issuecomment-1712055140 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [beam] robertwb opened a new pull request, #28380: Add Java variants of WriteTo{Csv,Json}.

2023-09-08 Thread via GitHub
robertwb opened a new pull request, #28380: URL: https://github.com/apache/beam/pull/28380 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue in yo

[GitHub] [beam] lostluck opened a new issue, #28379: [Task]: Update Java BOM dependencies

2023-09-08 Thread via GitHub
lostluck opened a new issue, #28379: URL: https://github.com/apache/beam/issues/28379 ### What needs to happen? Follow https://cwiki.apache.org/confluence/display/BEAM/Dependency+Upgrades to update Java Bom dependencies prior to the release cut, if necessary. If unsure, ask the

[GitHub] [beam] volatilemolotov commented on pull request #28373: update concurrency group definition

2023-09-08 Thread via GitHub
volatilemolotov commented on PR #28373: URL: https://github.com/apache/beam/pull/28373#issuecomment-1712018521 @Abacn Yeah i left in in draft until confirmation. Its ready now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] robertwb merged pull request #28268: Add standard {ReadFrom,WriteTo}{Csv,Json} yaml endpoints.

2023-09-08 Thread via GitHub
robertwb merged PR #28268: URL: https://github.com/apache/beam/pull/28268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] robertwb merged pull request #28310: Informed choice of yaml provider for root transforms.

2023-09-08 Thread via GitHub
robertwb merged PR #28310: URL: https://github.com/apache/beam/pull/28310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] robertwb merged pull request #28190: Add some documentation on yaml error handling.

2023-09-08 Thread via GitHub
robertwb merged PR #28190: URL: https://github.com/apache/beam/pull/28190 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] robertwb merged pull request #28169: [YAML] Implement a consistent BigQuery read and write transform.

2023-09-08 Thread via GitHub
robertwb merged PR #28169: URL: https://github.com/apache/beam/pull/28169 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] lostluck commented on issue #27165: [Bug]: When using beam.io.kinesis.ReadDataFromKinesis, a java error is raised in DataFlowRunner

2023-09-08 Thread via GitHub
lostluck commented on issue #27165: URL: https://github.com/apache/beam/issues/27165#issuecomment-1711953973 As an open source project, the primary means of fixes is via community and user involvement. As a rule, no one can expect someone else to fix an issue just because it's filed.

[GitHub] [beam] lostluck merged pull request #28042: New MLTransform documentation

2023-09-08 Thread via GitHub
lostluck merged PR #28042: URL: https://github.com/apache/beam/pull/28042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] github-actions[bot] commented on pull request #28344: Github Workflow Replacement for Jenkins Job, beam_PostCommit_TransformService_Direct

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28344: URL: https://github.com/apache/beam/pull/28344#issuecomment-1711945665 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for label build. Available commands: - `stop revi

[GitHub] [beam] github-actions[bot] commented on pull request #28377: Update build.gradle -- remove duplicate deps

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28377: URL: https://github.com/apache/beam/pull/28377#issuecomment-1711945368 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[GitHub] [beam] rszper commented on pull request #28042: New MLTransform documentation

2023-09-08 Thread via GitHub
rszper commented on PR #28042: URL: https://github.com/apache/beam/pull/28042#issuecomment-1711918436 @lostluck & @AnandInguva This is ready to merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [beam] magicgoody commented on a diff in pull request #28367: Github Workflow Replacement for Jenkins Job, beam_PreCommit_Java_Kafka_IO_Direct

2023-09-08 Thread via GitHub
magicgoody commented on code in PR #28367: URL: https://github.com/apache/beam/pull/28367#discussion_r1319484172 ## .github/workflows/beam_PreCommit_Java_Kafka_IO_Direct.yml: ## @@ -0,0 +1,109 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributo

[GitHub] [beam] robertwb commented on a diff in pull request #28208: Add an annotation to expose transforms to yaml.

2023-09-08 Thread via GitHub
robertwb commented on code in PR #28208: URL: https://github.com/apache/beam/pull/28208#discussion_r1320051985 ## sdks/python/apache_beam/yaml/yaml_transform_test.py: ## @@ -229,6 +229,24 @@ def test_name_is_ambiguous(self): output: AnotherFilter ''')

[GitHub] [beam] robertwb commented on pull request #28208: Add an annotation to expose transforms to yaml.

2023-09-08 Thread via GitHub
robertwb commented on PR #28208: URL: https://github.com/apache/beam/pull/28208#issuecomment-1711899140 This won't replace the assembly performed in `yaml_transform.py, this merely gives indications of how to use other (posibly not-directly-exposed) Beam transforms directly from yaml. --

[GitHub] [beam] Abacn commented on issue #28359: [Task]: Switch to portable Data and Time type for JdbcIO

2023-09-08 Thread via GitHub
Abacn commented on issue #28359: URL: https://github.com/apache/beam/issues/28359#issuecomment-1711874721 Note: the following workaround can be used for columns involving Date (or Time) type, that is register your own logical type: ```python @LogicalType.register_logical_type

[GitHub] [beam] codecov[bot] commented on pull request #28375: Remove level and errors arguments for Pandas 2 compatibility.

2023-09-08 Thread via GitHub
codecov[bot] commented on PR #28375: URL: https://github.com/apache/beam/pull/28375#issuecomment-1711871585 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/28375?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report

[GitHub] [beam] Abacn commented on pull request #28373: update concurrency group definition

2023-09-08 Thread via GitHub
Abacn commented on PR #28373: URL: https://github.com/apache/beam/pull/28373#issuecomment-1711864716 ah it's still in draft status (and looks good for me) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [beam] chleech commented on issue #28246: [Bug]: [Python SDK] Memory leak in 2.47.0 - 2.50.0 SDKs.

2023-09-08 Thread via GitHub
chleech commented on issue #28246: URL: https://github.com/apache/beam/issues/28246#issuecomment-1711846057 I'm getting dependency conflict when trying to build a runtime image with ``` FROM apache/beam_python3.9_sdk:2.50.0 ARG WORKDIR=/dataflow/container ARG TEMPLATE_NAME=non

[GitHub] [beam] github-actions[bot] commented on pull request #28376: Add notebook for RunInference with Hugging Face

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28376: URL: https://github.com/apache/beam/pull/28376#issuecomment-1711834619 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] riteshghorse commented on pull request #28376: Add notebook for RunInference with Hugging Face

2023-09-08 Thread via GitHub
riteshghorse commented on PR #28376: URL: https://github.com/apache/beam/pull/28376#issuecomment-1711832709 R: @rszper -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [beam] gDuperran opened a new pull request, #28377: Update build.gradle -- remove duplicate deps

2023-09-08 Thread via GitHub
gDuperran opened a new pull request, #28377: URL: https://github.com/apache/beam/pull/28377 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quic

[GitHub] [beam] github-actions[bot] commented on pull request #28374: Add mechanism to override bigtable client for testing

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28374: URL: https://github.com/apache/beam/pull/28374#issuecomment-1711817925 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. R: @chamikaramj for label io.

[GitHub] [beam] riteshghorse opened a new pull request, #28376: Add notebook for RunInference with Hugging Face

2023-09-08 Thread via GitHub
riteshghorse opened a new pull request, #28376: URL: https://github.com/apache/beam/pull/28376 Adds a notebook with 2 examples of using Hugging Face model handlers. [Notebook](https://github.com/riteshghorse/beam/blob/hf-notebook/examples/notebooks/beam-ml/run_inference_huggingface.ipynb)

[GitHub] [beam] chleech commented on issue #28246: [Bug]: [Python SDK] Memory leak in 2.47.0 - 2.50.0 SDKs.

2023-09-08 Thread via GitHub
chleech commented on issue #28246: URL: https://github.com/apache/beam/issues/28246#issuecomment-1711802360 @tvalentyn I'm trying to build a runtime image using `FROM apache/beam_python3.9_sdk:2.50.0` but I'm getting this error when I run `pip check` ``` Step #1 - "Build runtime c

[GitHub] [beam] github-actions[bot] commented on pull request #28375: Remove level and errors arguments for Pandas 2 compatibility.

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28375: URL: https://github.com/apache/beam/pull/28375#issuecomment-1711788990 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] caneff commented on pull request #28375: Remove level and errors arguments for Pandas 2 compatibility.

2023-09-08 Thread via GitHub
caneff commented on PR #28375: URL: https://github.com/apache/beam/pull/28375#issuecomment-1711786642 R: @tvalentyn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

[GitHub] [beam] github-actions[bot] commented on pull request #28373: update concurrency group definition

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28373: URL: https://github.com/apache/beam/pull/28373#issuecomment-1711785500 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] caneff opened a new pull request, #28375: Remove level and errors arguments for Pandas 2 compatibility. Level was always NotImplemented anyway, and errors I conditionally disable based

2023-09-08 Thread via GitHub
caneff opened a new pull request, #28375: URL: https://github.com/apache/beam/pull/28375 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

[GitHub] [beam] volatilemolotov commented on pull request #28373: update concurrency group definition

2023-09-08 Thread via GitHub
volatilemolotov commented on PR #28373: URL: https://github.com/apache/beam/pull/28373#issuecomment-1711783401 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] tonytanger commented on pull request #28374: Add mechanism to override bigtable client for testing

2023-09-08 Thread via GitHub
tonytanger commented on PR #28374: URL: https://github.com/apache/beam/pull/28374#issuecomment-1711772324 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[GitHub] [beam] jrmccluskey merged pull request #28361: Add descriptions for the 2 vertex examples

2023-09-08 Thread via GitHub
jrmccluskey merged PR #28361: URL: https://github.com/apache/beam/pull/28361 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apa

[GitHub] [beam] ahmedabu98 commented on pull request #28272: Fix pane info in BigQuery load job id

2023-09-08 Thread via GitHub
ahmedabu98 commented on PR #28272: URL: https://github.com/apache/beam/pull/28272#issuecomment-1711735695 Run PostCommit_Java_Dataflow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [beam] ahmedabu98 commented on pull request #28272: Fix pane info in BigQuery load job id

2023-09-08 Thread via GitHub
ahmedabu98 commented on PR #28272: URL: https://github.com/apache/beam/pull/28272#issuecomment-1711736067 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [beam] ahmedabu98 commented on a diff in pull request #28272: Fix pane info in BigQuery load job id

2023-09-08 Thread via GitHub
ahmedabu98 commented on code in PR #28272: URL: https://github.com/apache/beam/pull/28272#discussion_r1319920266 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java: ## @@ -156,27 +149,36 @@ private class WriteTablesDoFn

[GitHub] [beam] github-actions[bot] commented on pull request #28361: Add descriptions for the 2 vertex examples

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28361: URL: https://github.com/apache/beam/pull/28361#issuecomment-1711728367 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] damccorm commented on pull request #28361: Add descriptions for the 2 vertex examples

2023-09-08 Thread via GitHub
damccorm commented on PR #28361: URL: https://github.com/apache/beam/pull/28361#issuecomment-1711725722 R: @jrmccluskey -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] damccorm merged pull request #28358: Add vertex ai llm example + integration test

2023-09-08 Thread via GitHub
damccorm merged PR #28358: URL: https://github.com/apache/beam/pull/28358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] Polber commented on a diff in pull request #28208: Add an annotation to expose transforms to yaml.

2023-09-08 Thread via GitHub
Polber commented on code in PR #28208: URL: https://github.com/apache/beam/pull/28208#discussion_r1319891912 ## sdks/python/apache_beam/yaml/yaml_transform_test.py: ## @@ -229,6 +229,24 @@ def test_name_is_ambiguous(self): output: AnotherFilter ''') +

[GitHub] [beam] jackdingilian commented on pull request #28374: Add mechanism to override bigtable client for testing

2023-09-08 Thread via GitHub
jackdingilian commented on PR #28374: URL: https://github.com/apache/beam/pull/28374#issuecomment-1711707141 @tonytanger -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] jackdingilian opened a new pull request, #28374: Add mechanism to override bigtable client for testing

2023-09-08 Thread via GitHub
jackdingilian opened a new pull request, #28374: URL: https://github.com/apache/beam/pull/28374 Adds `BigtableClientOverride` interface to provide a way to override client settings we need to change for testing. Thank you for your contribution! Fol

[GitHub] [beam] bzablocki commented on a diff in pull request #28310: Informed choice of yaml provider for root transforms.

2023-09-08 Thread via GitHub
bzablocki commented on code in PR #28310: URL: https://github.com/apache/beam/pull/28310#discussion_r1319893045 ## sdks/python/apache_beam/yaml/yaml_transform.py: ## @@ -208,6 +229,77 @@ def get_outputs(self, transform_name): def compute_outputs(self, transform_id): retu

[GitHub] [beam] codecov[bot] commented on pull request #28363: [yaml] Add UDF docs

2023-09-08 Thread via GitHub
codecov[bot] commented on PR #28363: URL: https://github.com/apache/beam/pull/28363#issuecomment-1711700302 ## [Codecov](https://app.codecov.io/gh/apache/beam/pull/28363?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report

[GitHub] [beam] bzablocki commented on a diff in pull request #28310: Informed choice of yaml provider for root transforms.

2023-09-08 Thread via GitHub
bzablocki commented on code in PR #28310: URL: https://github.com/apache/beam/pull/28310#discussion_r1319892050 ## sdks/python/apache_beam/yaml/yaml_transform.py: ## @@ -208,6 +220,56 @@ def get_outputs(self, transform_name): def compute_outputs(self, transform_id): retu

[GitHub] [beam] bzablocki commented on a diff in pull request #28310: Informed choice of yaml provider for root transforms.

2023-09-08 Thread via GitHub
bzablocki commented on code in PR #28310: URL: https://github.com/apache/beam/pull/28310#discussion_r1319892334 ## sdks/python/apache_beam/yaml/yaml_transform.py: ## @@ -208,6 +220,56 @@ def get_outputs(self, transform_name): def compute_outputs(self, transform_id): retu

[GitHub] [beam] damccorm commented on a diff in pull request #28116: Bump classgraph version to 4.8.162

2023-09-08 Thread via GitHub
damccorm commented on code in PR #28116: URL: https://github.com/apache/beam/pull/28116#discussion_r1319838762 ## runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/resources/ClasspathScanningResourcesDetectorTest.java: ## @@ -120,7 +120,7 @@

[GitHub] [beam] damccorm merged pull request #28116: Bump classgraph version to 4.8.162

2023-09-08 Thread via GitHub
damccorm merged PR #28116: URL: https://github.com/apache/beam/pull/28116 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] scwhittle closed pull request #28371: Change WindmillStateReader to not batch OrderedListFetches for the same family and tag.

2023-09-08 Thread via GitHub
scwhittle closed pull request #28371: Change WindmillStateReader to not batch OrderedListFetches for the same family and tag. URL: https://github.com/apache/beam/pull/28371 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [beam] bvolpato commented on a diff in pull request #28116: Bump classgraph version to 4.8.162

2023-09-08 Thread via GitHub
bvolpato commented on code in PR #28116: URL: https://github.com/apache/beam/pull/28116#discussion_r1319829087 ## runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/resources/ClasspathScanningResourcesDetectorTest.java: ## @@ -120,7 +120,7 @@

[GitHub] [beam] github-actions[bot] commented on pull request #28371: Change WindmillStateReader to not batch OrderedListFetches for the same family and tag.

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28371: URL: https://github.com/apache/beam/pull/28371#issuecomment-1711609180 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[GitHub] [beam] github-actions[bot] commented on pull request #27340: Github Workflow Replacement for Jenkins Job, beam_PreCommit_SQL (Cron, Commit, Phrase)

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #27340: URL: https://github.com/apache/beam/pull/27340#issuecomment-1711607639 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is

[GitHub] [beam] github-actions[bot] closed pull request #27340: Github Workflow Replacement for Jenkins Job, beam_PreCommit_SQL (Cron, Commit, Phrase)

2023-09-08 Thread via GitHub
github-actions[bot] closed pull request #27340: Github Workflow Replacement for Jenkins Job, beam_PreCommit_SQL (Cron, Commit, Phrase) URL: https://github.com/apache/beam/pull/27340 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] bzablocki commented on a diff in pull request #28310: Informed choice of yaml provider for root transforms.

2023-09-08 Thread via GitHub
bzablocki commented on code in PR #28310: URL: https://github.com/apache/beam/pull/28310#discussion_r1319812013 ## sdks/python/apache_beam/yaml/yaml_transform.py: ## @@ -225,19 +287,7 @@ def create_ptransform(self, spec, input_pcolls): providers_by_input[pcoll] for pcol

[GitHub] [beam] damccorm commented on a diff in pull request #28116: Bump classgraph version to 4.8.162

2023-09-08 Thread via GitHub
damccorm commented on code in PR #28116: URL: https://github.com/apache/beam/pull/28116#discussion_r1319811100 ## runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/resources/ClasspathScanningResourcesDetectorTest.java: ## @@ -120,7 +120,7 @@

[GitHub] [beam] github-actions[bot] commented on pull request #28243: Add docs for per key inference

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28243: URL: https://github.com/apache/beam/pull/28243#issuecomment-1711579921 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] damccorm commented on pull request #28243: Add docs for per key inference

2023-09-08 Thread via GitHub
damccorm commented on PR #28243: URL: https://github.com/apache/beam/pull/28243#issuecomment-1711578403 R: @tvalentyn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [beam] damccorm commented on pull request #28358: Add vertex ai llm example + integration test

2023-09-08 Thread via GitHub
damccorm commented on PR #28358: URL: https://github.com/apache/beam/pull/28358#issuecomment-1711576492 (Note that the failing tests are io related, not due to vertex changes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [beam] github-actions[bot] commented on pull request #26943: Vortex test

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #26943: URL: https://github.com/apache/beam/pull/26943#issuecomment-1711575731 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @robertwb f

[GitHub] [beam] github-actions[bot] commented on pull request #28050: Don't invalidate streams on quota errors

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28050: URL: https://github.com/apache/beam/pull/28050#issuecomment-1711575547 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for

[GitHub] [beam] github-actions[bot] commented on pull request #28116: Bump classgraph version to 4.8.162

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28116: URL: https://github.com/apache/beam/pull/28116#issuecomment-1711575414 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm f

[GitHub] [beam] github-actions[bot] commented on pull request #28181: Publish SpotBugs Results

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28181: URL: https://github.com/apache/beam/pull/28181#issuecomment-1711575270 Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`: R: @Abacn for

[GitHub] [beam] github-actions[bot] commented on pull request #28226: Add govulncheck to Go Tests workflow

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28226: URL: https://github.com/apache/beam/pull/28226#issuecomment-1711575238 Reminder, please take a look at this pr: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [beam] github-actions[bot] commented on pull request #28267: Increase transform service timeout

2023-09-08 Thread via GitHub
github-actions[bot] commented on PR #28267: URL: https://github.com/apache/beam/pull/28267#issuecomment-1711575186 Reminder, please take a look at this pr: @damccorm @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] volatilemolotov commented on issue #28320: [Bug]: GitHub Action Phrase Issues

2023-09-08 Thread via GitHub
volatilemolotov commented on issue #28320: URL: https://github.com/apache/beam/issues/28320#issuecomment-1711552172 Created a draft: https://github.com/apache/beam/pull/28373 I agree with the approach and some limited testing shows it should be correct If we agree I will move from draft

[GitHub] [beam] volatilemolotov opened a new pull request, #28373: update concurrency group definition

2023-09-08 Thread via GitHub
volatilemolotov opened a new pull request, #28373: URL: https://github.com/apache/beam/pull/28373 Adds comment body to concurrency group definition to prevent different PRs canceling other jobs. Fixes part of: #28320 Thank you for your contribution! Fo

[GitHub] [beam] Amar3tto opened a new pull request, #28372: Add Load Tests CoGBK Dataflow Streaming Java github action

2023-09-08 Thread via GitHub
Amar3tto opened a new pull request, #28372: URL: https://github.com/apache/beam/pull/28372 The following jenkins job has been migrated: https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_Streaming https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_Stre

[GitHub] [beam] scwhittle opened a new pull request, #28371: Change WindmillStateReader to not batch OrderedListFetches for the same family and tag.

2023-09-08 Thread via GitHub
scwhittle opened a new pull request, #28371: URL: https://github.com/apache/beam/pull/28371 Fix issue with MultimapState delayed fetches due to batching. Thank you for your contribution! Follow this checklist to help us incorporate your contribution qui

[GitHub] [beam] RustedBones closed pull request #27971: Increase consistency for BQ typed read with avro source

2023-09-08 Thread via GitHub
RustedBones closed pull request #27971: Increase consistency for BQ typed read with avro source URL: https://github.com/apache/beam/pull/27971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [beam] RustedBones commented on pull request #27971: Increase consistency for BQ typed read with avro source

2023-09-08 Thread via GitHub
RustedBones commented on PR #27971: URL: https://github.com/apache/beam/pull/27971#issuecomment-1711464530 I did not have time to start the discussion on the mailing list to discuss the breaking changes and unify the API between Avro and BQ Avro dump. We can close this one, and in case this

  1   2   >