[GitHub] [beam] Malarg opened a new pull request, #25397: Issue25329 fix cors issue when no internet

2023-02-08 Thread via GitHub
Malarg opened a new pull request, #25397: URL: https://github.com/apache/beam/pull/25397 resolves 25329 resolves 25331 resolves 25336 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easil

[GitHub] [beam] github-actions[bot] commented on pull request #25301: [Tour of Beam] Learning content for "IO Connectors" module

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25301: URL: https://github.com/apache/beam/pull/25301#issuecomment-1423700975 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] sirenbyte commented on pull request #25301: [Tour of Beam] Learning content for "IO Connectors" module

2023-02-08 Thread via GitHub
sirenbyte commented on PR #25301: URL: https://github.com/apache/beam/pull/25301#issuecomment-1423700115 R: @alxp1982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [beam] kmjung commented on a diff in pull request #25392: Read API Source v2

2023-02-08 Thread via GitHub
kmjung commented on code in PR #25392: URL: https://github.com/apache/beam/pull/25392#discussion_r1101005715 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryOptions.java: ## @@ -150,4 +150,12 @@ Long getStorageWriteApiMaxRequest

[GitHub] [beam] pabloem merged pull request #24999: Adding SpannerIO.readChangeStreams support for SchemaTransform

2023-02-08 Thread via GitHub
pabloem merged PR #24999: URL: https://github.com/apache/beam/pull/24999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.

[GitHub] [beam] pabloem merged pull request #25268: Adding support for @SchemaFieldDescription annotation that allows ann…

2023-02-08 Thread via GitHub
pabloem merged PR #25268: URL: https://github.com/apache/beam/pull/25268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.

[GitHub] [beam] pabloem commented on pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
pabloem commented on PR #25364: URL: https://github.com/apache/beam/pull/25364#issuecomment-1423632530 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] ragyibrahim commented on pull request #25325: Remove from _KNOWN_TABLES on 404 insert to allow table re-creation for Python SDK

2023-02-08 Thread via GitHub
ragyibrahim commented on PR #25325: URL: https://github.com/apache/beam/pull/25325#issuecomment-1423626173 @Abacn @ahmedabu98 Finally worked. See output below: ```bash tox -e py3-yapf .pkg: _optional_hooks> python /opt/homebrew/lib/python3.10/site-packages/pyproject_api/_bac

[GitHub] [beam] kamrankoupayi commented on pull request #25056: Beam/sdks/io/gcp/java/healcare/hl7v2 io read

2023-02-08 Thread via GitHub
kamrankoupayi commented on PR #25056: URL: https://github.com/apache/beam/pull/25056#issuecomment-1423610423 > Thanks, looks pretty good. Got a couple of minor comment. > > Also, the added integration test has a warning: > > https://ci-beam.apache.org/job/beam_PreCommit_Java_GC

[GitHub] [beam] github-actions[bot] commented on pull request #25396: Support gauge metrics in portable mode

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25396: URL: https://github.com/apache/beam/pull/25396#issuecomment-1423581389 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[GitHub] [beam] kamrankoupayi commented on a diff in pull request #25056: Beam/sdks/io/gcp/java/healcare/hl7v2 io read

2023-02-08 Thread via GitHub
kamrankoupayi commented on code in PR #25056: URL: https://github.com/apache/beam/pull/25056#discussion_r1100952509 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java: ## @@ -260,15 +266,12 @@ public static class Read extends

[GitHub] [beam] kamrankoupayi commented on a diff in pull request #25056: Beam/sdks/io/gcp/java/healcare/hl7v2 io read

2023-02-08 Thread via GitHub
kamrankoupayi commented on code in PR #25056: URL: https://github.com/apache/beam/pull/25056#discussion_r1100951987 ## sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IOTest.java: ## @@ -59,6 +61,30 @@ public void test_HL7v2IO_failedRe

[GitHub] [beam] github-actions[bot] commented on pull request #25395: Add min and max batch size args to Modelhandler used in example

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25395: URL: https://github.com/apache/beam/pull/25395#issuecomment-1423552768 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label python. Available commands: - `stop

[GitHub] [beam] github-actions[bot] commented on pull request #25394: Validate that GBK coders are always set correctly.

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25394: URL: https://github.com/apache/beam/pull/25394#issuecomment-1423528200 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] rohdesamuel commented on a diff in pull request #25354: PCollection data sampling for Java SDK harness #25064

2023-02-08 Thread via GitHub
rohdesamuel commented on code in PR #25354: URL: https://github.com/apache/beam/pull/25354#discussion_r1100893265 ## sdks/java/harness/src/main/java/org/apache/beam/fn/harness/debug/DataSampler.java: ## @@ -0,0 +1,173 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [beam] github-actions[bot] commented on pull request #25394: Validate that GBK coders are always set correctly.

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25394: URL: https://github.com/apache/beam/pull/25394#issuecomment-1423511360 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[GitHub] [beam] ragyibrahim commented on pull request #25325: Remove from _KNOWN_TABLES on 404 insert to allow table re-creation for Python SDK

2023-02-08 Thread via GitHub
ragyibrahim commented on PR #25325: URL: https://github.com/apache/beam/pull/25325#issuecomment-1423498250 @ahmedabu98 as you suggested i tried running the below but still got the same error ```bash pip install --no-binary :all: grpcio --ignore-installed pip install --no-binary

[GitHub] [beam] Abacn commented on pull request #25325: Remove from _KNOWN_TABLES on 404 insert to allow table re-creation for Python SDK

2023-02-08 Thread via GitHub
Abacn commented on PR #25325: URL: https://github.com/apache/beam/pull/25325#issuecomment-1423496431 Yeah it's a known issue. See this comment to fix: https://github.com/apache/beam/issues/22742#issuecomment-1218216468 @ahmedabu98 uses linux and it's fine. I have an M1 mac exper

[GitHub] [beam] Abacn commented on a diff in pull request #25056: Beam/sdks/io/gcp/java/healcare/hl7v2 io read

2023-02-08 Thread via GitHub
Abacn commented on code in PR #25056: URL: https://github.com/apache/beam/pull/25056#discussion_r1100880375 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java: ## @@ -260,15 +266,12 @@ public static class Read extends PTransfo

[GitHub] [beam] ragyibrahim commented on pull request #25325: Remove from _KNOWN_TABLES on 404 insert to allow table re-creation for Python SDK

2023-02-08 Thread via GitHub
ragyibrahim commented on PR #25325: URL: https://github.com/apache/beam/pull/25325#issuecomment-1423492870 hi, @Abacn, unfortunately, got the same error still. I've tried running the `gen_protos.py` file and I get the below error message (FYI im running an M2 mac) ```bash o

[GitHub] [beam] codecov[bot] commented on pull request #25395: Add min and max batch size args to Modelhandler used in example

2023-02-08 Thread via GitHub
codecov[bot] commented on PR #25395: URL: https://github.com/apache/beam/pull/25395#issuecomment-1423476108 # [Codecov](https://codecov.io/gh/apache/beam/pull/25395?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Fo

[GitHub] [beam] github-actions[bot] commented on pull request #25392: Read API Source v2

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25392: URL: https://github.com/apache/beam/pull/25392#issuecomment-1423475139 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] ryucc commented on pull request #25396: Support gauge metrics in portable mode

2023-02-08 Thread via GitHub
ryucc commented on PR #25396: URL: https://github.com/apache/beam/pull/25396#issuecomment-1423465958 @robertwb seems to be the author of the counter metrics and distribution metrics steps. Would be nice if you can take a look -- This is an automated message from the Apache Git Service. To

[GitHub] [beam] ryucc opened a new pull request, #25396: Support gauge metrics in portable mode

2023-02-08 Thread via GitHub
ryucc opened a new pull request, #25396: URL: https://github.com/apache/beam/pull/25396 Support gauge metrics in portable mode. The `getMonitoringInfos()` wraps the worker side metrics into `MonitoringInfos`, that is later returned to the runner though `ProcessBundleResponse`.

[GitHub] [beam] codecov[bot] commented on pull request #25393: Add WatchFilePattern

2023-02-08 Thread via GitHub
codecov[bot] commented on PR #25393: URL: https://github.com/apache/beam/pull/25393#issuecomment-1423464195 # [Codecov](https://codecov.io/gh/apache/beam/pull/25393?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Fo

[GitHub] [beam] AnandInguva opened a new pull request, #25395: Add min and max batch size args to Modelhandler used in example

2023-02-08 Thread via GitHub
AnandInguva opened a new pull request, #25395: URL: https://github.com/apache/beam/pull/25395 Follow up on #25370 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention t

[GitHub] [beam] robertwb commented on pull request #25394: Validate that GBK coders are always set correctly.

2023-02-08 Thread via GitHub
robertwb commented on PR #25394: URL: https://github.com/apache/beam/pull/25394#issuecomment-1423451567 R: @lukecwik -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [beam] robertwb opened a new pull request, #25394: Validate that GBK coders are always set correctly.

2023-02-08 Thread via GitHub
robertwb opened a new pull request, #25394: URL: https://github.com/apache/beam/pull/25394 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropriate issue i

[GitHub] [beam] AnandInguva commented on issue #25328: [Bug]: Overly restrictive Protobuf version in PyPI repository

2023-02-08 Thread via GitHub
AnandInguva commented on issue #25328: URL: https://github.com/apache/beam/issues/25328#issuecomment-1423450230 Hi, I am working on updating protobuf. I will tackle this issue as part of that effort(early next week). -- This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] AnandInguva opened a new pull request, #25393: Add WatchFilePattern

2023-02-08 Thread via GitHub
AnandInguva opened a new pull request, #25393: URL: https://github.com/apache/beam/pull/25393 WatchFilePattern Ptransform. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: -

[GitHub] [beam] xinyuiscool merged pull request #25265: Support samza portable UDF metrics.

2023-02-08 Thread via GitHub
xinyuiscool merged PR #25265: URL: https://github.com/apache/beam/pull/25265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apa

[GitHub] [beam] ahmedabu98 commented on a diff in pull request #25056: Beam/sdks/io/gcp/java/healcare/hl7v2 io read

2023-02-08 Thread via GitHub
ahmedabu98 commented on code in PR #25056: URL: https://github.com/apache/beam/pull/25056#discussion_r1100722820 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/healthcare/HL7v2ReadParameter.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache

[GitHub] [beam] vachan-shetty commented on pull request #25392: Read API Source v2

2023-02-08 Thread via GitHub
vachan-shetty commented on PR #25392: URL: https://github.com/apache/beam/pull/25392#issuecomment-1423389995 R: @johnjcasey, @lukecwik, @kmjung -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [beam] chamikaramj commented on a diff in pull request #25222: Update PythonMap transform to accept extra packages

2023-02-08 Thread via GitHub
chamikaramj commented on code in PR #25222: URL: https://github.com/apache/beam/pull/25222#discussion_r1100789920 ## sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/PythonMap.java: ## @@ -59,13 +63,34 @@ public PythonMap withExpansionS

[GitHub] [beam] Abacn commented on issue #25390: [Failing Test]: Many failing tests in Python v2.44.0 SDK

2023-02-08 Thread via GitHub
Abacn commented on issue #25390: URL: https://github.com/apache/beam/issues/25390#issuecomment-1423372802 `@pytest.mark.no_xdist` is just a marker. If invoke pytest command directly it does not take effect. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [beam] Abacn commented on issue #25355: [Bug]: BigQuery BatchLoad incompatible table schema error

2023-02-08 Thread via GitHub
Abacn commented on issue #25355: URL: https://github.com/apache/beam/issues/25355#issuecomment-1423372093 The input of UpdateSchemaDestination should be KV> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [beam] Abacn commented on issue #25355: [Bug]: BigQuery BatchLoad incompatible table schema error

2023-02-08 Thread via GitHub
Abacn commented on issue #25355: URL: https://github.com/apache/beam/issues/25355#issuecomment-1423371153 I think I find the cause of the original issue (that in the issue description): https://github.com/apache/beam/blob/9fcb3a5b48a6db0dcf57f454d1d1eca10cf1c41b/sdks/java/io/google-c

[GitHub] [beam] github-actions[bot] commented on pull request #25389: Add a note on increased lower bounds.

2023-02-08 Thread via GitHub
github-actions[bot] commented on PR #25389: URL: https://github.com/apache/beam/pull/25389#issuecomment-1423366808 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] reuvenlax commented on pull request #25268: Adding support for @SchemaFieldDescription annotation that allows ann…

2023-02-08 Thread via GitHub
reuvenlax commented on PR #25268: URL: https://github.com/apache/beam/pull/25268#issuecomment-1423365902 Run Java_HBase_IO_Direct PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423364428 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] samuela commented on issue #25390: [Failing Test]: Many failing tests in Python v2.44.0 SDK

2023-02-08 Thread via GitHub
samuela commented on issue #25390: URL: https://github.com/apache/beam/issues/25390#issuecomment-1423359916 `@pytest.mark.no_xdist` should cause them to be skipped, no? FWIW I tried running without xdist and saw similar test failures in `apache_beam/coders/coders_test_common.py`, so I'm not

[GitHub] [beam] pabloem commented on pull request #24999: Adding SpannerIO.readChangeStreams support for SchemaTransform

2023-02-08 Thread via GitHub
pabloem commented on PR #24999: URL: https://github.com/apache/beam/pull/24999#issuecomment-1423355369 Run Java_GCP_IO_Direct PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [beam] vachan-shetty opened a new pull request, #25392: Read API Source v2

2023-02-08 Thread via GitHub
vachan-shetty opened a new pull request, #25392: URL: https://github.com/apache/beam/pull/25392 Adding the new Read API Source which supports bundling of Streams within Read Session. Customers can enable/disable this feature using the `setEnableBundling` BigQuery pipeline option (the option

[GitHub] [beam] Abacn commented on issue #25390: [Failing Test]: Many failing tests in Python v2.44.0 SDK

2023-02-08 Thread via GitHub
Abacn commented on issue #25390: URL: https://github.com/apache/beam/issues/25390#issuecomment-1423344439 Running these tests in parallel will cause this error. Reason: https://github.com/apache/beam/blob/9fcb3a5b48a6db0dcf57f454d1d1eca10cf1c41b/sdks/python/apache_beam/coders/coders_test_com

[GitHub] [beam] pabloem commented on pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
pabloem commented on PR #25364: URL: https://github.com/apache/beam/pull/25364#issuecomment-1423343636 Run Java PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] pabloem commented on pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
pabloem commented on PR #25364: URL: https://github.com/apache/beam/pull/25364#issuecomment-1423338476 Run Java_GCP_IO_Direct PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [beam] tonytanger commented on a diff in pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
tonytanger commented on code in PR #25364: URL: https://github.com/apache/beam/pull/25364#discussion_r1100757973 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/InitializeDoFn.java: ## @@ -53,6 +52,25 @@ public void pro

[GitHub] [beam] tvalentyn commented on pull request #25242: Add runtime metric to TFT tests

2023-02-08 Thread via GitHub
tvalentyn commented on PR #25242: URL: https://github.com/apache/beam/pull/25242#issuecomment-1423331372 (i also re-ran the seed job) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [beam] tvalentyn commented on pull request #25242: Add runtime metric to TFT tests

2023-02-08 Thread via GitHub
tvalentyn commented on PR #25242: URL: https://github.com/apache/beam/pull/25242#issuecomment-1423330935 Run TFT Criteo Benchmarks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [beam] tvalentyn commented on pull request #17784: [DO NOT MERGE] Run seed job on open PRs

2023-02-08 Thread via GitHub
tvalentyn commented on PR #17784: URL: https://github.com/apache/beam/pull/17784#issuecomment-1423318249 Run Seed Job -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [beam] lostluck opened a new pull request, #25391: [#24789][prism] Big Initial PR

2023-02-08 Thread via GitHub
lostluck opened a new pull request, #25391: URL: https://github.com/apache/beam/pull/25391 Initial version of the Go Direct Runner Replacement : Prism. See the readmes and #24789 for more information. Thank you for your contribution! Follow this c

[GitHub] [beam] tvalentyn merged pull request #25389: Add a note on increased lower bounds.

2023-02-08 Thread via GitHub
tvalentyn merged PR #25389: URL: https://github.com/apache/beam/pull/25389 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apach

[GitHub] [beam] Abacn commented on issue #22372: [Bug]: PR 17365 breaks BQ loads in some cases

2023-02-08 Thread via GitHub
Abacn commented on issue #22372: URL: https://github.com/apache/beam/issues/22372#issuecomment-1423311832 Decide to reopen this issue because there is still more than one breakage remain #25355 - incompatible schema error in copy when dynamicDestination - check equality between

[GitHub] [beam] steveniemitz opened a new issue, #22372: [Bug]: PR 17365 breaks BQ loads in some cases

2023-02-08 Thread via GitHub
steveniemitz opened a new issue, #22372: URL: https://github.com/apache/beam/issues/22372 ### What happened? In beam 2.39, this used to work: ``` BigQueryIO   .write()   .optimizedWrites()   .to(OutputTable)   .withAvroFormatFunction(abc)   .withAvroSchemaFact

[GitHub] [beam] samuela commented on issue #25390: [Failing Test]: Many failing tests in Python v2.44.0 SDK

2023-02-08 Thread via GitHub
samuela commented on issue #25390: URL: https://github.com/apache/beam/issues/25390#issuecomment-1423290139 See https://gist.github.com/samuela/87b4c7fdcabcc3992f7f7ac2fc21b9ba for a complete log of all the failures in their full pytest verbosity. -- This is an automated message from the

[GitHub] [beam] damccorm commented on a diff in pull request #25368: [Python] Added Tensorflow Model Handler

2023-02-08 Thread via GitHub
damccorm commented on code in PR #25368: URL: https://github.com/apache/beam/pull/25368#discussion_r1100711310 ## sdks/python/apache_beam/ml/inference/tensorflow_inference.py: ## @@ -0,0 +1,245 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contri

[GitHub] [beam] samuela opened a new issue, #25390: [Failing Test]: Many failing tests in Python v2.44.0 SDK

2023-02-08 Thread via GitHub
samuela opened a new issue, #25390: URL: https://github.com/apache/beam/issues/25390 ### What happened? I'm seeing the following test suite results when running the v2.44.0 Python tests: ``` === short test summary info FAI

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423275173 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] tvalentyn commented on pull request #25389: Add a note on increased lower bounds.

2023-02-08 Thread via GitHub
tvalentyn commented on PR #25389: URL: https://github.com/apache/beam/pull/25389#issuecomment-1423274232 R: @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [beam] tvalentyn opened a new pull request, #25389: Add a note on increased lower bounds.

2023-02-08 Thread via GitHub
tvalentyn opened a new pull request, #25389: URL: https://github.com/apache/beam/pull/25389 Follow up for #24725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[GitHub] [beam] AnandInguva commented on a diff in pull request #25368: [Python] Added Tensorflow Model Handler

2023-02-08 Thread via GitHub
AnandInguva commented on code in PR #25368: URL: https://github.com/apache/beam/pull/25368#discussion_r1100706014 ## sdks/python/apache_beam/ml/inference/tensorflow_inference.py: ## @@ -0,0 +1,226 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# con

[GitHub] [beam] damccorm merged pull request #25370: Support batching as config in RunInference (sklearn and pytorch)

2023-02-08 Thread via GitHub
damccorm merged PR #25370: URL: https://github.com/apache/beam/pull/25370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423253358 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] tonytanger commented on a diff in pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
tonytanger commented on code in PR #25364: URL: https://github.com/apache/beam/pull/25364#discussion_r1100685551 ## sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java: ## @@ -124,9 +130,7 @@ public void testGcpApiSurface() throws E

[GitHub] [beam] pabloem commented on pull request #24999: Adding SpannerIO.readChangeStreams support for SchemaTransform

2023-02-08 Thread via GitHub
pabloem commented on PR #24999: URL: https://github.com/apache/beam/pull/24999#issuecomment-1423245685 Run Java_GCP_IO_Direct PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [beam] pabloem commented on a diff in pull request #25364: Create metadata table in initialize stage if table doesn't exist.

2023-02-08 Thread via GitHub
pabloem commented on code in PR #25364: URL: https://github.com/apache/beam/pull/25364#discussion_r1100679077 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigtable/changestreams/dofn/InitializeDoFn.java: ## @@ -53,6 +52,25 @@ public void proces

[GitHub] [beam] damccorm commented on a diff in pull request #25370: Support batching as config in RunInference (sklearn and pytorch)

2023-02-08 Thread via GitHub
damccorm commented on code in PR #25370: URL: https://github.com/apache/beam/pull/25370#discussion_r1100683036 ## sdks/python/apache_beam/ml/inference/sklearn_inference_test.py: ## @@ -338,6 +372,42 @@ def test_pipeline_pandas(self): assert_that( actual, equal_

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423238097 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] AnandInguva commented on pull request #25242: Add runtime metric to TFT tests

2023-02-08 Thread via GitHub
AnandInguva commented on PR #25242: URL: https://github.com/apache/beam/pull/25242#issuecomment-1423226347 The pipelines execute successfully. The error happens with BigQueryClient ![image](https://user-images.githubusercontent.com/34158215/217648717-85aa3de7-3c4f-41b4-8500-667c1bcc36

[GitHub] [beam] AnandInguva closed pull request #25109: Model updates using side inputs

2023-02-08 Thread via GitHub
AnandInguva closed pull request #25109: Model updates using side inputs URL: https://github.com/apache/beam/pull/25109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423212972 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] liferoad commented on a diff in pull request #25373: Raise the RuntimeError message when some InteractiveRunner functions are misused

2023-02-08 Thread via GitHub
liferoad commented on code in PR #25373: URL: https://github.com/apache/beam/pull/25373#discussion_r1100645921 ## sdks/python/apache_beam/runners/interactive/pipeline_fragment.py: ## @@ -23,6 +23,7 @@ from apache_beam.pipeline import AppliedPTransform from apache_beam.pipeline

[GitHub] [beam] AnandInguva opened a new issue, #25388: [Task]: Add 1 TB Criteo benchmark tests

2023-02-08 Thread via GitHub
AnandInguva opened a new issue, #25388: URL: https://github.com/apache/beam/issues/25388 ### What needs to happen? We have small and medium(10 GB) TFT Criteo benchmark tests. This is a tracking issue to add 1 TB TFT Criteo benchmark tests. ### Issue Priority Priority: 3

[GitHub] [beam] Abacn commented on issue #25355: [Bug]: BigQuery BatchLoad incompatible table schema error

2023-02-08 Thread via GitHub
Abacn commented on issue #25355: URL: https://github.com/apache/beam/issues/25355#issuecomment-1423190926 As @ahmedabu98 pointed out the original working example has typo. Initiated another job 2023-02-08_11_47_55-389031392081500435 branch: https://github.com/apache/beam/commit/f446e5c13667

[GitHub] [beam] kennknowles commented on pull request #25055: Remove trailing whitespace from blog to fix precommit

2023-02-08 Thread via GitHub
kennknowles commented on PR #25055: URL: https://github.com/apache/beam/pull/25055#issuecomment-1423170940 Just to answer my own question, yes: this PR is not equivalent to what it replaced, but adds a lot of whitespace to the blog post. It reads well anyhow, but wanted to be clear that thi

[GitHub] [beam] tvalentyn commented on a diff in pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
tvalentyn commented on code in PR #24725: URL: https://github.com/apache/beam/pull/24725#discussion_r1100591921 ## sdks/python/setup.py: ## @@ -343,15 +345,12 @@ def get_portability_package_data(): 'azure-core >=1.7.0', 'azure-identity >=1.12.0',

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423149234 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] Abacn commented on pull request #23785: [Test only][Do not merge] Draft PR for testing remote runs

2023-02-08 Thread via GitHub
Abacn commented on PR #23785: URL: https://github.com/apache/beam/pull/23785#issuecomment-1423145329 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] tvalentyn merged pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
tvalentyn merged PR #24725: URL: https://github.com/apache/beam/pull/24725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apach

[GitHub] [beam] tvalentyn commented on a diff in pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
tvalentyn commented on code in PR #24725: URL: https://github.com/apache/beam/pull/24725#discussion_r1100597737 ## sdks/python/apache_beam/ml/inference/tensorrt_inference.py: ## @@ -122,7 +122,16 @@ def __init__(self, engine: trt.ICudaEngine): self.outputs = [] self.gp

[GitHub] [beam] damccorm commented on a diff in pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
damccorm commented on code in PR #24725: URL: https://github.com/apache/beam/pull/24725#discussion_r1100597316 ## sdks/python/setup.py: ## @@ -343,15 +345,12 @@ def get_portability_package_data(): 'azure-core >=1.7.0', 'azure-identity >=1.12.0',

[GitHub] [beam] tvalentyn opened a new issue, #25387: [Task]: Cleanup monkey-patching of numpy once TensorRT supports numpy 1.24.0.

2023-02-08 Thread via GitHub
tvalentyn opened a new issue, #25387: URL: https://github.com/apache/beam/issues/25387 ### What needs to happen? 1) Wait until https://github.com/NVIDIA/TensorRT/issues/2557 is fixed 2) Remove monkey-patching in tensorrt_inference.py introduced in https://github.com/apache/beam/pul

[GitHub] [beam] tvalentyn commented on a diff in pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
tvalentyn commented on code in PR #24725: URL: https://github.com/apache/beam/pull/24725#discussion_r1100591921 ## sdks/python/setup.py: ## @@ -343,15 +345,12 @@ def get_portability_package_data(): 'azure-core >=1.7.0', 'azure-identity >=1.12.0',

[GitHub] [beam] reuvenlax commented on pull request #25268: Adding support for @SchemaFieldDescription annotation that allows ann…

2023-02-08 Thread via GitHub
reuvenlax commented on PR #25268: URL: https://github.com/apache/beam/pull/25268#issuecomment-1423137174 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [beam] sultanalieva-s commented on pull request #25365: Refactoring: Moved hard coded environment variables' names into constants.

2023-02-08 Thread via GitHub
sultanalieva-s commented on PR #25365: URL: https://github.com/apache/beam/pull/25365#issuecomment-1423125698 .retest -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] sultanalieva-s commented on pull request #25365: Refactoring: Moved hard coded environment variables' names into constants.

2023-02-08 Thread via GitHub
sultanalieva-s commented on PR #25365: URL: https://github.com/apache/beam/pull/25365#issuecomment-1423121511 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [beam] codecov[bot] commented on pull request #25386: (No need to review) Testing grpc 1.45.4 on previous version 2.40.0

2023-02-08 Thread via GitHub
codecov[bot] commented on PR #25386: URL: https://github.com/apache/beam/pull/25386#issuecomment-1423113526 # [Codecov](https://codecov.io/gh/apache/beam/pull/25386?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Fo

[GitHub] [beam] Abacn merged pull request #25369: Explicitly set mongo container version for testing

2023-02-08 Thread via GitHub
Abacn merged PR #25369: URL: https://github.com/apache/beam/pull/25369 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.or

[GitHub] [beam] Abacn closed issue #25359: [Failing Test]: Python MongoDB integration test failing

2023-02-08 Thread via GitHub
Abacn closed issue #25359: [Failing Test]: Python MongoDB integration test failing URL: https://github.com/apache/beam/issues/25359 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [beam] ahmedabu98 commented on issue #25355: [Bug]: BigQuery BatchLoad incompatible table schema error

2023-02-08 Thread via GitHub
ahmedabu98 commented on issue #25355: URL: https://github.com/apache/beam/issues/25355#issuecomment-1423106192 .take-issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [beam] Abacn commented on pull request #25325: Remove from _KNOWN_TABLES on 404 insert to allow table re-creation for Python SDK

2023-02-08 Thread via GitHub
Abacn commented on PR #25325: URL: https://github.com/apache/beam/pull/25325#issuecomment-1423087776 I never got tox yapf working locally. I am doing this to fix lint: ``` # Run from root beam repo dir pip install yapf==0.29.0 git diff master --name-only | grep "\.py$" | xargs

[GitHub] [beam] burkedavison commented on pull request #25386: (No need to review) Testing grpc 1.45.4 on previous version 2.40.0

2023-02-08 Thread via GitHub
burkedavison commented on PR #25386: URL: https://github.com/apache/beam/pull/25386#issuecomment-1423080913 Run Java_Spark3_Versions PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] burkedavison opened a new pull request, #25386: (No need to review) Testing grpc 1.45.4 on previous version 2.40.0

2023-02-08 Thread via GitHub
burkedavison opened a new pull request, #25386: URL: https://github.com/apache/beam/pull/25386 We are testing compatibility with gRPC 1.45.4 with Beam 2.40.0 This PR is just to check the CI status. **This PR does not need to be merged.** Thank you fo

[GitHub] [beam] riteshghorse commented on a diff in pull request #25368: [Python] Added Tensorflow Model Handler

2023-02-08 Thread via GitHub
riteshghorse commented on code in PR #25368: URL: https://github.com/apache/beam/pull/25368#discussion_r1100532371 ## sdks/python/apache_beam/ml/inference/tensorflow_inference.py: ## @@ -0,0 +1,226 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# co

[GitHub] [beam] tvalentyn commented on a diff in pull request #25370: Support batching as config in RunInference (sklearn and pytorch)

2023-02-08 Thread via GitHub
tvalentyn commented on code in PR #25370: URL: https://github.com/apache/beam/pull/25370#discussion_r1100531179 ## sdks/python/apache_beam/ml/inference/sklearn_inference_test.py: ## @@ -338,6 +372,42 @@ def test_pipeline_pandas(self): assert_that( actual, equal

[GitHub] [beam] Abacn commented on a diff in pull request #25369: Explicitly set mongo container version for testing

2023-02-08 Thread via GitHub
Abacn commented on code in PR #25369: URL: https://github.com/apache/beam/pull/25369#discussion_r1100529085 ## build.gradle.kts: ## @@ -454,7 +454,6 @@ tasks.register("python37PostCommit") { dependsOn(":sdks:python:test-suites:direct:py37:directRunnerIT") dependsOn(":sdks:

[GitHub] [beam] Abacn commented on a diff in pull request #25369: Explicitly set mongo container version for testing

2023-02-08 Thread via GitHub
Abacn commented on code in PR #25369: URL: https://github.com/apache/beam/pull/25369#discussion_r1100529085 ## build.gradle.kts: ## @@ -454,7 +454,6 @@ tasks.register("python37PostCommit") { dependsOn(":sdks:python:test-suites:direct:py37:directRunnerIT") dependsOn(":sdks:

[GitHub] [beam] damccorm commented on a diff in pull request #24725: Update upper bound for numpy.

2023-02-08 Thread via GitHub
damccorm commented on code in PR #24725: URL: https://github.com/apache/beam/pull/24725#discussion_r1100519220 ## sdks/python/apache_beam/ml/inference/tensorrt_inference.py: ## @@ -122,7 +122,16 @@ def __init__(self, engine: trt.ICudaEngine): self.outputs = [] self.gpu

[GitHub] [beam] ahmedabu98 commented on a diff in pull request #25369: Explicitly set mongo container version for testing

2023-02-08 Thread via GitHub
ahmedabu98 commented on code in PR #25369: URL: https://github.com/apache/beam/pull/25369#discussion_r1100524800 ## build.gradle.kts: ## @@ -454,7 +454,6 @@ tasks.register("python37PostCommit") { dependsOn(":sdks:python:test-suites:direct:py37:directRunnerIT") dependsOn(":

[GitHub] [beam] dependabot[bot] commented on pull request #25375: Bump cryptography from 39.0.0 to 39.0.1 in /sdks/python/container/py38

2023-02-08 Thread via GitHub
dependabot[bot] commented on PR #25375: URL: https://github.com/apache/beam/pull/25375#issuecomment-1423058245 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let

  1   2   >