[GitHub] [beam] TommyCpp commented on issue #21089: Rust SDK

2022-12-28 Thread GitBox
TommyCpp commented on issue #21089: URL: https://github.com/apache/beam/issues/21089#issuecomment-1367123079 I also like to help! I have some experience with Rust but am pretty new to Beam or large-scale data processing framework -- This is an automated message from the Apache Git Service

[GitHub] [beam] alxp1982 commented on a diff in pull request #24488: add schema-based trnasforms

2022-12-28 Thread GitBox
alxp1982 commented on code in PR #24488: URL: https://github.com/apache/beam/pull/24488#discussion_r1058641412 ## learning/tour-of-beam/learning-content/java/schema-based-transforms/co-group/description.md: ## @@ -0,0 +1,88 @@ + + +# CoGroup + +A transform that performs equijoin

[GitHub] [beam] mosche closed issue #23750: [Task]: Remove {runner}PreCommit tasks from :examples:java

2022-12-28 Thread GitBox
mosche closed issue #23750: [Task]: Remove {runner}PreCommit tasks from :examples:java URL: https://github.com/apache/beam/issues/23750 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [beam] mosche commented on issue #23750: [Task]: Remove {runner}PreCommit tasks from :examples:java

2022-12-28 Thread GitBox
mosche commented on issue #23750: URL: https://github.com/apache/beam/issues/23750#issuecomment-1367116627 Invalid, still run as part of `javaPreCommit` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] mosche opened a new pull request, #24810: [Spark Dataset runner] Fix initialization of metrics accumulator on driver

2022-12-28 Thread GitBox
mosche opened a new pull request, #24810: URL: https://github.com/apache/beam/pull/24810 Fix initialization of metrics accumulator on driver (#24809). This PR fixes all related APIs to prevent similar bugs in the future. Thank you for your contributio

[GitHub] [beam] Miuler commented on pull request #23610: feat(azure-cosmosdb): Add support for cosmosdb with Core (SQL) API

2022-12-28 Thread GitBox
Miuler commented on PR #23610: URL: https://github.com/apache/beam/pull/23610#issuecomment-1367104756 @kennknowles What would I need to continue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] mosche commented on issue #24035: [Bug]: Improve handling of side inputs in Spark dataset runner

2022-12-28 Thread GitBox
mosche commented on issue #24035: URL: https://github.com/apache/beam/issues/24035#issuecomment-1367103287 Closed by https://github.com/apache/beam/pull/24288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [beam] mosche closed issue #24035: [Bug]: Improve handling of side inputs in Spark dataset runner

2022-12-28 Thread GitBox
mosche closed issue #24035: [Bug]: Improve handling of side inputs in Spark dataset runner URL: https://github.com/apache/beam/issues/24035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [beam] github-actions[bot] commented on pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24806: URL: https://github.com/apache/beam/pull/24806#issuecomment-1367051925 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kennknowles for label java. R: @Abacn for label build. R: @

[GitHub] [beam] damondouglas commented on pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
damondouglas commented on PR #24806: URL: https://github.com/apache/beam/pull/24806#issuecomment-1367051601 assign set of reviewers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] zhengbuqian commented on pull request #23492: Add Windmill support for MultimapState

2022-12-28 Thread GitBox
zhengbuqian commented on PR #23492: URL: https://github.com/apache/beam/pull/23492#issuecomment-1367032098 Gentle ping R: @reuvenlax @lukecwik @y1chi @scwhittle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [beam] Abacn commented on a diff in pull request #24204: Shard python precommit

2022-12-28 Thread GitBox
Abacn commented on code in PR #24204: URL: https://github.com/apache/beam/pull/24204#discussion_r1058662550 ## .test-infra/jenkins/job_PreCommit_Python.groovy: ## @@ -22,6 +22,9 @@ PrecommitJobBuilder builder = new PrecommitJobBuilder( scope: this, nameBase: 'Python',

[GitHub] [beam] github-actions[bot] commented on pull request #24808: [#24789][Go SDK] Fix Minor race conditions

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24808: URL: https://github.com/apache/beam/pull/24808#issuecomment-1366993420 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @damccorm for label go. Available commands: - `stop revi

[GitHub] [beam] Abacn commented on a diff in pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
Abacn commented on code in PR #24803: URL: https://github.com/apache/beam/pull/24803#discussion_r1058656898 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java: ## @@ -506,9 +506,9 @@ public abstract List listSubscriptions(Pro

[GitHub] [beam] Abacn commented on a diff in pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
Abacn commented on code in PR #24803: URL: https://github.com/apache/beam/pull/24803#discussion_r1058656898 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java: ## @@ -506,9 +506,9 @@ public abstract List listSubscriptions(Pro

[GitHub] [beam] Abacn commented on a diff in pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
Abacn commented on code in PR #24803: URL: https://github.com/apache/beam/pull/24803#discussion_r1058656898 ## sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubClient.java: ## @@ -506,9 +506,9 @@ public abstract List listSubscriptions(Pro

[GitHub] [beam] lukecwik commented on pull request #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
lukecwik commented on PR #24805: URL: https://github.com/apache/beam/pull/24805#issuecomment-1366986761 R: @bvolpato -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [beam] codecov[bot] commented on pull request #24808: [#24789][Go SDK] Fix Minor race conditions

2022-12-28 Thread GitBox
codecov[bot] commented on PR #24808: URL: https://github.com/apache/beam/pull/24808#issuecomment-1366982230 # [Codecov](https://codecov.io/gh/apache/beam/pull/24808?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Fo

[GitHub] [beam] github-actions[bot] commented on pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24806: URL: https://github.com/apache/beam/pull/24806#issuecomment-1366981304 Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers` -- This is an automated me

[GitHub] [beam] codecov[bot] commented on pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
codecov[bot] commented on PR #24806: URL: https://github.com/apache/beam/pull/24806#issuecomment-1366981043 # [Codecov](https://codecov.io/gh/apache/beam/pull/24806?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Fo

[GitHub] [beam] lostluck opened a new pull request, #24808: [#24789][Go SDK] Fix Minor race conditions

2022-12-28 Thread GitBox
lostluck opened a new pull request, #24808: URL: https://github.com/apache/beam/pull/24808 Resolving SDK side race conditions that lead to flaky or hung unit tests while I've been iterating on #24789 These can cause the unit test runs to hand or fail spuriously. * Logger can sometime

[GitHub] [beam] Abacn commented on pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
Abacn commented on PR #24803: URL: https://github.com/apache/beam/pull/24803#issuecomment-1366978774 Run PostCommit_Java_Dataflow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [beam] Abacn commented on pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
Abacn commented on PR #24803: URL: https://github.com/apache/beam/pull/24803#issuecomment-1366978796 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] github-actions[bot] commented on pull request #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24805: URL: https://github.com/apache/beam/pull/24805#issuecomment-1366973548 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] lukecwik commented on pull request #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
lukecwik commented on PR #24805: URL: https://github.com/apache/beam/pull/24805#issuecomment-1366973042 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [beam] lukecwik commented on issue #24802: [Failing Test]: org.apache.beam.sdk.io.gcp.pubsub.PubsubSchemaIT.classMethod in beam_PostCommit_Java_DataflowV1 and V2

2022-12-28 Thread GitBox
lukecwik commented on issue #24802: URL: https://github.com/apache/beam/issues/24802#issuecomment-1366972109 Can we disable the test if the fix is not obvious? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [beam] damondouglas opened a new issue, #24807: [Bug]: Playground Examples CI / check (java) / pre-check (pull_request)

2022-12-28 Thread GitBox
damondouglas opened a new issue, #24807: URL: https://github.com/apache/beam/issues/24807 ### What happened? The Playground Examples CI / check (java) / pre-check (pull_request) fails when changing the names of files. For example, https://github.com/apache/beam/actions/runs/379703

[GitHub] [beam] damondouglas commented on a diff in pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
damondouglas commented on code in PR #24806: URL: https://github.com/apache/beam/pull/24806#discussion_r1058640533 ## sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapter.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apac

[GitHub] [beam] damondouglas commented on a diff in pull request #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
damondouglas commented on code in PR #24806: URL: https://github.com/apache/beam/pull/24806#discussion_r1058640491 ## sdks/java/io/file-schema-transform/src/main/java/org/apache/beam/sdk/io/fileschematransform/XmlRowAdapter.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apac

[GitHub] [beam] damondouglas opened a new pull request, #24806: 24472 Implement FileWriteSchemaTransformProvider

2022-12-28 Thread GitBox
damondouglas opened a new pull request, #24806: URL: https://github.com/apache/beam/pull/24806 This PR closes #24472 by implementing a FileWriteSchemaTransformProvider supporting configuration driven file writes in AVRO, JSON, Parquet, and XML formats. Please see #24472 for more details.

[GitHub] [beam] lukecwik commented on pull request #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
lukecwik commented on PR #24805: URL: https://github.com/apache/beam/pull/24805#issuecomment-1366964449 Run PostCommit_Java_DataflowV2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [beam] lukecwik commented on pull request #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
lukecwik commented on PR #24805: URL: https://github.com/apache/beam/pull/24805#issuecomment-1366964406 Run PostCommit_Java_Dataflow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [beam] lukecwik opened a new pull request, #24805: [#24515] Ensure that portable Java pipelines on Dataflow are not able to opt out of runner v2.

2022-12-28 Thread GitBox
lukecwik opened a new pull request, #24805: URL: https://github.com/apache/beam/pull/24805 Fixes #24515 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Mention the appropr

[GitHub] [beam] github-actions[bot] commented on pull request #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24803: URL: https://github.com/apache/beam/pull/24803#issuecomment-1366951363 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @apilloud for label java. R: @pabloem for label io. Avai

[GitHub] [beam] github-actions[bot] commented on pull request #24804: [WebSite] Add new Python quickstart

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24804: URL: https://github.com/apache/beam/pull/24804#issuecomment-1366933085 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] VeronicaWasson commented on pull request #24804: [WebSite] Add new Python quickstart

2022-12-28 Thread GitBox
VeronicaWasson commented on PR #24804: URL: https://github.com/apache/beam/pull/24804#issuecomment-1366932569 R: @tvalentyn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [beam] VeronicaWasson opened a new pull request, #24804: [WebSite] Add new Python quickstart

2022-12-28 Thread GitBox
VeronicaWasson opened a new pull request, #24804: URL: https://github.com/apache/beam/pull/24804 Adds a new quickstart for Python based on https://github.com/apache/beam-starter-python Following #22747, this PR keeps the original Python quickstart but renames it to "WordCount sample

[GitHub] [beam] damondouglas opened a new pull request, #24803: 24802 fix missing test resources

2022-12-28 Thread GitBox
damondouglas opened a new pull request, #24803: URL: https://github.com/apache/beam/pull/24803 This PR closes #24802 by removing the test dependency on resource files. It refactors the Pub/Sub schema creation method by receiving the full content of the definition instead of a resource path

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366910134 ready for review now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366909756 Test passed: https://ci-beam.apache.org/job/beam_PostCommit_Java_InfluxDbIO_IT_PR/1/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [beam] damondouglas commented on issue #24802: [Failing Test]: org.apache.beam.sdk.io.gcp.pubsub.PubsubSchemaIT.classMethod in beam_PostCommit_Java_DataflowV1 and V2

2022-12-28 Thread GitBox
damondouglas commented on issue #24802: URL: https://github.com/apache/beam/issues/24802#issuecomment-1366895497 .take-issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366894732 Run Java InfluxDbIO_IT -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [beam-starter-kotlin] dependabot[bot] closed pull request #12: Bump org.jetbrains.kotlin.jvm from 1.7.20 to 1.7.22

2022-12-28 Thread GitBox
dependabot[bot] closed pull request #12: Bump org.jetbrains.kotlin.jvm from 1.7.20 to 1.7.22 URL: https://github.com/apache/beam-starter-kotlin/pull/12 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [beam-starter-kotlin] dependabot[bot] commented on pull request #12: Bump org.jetbrains.kotlin.jvm from 1.7.20 to 1.7.22

2022-12-28 Thread GitBox
dependabot[bot] commented on PR #12: URL: https://github.com/apache/beam-starter-kotlin/pull/12#issuecomment-1366892807 Superseded by #14. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [beam-starter-kotlin] dependabot[bot] commented on pull request #14: Bump org.jetbrains.kotlin.jvm from 1.7.20 to 1.8.0

2022-12-28 Thread GitBox
dependabot[bot] commented on PR #14: URL: https://github.com/apache/beam-starter-kotlin/pull/14#issuecomment-1366892796 Dependabot tried to add `@tosun-si`, `@lukecwik`, `@kennknowles`, `@robertwb` and `@kileys` as reviewers to this PR, but received the following error from GitHub:

[GitHub] [beam-starter-kotlin] dependabot[bot] opened a new pull request, #14: Bump org.jetbrains.kotlin.jvm from 1.7.20 to 1.8.0

2022-12-28 Thread GitBox
dependabot[bot] opened a new pull request, #14: URL: https://github.com/apache/beam-starter-kotlin/pull/14 Bumps [org.jetbrains.kotlin.jvm](https://github.com/JetBrains/kotlin) from 1.7.20 to 1.8.0. Release notes Sourced from https://github.com/JetBrains/kotlin/releases";>org.jetbr

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366889789 run seed job -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubsc

[GitHub] [beam] github-actions[bot] commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-136682 Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`: R: @kileys for label java. R: @ahmedabu98 for label io. Ava

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366885409 Nevertheless this test appears to be the only job exercises InfluxDBIOIT (InfluxDb_IO_Direct only runs unit tests). Preserve it and rename as PostCommit -- This is an automated message fro

[GitHub] [beam] Abacn commented on pull request #24633: Add Python xlang KafkaIO performance test

2022-12-28 Thread GitBox
Abacn commented on PR #24633: URL: https://github.com/apache/beam/pull/24633#issuecomment-1366883064 Issue related to #20806 (remove `shuffle_mode=appliance` flag) will be dealed in separate PR once that issue is clarified. @johnjcasey PTAL, thanks! -- This is an automated message from

[GitHub] [beam] Abacn opened a new issue, #24802: [Failing Test]: org.apache.beam.sdk.io.gcp.pubsub.PubsubSchemaIT.classMethod in beam_PostCommit_Java_DataflowV1 and V2

2022-12-28 Thread GitBox
Abacn opened a new issue, #24802: URL: https://github.com/apache/beam/issues/24802 ### What happened? error message: ``` org.apache.beam.sdk.io.gcp.pubsub.PubsubSchemaIT.classMethod Error Message java.nio.file.NoSuchFileException: file:/home/jenkins/jenkins-slave/works

[GitHub] [beam] Abacn commented on pull request #24443: Implement PubsubWriteSchemaTransformProvider

2022-12-28 Thread GitBox
Abacn commented on PR #24443: URL: https://github.com/apache/beam/pull/24443#issuecomment-1366879917 Looks like this PR broke https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV1 and https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2 . Error message: ``` org.apac

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366875492 Well, apparently "influxdbio performance test" is not a true performance test. It is an integration test running on direct runner actually. -- This is an automated message from the Apache

[GitHub] [beam] jlentin opened a new issue, #24801: [Bug]: ReadableState.read() marked not nullable, but it does return null when not initialized

2022-12-28 Thread GitBox
jlentin opened a new issue, #24801: URL: https://github.com/apache/beam/issues/24801 ### What happened? We are trying to upgrade version of Beam from version `2.31.0` to `2.43.0` While upgrading, we noticed that `@Nullable` annotation has been removed from `org.apache.beam.sdk.

[GitHub] [beam] Miuler commented on a diff in pull request #23610: feat(azure-cosmosdb): Add support for cosmosdb with Core (SQL) API

2022-12-28 Thread GitBox
Miuler commented on code in PR #23610: URL: https://github.com/apache/beam/pull/23610#discussion_r1058539985 ## CHANGES.md: ## @@ -96,6 +96,7 @@ * S3 implementation of the Beam filesystem (Go) ([#23991](https://github.com/apache/beam/issues/23991)). * Support for SingleStoreD

[GitHub] [beam] Abacn commented on pull request #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn commented on PR #24800: URL: https://github.com/apache/beam/pull/24800#issuecomment-1366861154 Run Java InfluxDbIO Performance Test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [beam] Abacn opened a new pull request, #24800: Retry create database in InfluxDbIOIT

2022-12-28 Thread GitBox
Abacn opened a new pull request, #24800: URL: https://github.com/apache/beam/pull/24800 This should resolve the initial issue raised in #23286 (though https://github.com/apache/beam/issues/23286#issuecomment-1341548232 needs further investigation) **Please** add a meaningful descript

[GitHub] [beam] robertwb merged pull request #24735: Better naming and inference for PythonCallable.

2022-12-28 Thread GitBox
robertwb merged PR #24735: URL: https://github.com/apache/beam/pull/24735 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] johnjcasey opened a new pull request, #24799: Create an Example IO to pair with the How to write an IO guide

2022-12-28 Thread GitBox
johnjcasey opened a new pull request, #24799: URL: https://github.com/apache/beam/pull/24799 Guide available here: https://docs.google.com/document/d/1-WxZTNu9RrLhh5O7Dl5PbnKqz3e5gm1x3gDBBhszVF8/edit?usp=sharing Thank you for your contribution! Follow this

[GitHub] [beam] nickcaballero commented on issue #21605: Slow Side input pattern bug in sample

2022-12-28 Thread GitBox
nickcaballero commented on issue #21605: URL: https://github.com/apache/beam/issues/21605#issuecomment-1366852926 I think this was fixed by https://github.com/apache/beam/pull/17371? One thing that is missing from the sample code is the use of a default value. -- This is an automated mess

[GitHub] [beam] JozoVilcek commented on issue #23852: [Task]: Spark runner flatMap output should not be required to fit in the memory

2022-12-28 Thread GitBox
JozoVilcek commented on issue #23852: URL: https://github.com/apache/beam/issues/23852#issuecomment-1366846870 e-mail thread for collecting feedback on initial WIP implementation https://www.mail-archive.com/dev@beam.apache.org/msg27521.html -- This is an automated message from the Apa

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058515439 ## sdks/go/pkg/beam/io/mongodbio/read.go: ## @@ -0,0 +1,469 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agre

[GitHub] [beam] lostluck closed issue #24575: [Feature Request][Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
lostluck closed issue #24575: [Feature Request][Go SDK]: MongoDB IO connector URL: https://github.com/apache/beam/issues/24575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [beam] lostluck commented on pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
lostluck commented on PR #24663: URL: https://github.com/apache/beam/pull/24663#issuecomment-1366824848 Thank you for another excellent contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] lostluck merged pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
lostluck merged PR #24663: URL: https://github.com/apache/beam/pull/24663 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] lostluck commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
lostluck commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058509455 ## sdks/go/pkg/beam/io/mongodbio/read.go: ## @@ -0,0 +1,469 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements

[GitHub] [beam] damccorm commented on a diff in pull request #24204: Shard python precommit

2022-12-28 Thread GitBox
damccorm commented on code in PR #24204: URL: https://github.com/apache/beam/pull/24204#discussion_r1058456102 ## .test-infra/jenkins/job_PreCommit_Python.groovy: ## @@ -22,6 +22,9 @@ PrecommitJobBuilder builder = new PrecommitJobBuilder( scope: this, nameBase: 'Python

[GitHub] [beam] johnjcasey commented on pull request #24205: Update Kafka watermark based on policy when records poll empty

2022-12-28 Thread GitBox
johnjcasey commented on PR #24205: URL: https://github.com/apache/beam/pull/24205#issuecomment-1366805833 run Java_Examples_Dataflow precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] johnjcasey commented on pull request #24205: Update Kafka watermark based on policy when records poll empty

2022-12-28 Thread GitBox
johnjcasey commented on PR #24205: URL: https://github.com/apache/beam/pull/24205#issuecomment-1366805693 Run Java_Examples_Dataflow_Java17 precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] Abacn closed issue #21426: beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog are vulnerable to CVE-2021-44228

2022-12-28 Thread GitBox
Abacn closed issue #21426: beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog are vulnerable to CVE-2021-44228 URL: https://github.com/apache/beam/issues/21426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [beam] Abacn commented on issue #21426: beam-sdks-java-io-hcatalog and beam-sdks-java-extensions-sql-hcatalog are vulnerable to CVE-2021-44228

2022-12-28 Thread GitBox
Abacn commented on issue #21426: URL: https://github.com/apache/beam/issues/21426#issuecomment-1366805647 beam bumped provided dep hive to v3.1.3 which uses log4j 2.17.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [beam] johnjcasey commented on pull request #24205: Update Kafka watermark based on policy when records poll empty

2022-12-28 Thread GitBox
johnjcasey commented on PR #24205: URL: https://github.com/apache/beam/pull/24205#issuecomment-1366805534 Run Spotless Precommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [beam] johnjcasey commented on a diff in pull request #24757: Adding watermark support to DebeziumIO transform

2022-12-28 Thread GitBox
johnjcasey commented on code in PR #24757: URL: https://github.com/apache/beam/pull/24757#discussion_r1058485490 ## sdks/java/io/debezium/src/main/java/org/apache/beam/io/debezium/KafkaConnectUtils.java: ## @@ -78,6 +79,18 @@ public static Schema.FieldType beamSchemaTypeFromKafk

[GitHub] [beam] johannaojeling commented on pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on PR #24663: URL: https://github.com/apache/beam/pull/24663#issuecomment-1366803851 Thank you for reviewing and for taking the time to answer my questions! As mentioned I'll attempt the splittable DoFn implementation in a separate branch so feel free to merge this

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058485073 ## sdks/go/pkg/beam/io/mongodbio/read.go: ## @@ -0,0 +1,469 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agre

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058477281 ## sdks/go/test/integration/internal/containers/containers.go: ## @@ -0,0 +1,81 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// cont

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058477010 ## sdks/go/pkg/beam/io/mongodbio/read_option.go: ## @@ -0,0 +1,60 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor licens

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058476483 ## sdks/go/test/integration/io/mongodbio/mongodbio_test.go: ## @@ -0,0 +1,232 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contri

[GitHub] [beam] johannaojeling commented on a diff in pull request #24663: [Go SDK]: MongoDB IO connector

2022-12-28 Thread GitBox
johannaojeling commented on code in PR #24663: URL: https://github.com/apache/beam/pull/24663#discussion_r1058476211 ## sdks/go/pkg/beam/io/mongodbio/read.go: ## @@ -0,0 +1,469 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agre

[GitHub] [beam] mosche merged pull request #24711: [Spark Dataset runner] Skip unconsumed additional outputs of ParDo.MultiOutput to avoid caching if not necessary

2022-12-28 Thread GitBox
mosche merged PR #24711: URL: https://github.com/apache/beam/pull/24711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.o

[GitHub] [beam] mosche closed issue #24710: [Feature Request]: Optimize for unused outputs of ParDo.MultiOutput in Spark Dataset runner

2022-12-28 Thread GitBox
mosche closed issue #24710: [Feature Request]: Optimize for unused outputs of ParDo.MultiOutput in Spark Dataset runner URL: https://github.com/apache/beam/issues/24710 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [beam] damccorm commented on pull request #24769: Disable sonarqube cron job

2022-12-28 Thread GitBox
damccorm commented on PR #24769: URL: https://github.com/apache/beam/pull/24769#issuecomment-1366763832 @kennknowles @apilloud @kileys is this job worth keeping around? Should it make the move to GHA or should we just remove it entirely? Are we ever going to invest in fixing it? -- This

[GitHub] [beam] damccorm merged pull request #24788: Install sklearn < 1.20.0 for sklearn postcommit tests

2022-12-28 Thread GitBox
damccorm merged PR #24788: URL: https://github.com/apache/beam/pull/24788 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache

[GitHub] [beam] Abacn commented on pull request #24769: Disable sonarqube cron job

2022-12-28 Thread GitBox
Abacn commented on PR #24769: URL: https://github.com/apache/beam/pull/24769#issuecomment-1366752792 R: @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [beam] AnandInguva commented on issue #24787: [Failing Test]: sklearnInferenceTest is failing in Python 3.9 PostCommit

2022-12-28 Thread GitBox
AnandInguva commented on issue #24787: URL: https://github.com/apache/beam/issues/24787#issuecomment-1366726781 https://scikit-learn.org/stable/model_persistence.html#interoperable-formats. This explains sklearn models should be used for inference on the same version it is trained on.

[GitHub] [beam] Abacn commented on a diff in pull request #24204: Shard python precommit

2022-12-28 Thread GitBox
Abacn commented on code in PR #24204: URL: https://github.com/apache/beam/pull/24204#discussion_r1058408893 ## .test-infra/jenkins/job_PreCommit_Python.groovy: ## @@ -22,6 +22,9 @@ PrecommitJobBuilder builder = new PrecommitJobBuilder( scope: this, nameBase: 'Python',

[GitHub] [beam] github-actions[bot] commented on pull request #24788: Install sklearn < 1.20.0 for sklearn postcommit tests

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24788: URL: https://github.com/apache/beam/pull/24788#issuecomment-1366720575 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] AnandInguva commented on pull request #24788: Install sklearn < 1.20.0 for sklearn postcommit tests

2022-12-28 Thread GitBox
AnandInguva commented on PR #24788: URL: https://github.com/apache/beam/pull/24788#issuecomment-1366719187 R: @damccorm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [beam] aromanenko-dev closed issue #24293: [Task]: [Avro] Create Avro extension for Java SDK

2022-12-28 Thread GitBox
aromanenko-dev closed issue #24293: [Task]: [Avro] Create Avro extension for Java SDK URL: https://github.com/apache/beam/issues/24293 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [beam] aromanenko-dev merged pull request #24294: Create Avro extension for Java SDK

2022-12-28 Thread GitBox
aromanenko-dev merged PR #24294: URL: https://github.com/apache/beam/pull/24294 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.

[GitHub] [beam] aromanenko-dev merged pull request #22221: [Website] update check-links

2022-12-28 Thread GitBox
aromanenko-dev merged PR #1: URL: https://github.com/apache/beam/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.

[GitHub] [beam] mosche commented on pull request #24711: [Spark Dataset runner] Skip unconsumed additional outputs of ParDo.MultiOutput to avoid caching if not necessary

2022-12-28 Thread GitBox
mosche commented on PR #24711: URL: https://github.com/apache/beam/pull/24711#issuecomment-1366678148 Run Spark StructuredStreaming ValidatesRunner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] github-actions[bot] commented on pull request #24204: Shard python precommit

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24204: URL: https://github.com/apache/beam/pull/24204#issuecomment-139289 Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control -- This is an automated message from the Apache Git Serv

[GitHub] [beam] damccorm commented on pull request #24204: Shard python precommit

2022-12-28 Thread GitBox
damccorm commented on PR #24204: URL: https://github.com/apache/beam/pull/24204#issuecomment-138547 R: @Abacn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [beam] JozoVilcek commented on issue #23852: [Task]: Spark runner flatMap output should not be required to fit in the memory

2022-12-28 Thread GitBox
JozoVilcek commented on issue #23852: URL: https://github.com/apache/beam/issues/23852#issuecomment-130219 .take-issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[GitHub] [beam] damccorm commented on pull request #24204: WIP: Shard python precommit

2022-12-28 Thread GitBox
damccorm commented on PR #24204: URL: https://github.com/apache/beam/pull/24204#issuecomment-1366633049 Run Python_Runners PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [beam] damccorm commented on pull request #24204: WIP: Shard python precommit

2022-12-28 Thread GitBox
damccorm commented on PR #24204: URL: https://github.com/apache/beam/pull/24204#issuecomment-1366621144 Run Seed Job -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [beam] github-actions[bot] commented on pull request #24630: [24469] Implement CsvIO.Write and supporting classes

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24630: URL: https://github.com/apache/beam/pull/24630#issuecomment-1366607693 Reminder, please take a look at this pr: @lukecwik @Abacn @pabloem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [beam] github-actions[bot] commented on pull request #24573: Bump httplib2 upper bound to 0.22.0.

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24573: URL: https://github.com/apache/beam/pull/24573#issuecomment-1366607729 Reminder, please take a look at this pr: @jrmccluskey -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [beam] github-actions[bot] commented on pull request #24619: Add static override for JDK TLS disabled/legacy algorithms in Java container

2022-12-28 Thread GitBox
github-actions[bot] commented on PR #24619: URL: https://github.com/apache/beam/pull/24619#issuecomment-1366607711 Reminder, please take a look at this pr: @kileys -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

  1   2   >