[GitHub] [beam] piotr-szuberski commented on pull request #13319: [BEAM-11173] Add Bigtable table with read operation

piotr-szuberski commented on pull request #13319: URL: https://github.com/apache/beam/pull/13319#issuecomment-727823590 > My current question is, for the problem you find that ARRAY is not well supported, what part of design have it affected? It affects the possibility of reading mul

[GitHub] [beam] piotr-szuberski commented on pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

piotr-szuberski commented on pull request #13287: URL: https://github.com/apache/beam/pull/13287#issuecomment-727846119 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [beam] bntnam commented on a change in pull request #13331: [BEAM-11219][Website revamp] Development of All about Apache Beam component

bntnam commented on a change in pull request #13331: URL: https://github.com/apache/beam/pull/13331#discussion_r524026653 ## File path: website/www/site/layouts/partials/pillars/pillars-social.html ## @@ -0,0 +1,39 @@ +{{/* + Licensed under the Apache License, Version 2.0 (the

[GitHub] [beam] nikie commented on pull request #12779: [BEAM-10856] Support for NestedValueProvider for Python SDK

nikie commented on pull request #12779: URL: https://github.com/apache/beam/pull/12779#issuecomment-727860809 @epicfaace I am not sure why the tests are still failing. Rebasing against current `master` might help. Pylint runs fine locally after my docstrings fix. I have also

[GitHub] [beam] bntnam commented on a change in pull request #13331: [BEAM-11219][Website revamp] Development of All about Apache Beam component

bntnam commented on a change in pull request #13331: URL: https://github.com/apache/beam/pull/13331#discussion_r524034050 ## File path: website/www/site/assets/icons/github-icon.svg ## @@ -0,0 +1,10 @@ +http://www.w3.org/2000/svg"; width="32" height="32" fill="none" viewBox="0

[GitHub] [beam] kamilwu commented on pull request #13272: [BEAM-11207] Metric Extraction via proto RPC API

kamilwu commented on pull request #13272: URL: https://github.com/apache/beam/pull/13272#issuecomment-727912681 Run Go PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] kamilwu commented on a change in pull request #13272: [BEAM-11207] Metric Extraction via proto RPC API

kamilwu commented on a change in pull request #13272: URL: https://github.com/apache/beam/pull/13272#discussion_r524143231 ## File path: sdks/go/pkg/beam/core/runtime/metricsx/metricsx.go ## @@ -0,0 +1,181 @@ +// Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [beam] kamilwu commented on pull request #13272: [BEAM-11207] Metric Extraction via proto RPC API

kamilwu commented on pull request #13272: URL: https://github.com/apache/beam/pull/13272#issuecomment-727926020 @lostluck That should be it. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] pawelpasterz commented on pull request #13351: [BEAM-9278] hbase-shaded-client as provided dependency

pawelpasterz commented on pull request #13351: URL: https://github.com/apache/beam/pull/13351#issuecomment-727937165 R: @iemejia This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] piotr-szuberski commented on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-727941678 Run Java KafkaIO Performance Test This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] piotr-szuberski removed a comment on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski removed a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-727941678 Run Java KafkaIO Performance Test This is an automated message from the Apache Git Service. To re

[GitHub] [beam] codecov[bot] edited a comment on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

codecov[bot] edited a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-704839820 # [Codecov](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=h1) Report > Merging [#13026](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=desc) (000ac

[GitHub] [beam] codecov[bot] edited a comment on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

codecov[bot] edited a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-704839820 # [Codecov](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=h1) Report > Merging [#13026](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=desc) (000ac

[GitHub] [beam] piotr-szuberski commented on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-727942075 Run Java KafkaIO Performance Test This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] codecov[bot] edited a comment on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

codecov[bot] edited a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-704839820 # [Codecov](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=h1) Report > Merging [#13026](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=desc) (000ac

[GitHub] [beam] piotr-szuberski removed a comment on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski removed a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-727942075 Run Java KafkaIO Performance Test This is an automated message from the Apache Git Service. To re

[GitHub] [beam] piotr-szuberski commented on pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-727947192 Run Java KafkaIO Performance Test This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] aromanenko-dev commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

aromanenko-dev commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524226883 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.po

[GitHub] [beam] aromanenko-dev commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

aromanenko-dev commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524228547 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.po

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524248460 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524248460 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524248460 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] iemejia commented on a change in pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

iemejia commented on a change in pull request #13287: URL: https://github.com/apache/beam/pull/13287#discussion_r524046472 ## File path: runners/spark/build.gradle ## @@ -41,10 +42,15 @@ evaluationDependsOn(":runners:core-java") configurations { validatesRunner - hadoopV

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

piotr-szuberski commented on a change in pull request #13287: URL: https://github.com/apache/beam/pull/13287#discussion_r524253396 ## File path: sdks/java/io/hadoop-common/build.gradle ## @@ -16,17 +16,21 @@ * limitations under the License. */ +import java.util.stream.Col

[GitHub] [beam] iemejia commented on a change in pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

iemejia commented on a change in pull request #13287: URL: https://github.com/apache/beam/pull/13287#discussion_r524129225 ## File path: sdks/java/io/hadoop-common/build.gradle ## @@ -16,17 +16,21 @@ * limitations under the License. */ +import java.util.stream.Collectors

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

piotr-szuberski commented on a change in pull request #13287: URL: https://github.com/apache/beam/pull/13287#discussion_r524253396 ## File path: sdks/java/io/hadoop-common/build.gradle ## @@ -16,17 +16,21 @@ * limitations under the License. */ +import java.util.stream.Col

[GitHub] [beam] iemejia merged pull request #13351: [BEAM-9278] hbase-shaded-client as provided dependency

iemejia merged pull request #13351: URL: https://github.com/apache/beam/pull/13351 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] piotr-szuberski commented on pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

piotr-szuberski commented on pull request #13287: URL: https://github.com/apache/beam/pull/13287#issuecomment-727989092 > As a follow up it would be extremely nice to have also: > > 1. A phrase based task to invoke the Hadoop Versions tests from a PR e.g. "Run Hadoop Versions Com

[GitHub] [beam] iemejia merged pull request #13287: [BEAM-8876] Run hadoop tests with different versions and enable HadoopFormatIT test on Postcommit

iemejia merged pull request #13287: URL: https://github.com/apache/beam/pull/13287 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] iemejia commented on pull request #13350: [BEAM-11266] Python IO MongoDB: add bucket_auto aggregation option for bundling in Atlas.

iemejia commented on pull request #13350: URL: https://github.com/apache/beam/pull/13350#issuecomment-728008920 @nikie in case you have the extra cycles to create a PR for the Java connector I will be glad to take a look Thi

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524267637 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524267637 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] piotr-szuberski opened a new pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

piotr-szuberski opened a new pull request #13352: URL: https://github.com/apache/beam/pull/13352 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**

[GitHub] [beam] piotr-szuberski commented on pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

piotr-szuberski commented on pull request #13352: URL: https://github.com/apache/beam/pull/13352#issuecomment-728029398 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] piotr-szuberski commented on pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

piotr-szuberski commented on pull request #13352: URL: https://github.com/apache/beam/pull/13352#issuecomment-728030089 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [beam] piotr-szuberski commented on pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

piotr-szuberski commented on pull request #13352: URL: https://github.com/apache/beam/pull/13352#issuecomment-728035522 Run SQL PostCommit This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [beam] dmvk opened a new pull request #13353: [BEAM-11267] Remove unecessary reshuffle for stateful ParDo after key…

dmvk opened a new pull request #13353: URL: https://github.com/apache/beam/pull/13353 https://issues.apache.org/jira/browse/BEAM-11267 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[GitHub] [beam] je-ik commented on a change in pull request #13353: [BEAM-11267] Remove unecessary reshuffle for stateful ParDo after key…

je-ik commented on a change in pull request #13353: URL: https://github.com/apache/beam/pull/13353#discussion_r524333570 ## File path: runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkStreamingTranslationContext.java ## @@ -84,6 +85,17 @@ public void setOutputDat

[GitHub] [beam] piotr-szuberski commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524364352 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.p

[GitHub] [beam] suztomo commented on a change in pull request #13342: [BEAM-11260][BEAM-11261] Remove inappropriate assumptions about repo from linkage check script

suztomo commented on a change in pull request #13342: URL: https://github.com/apache/beam/pull/13342#discussion_r524361509 ## File path: sdks/java/build-tools/beam-linkage-check.sh ## @@ -36,46 +36,60 @@ set -o pipefail set -e # These default artifacts are common causes of

[GitHub] [beam] suztomo commented on a change in pull request #13342: [BEAM-11260][BEAM-11261] Remove inappropriate assumptions about repo from linkage check script

suztomo commented on a change in pull request #13342: URL: https://github.com/apache/beam/pull/13342#discussion_r524373408 ## File path: sdks/java/build-tools/beam-linkage-check.sh ## @@ -36,46 +36,60 @@ set -o pipefail set -e # These default artifacts are common causes of

[GitHub] [beam] allenpradeep commented on pull request #12255: [BEAM-9910]Add mutation byte counters to SpannerIO.write

allenpradeep commented on pull request #12255: URL: https://github.com/apache/beam/pull/12255#issuecomment-728161911 Retest this please This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [beam] lazylynx commented on pull request #13337: Add trigger commands for Direct, Spark XVR Postcommits.

lazylynx commented on pull request #13337: URL: https://github.com/apache/beam/pull/13337#issuecomment-728163435 @tvalentyn LGTM, Thanks! This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [beam] rosetn commented on a change in pull request #13326: [BEAM-10480] Update SDF programming guide

rosetn commented on a change in pull request #13326: URL: https://github.com/apache/beam/pull/13326#discussion_r524389224 ## File path: website/www/site/content/en/documentation/programming-guide.md ## @@ -5188,16 +5188,60 @@ restriction pairs. 12.1.1. A basic SDF {#a-bas

[GitHub] [beam] aromanenko-dev commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

aromanenko-dev commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524416279 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.po

[GitHub] [beam] tysonjh commented on pull request #13128: [BEAM-11265] Update quickstart-java.md

tysonjh commented on pull request #13128: URL: https://github.com/apache/beam/pull/13128#issuecomment-728191942 > Java precommit - I think this is a flaky test : "org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAllAttemptedMetrics". @tysonjh @kennknowles - could we deflake

[GitHub] [beam] tysonjh commented on pull request #13128: [BEAM-11265] Update quickstart-java.md

tysonjh commented on pull request #13128: URL: https://github.com/apache/beam/pull/13128#issuecomment-728196383 > > Java precommit - I think this is a flaky test : "org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAllAttemptedMetrics". @tysonjh @kennknowles - could we defla

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13345: [BEAM-4136] Keep strong reference to loggers to avoid potential NPE

TheNeuralBit commented on a change in pull request #13345: URL: https://github.com/apache/beam/pull/13345#discussion_r524431127 ## File path: sdks/java/harness/src/test/java/org/apache/beam/fn/harness/logging/BeamFnLoggingClientTest.java ## @@ -146,20 +147,25 @@ public void te

[GitHub] [beam] tysonjh edited a comment on pull request #13128: [BEAM-11265] Update quickstart-java.md

tysonjh edited a comment on pull request #13128: URL: https://github.com/apache/beam/pull/13128#issuecomment-728196383 > > Java precommit - I think this is a flaky test : "org.apache.beam.sdk.metrics.MetricsTest$AttemptedMetricTests.testAllAttemptedMetrics". @tysonjh @kennknowles - could w

[GitHub] [beam] tysonjh commented on pull request #12416: Update google-api-services versions.

tysonjh commented on pull request #12416: URL: https://github.com/apache/beam/pull/12416#issuecomment-728201724 > @tysonjh Hey, Seems like this commit breaks my code. Building my code, I get a `NoSuchMethodError` over `setOrderingKey`/`getOrderingKey` for the `pubsubMessage` (the `protoMes

[GitHub] [beam] iemejia merged pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

iemejia merged pull request #13352: URL: https://github.com/apache/beam/pull/13352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [beam] tvalentyn merged pull request #13337: Add trigger commands for Direct, Spark XVR Postcommits.

tvalentyn merged pull request #13337: URL: https://github.com/apache/beam/pull/13337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] tvalentyn merged pull request #13349: [BEAM-9980] use constants of python versions in PythonTestProperties.groovy

tvalentyn merged pull request #13349: URL: https://github.com/apache/beam/pull/13349 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] piotr-szuberski commented on pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

piotr-szuberski commented on pull request #13352: URL: https://github.com/apache/beam/pull/13352#issuecomment-728214918 > LGTM > > Fantastic! I am really happy to see this one in. Gives me so much more confidence on future changes compatibility! Glad to hear that! :) ---

[GitHub] [beam] TheNeuralBit commented on pull request #13341: Avoid unnecessary shuffling for single-input elementwise operations.

TheNeuralBit commented on pull request #13341: URL: https://github.com/apache/beam/pull/13341#issuecomment-728215583 Run Portable_Python PreCommit This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [beam] iemejia commented on pull request #13352: [BEAM-8615 BEAM-8569 BEAM-7937] Add hadoop 3 compatibility tests

iemejia commented on pull request #13352: URL: https://github.com/apache/beam/pull/13352#issuecomment-728219821 This deserves a mention in the CHANGES.md release notes, so don't hesitate to open a PR for this too. This is an

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12611: [BEAM-10139][BEAM-10140] Add cross-language support for Java SpannerIO with python wrapper

TheNeuralBit commented on a change in pull request #12611: URL: https://github.com/apache/beam/pull/12611#discussion_r524468664 ## File path: sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/StructUtils.java ## @@ -0,0 +1,380 @@ +/* + * Licen

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12611: [BEAM-10139][BEAM-10140] Add cross-language support for Java SpannerIO with python wrapper

TheNeuralBit commented on a change in pull request #12611: URL: https://github.com/apache/beam/pull/12611#discussion_r524471761 ## File path: sdks/python/apache_beam/io/gcp/spanner.py ## @@ -0,0 +1,635 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [beam] TheNeuralBit commented on pull request #12611: [BEAM-10139][BEAM-10140] Add cross-language support for Java SpannerIO with python wrapper

TheNeuralBit commented on pull request #12611: URL: https://github.com/apache/beam/pull/12611#issuecomment-728234466 Looks good, merging now. Thanks for all your work on this @piotr-szuberski :) This is an automated message f

[GitHub] [beam] TheNeuralBit merged pull request #12611: [BEAM-10139][BEAM-10140] Add cross-language support for Java SpannerIO with python wrapper

TheNeuralBit merged pull request #12611: URL: https://github.com/apache/beam/pull/12611 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] KevinGG commented on a change in pull request #13335: [BEAM-10921]: Fix BEAM-10921 and underlying issues

KevinGG commented on a change in pull request #13335: URL: https://github.com/apache/beam/pull/13335#discussion_r524478709 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -510,11 +529,10 @@ def track_user_pipelines(self): functio

[GitHub] [beam] boyuanzz commented on pull request #13344: [BEAM-11263] Java cleanUpDockerImages now force removes container images.

boyuanzz commented on pull request #13344: URL: https://github.com/apache/beam/pull/13344#issuecomment-728243326 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [beam] boyuanzz commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

boyuanzz commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524484370 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.powermoc

[GitHub] [beam] KevinGG commented on a change in pull request #13335: [BEAM-10921]: Fix BEAM-10921 and underlying issues

KevinGG commented on a change in pull request #13335: URL: https://github.com/apache/beam/pull/13335#discussion_r524478709 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -510,11 +529,10 @@ def track_user_pipelines(self): functio

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13341: Avoid unnecessary shuffling for single-input elementwise operations.

TheNeuralBit commented on a change in pull request #13341: URL: https://github.com/apache/beam/pull/13341#discussion_r524484614 ## File path: sdks/python/apache_beam/dataframe/frames.py ## @@ -1917,9 +1918,12 @@ def func(df, *args, **kwargs): '__i%s__' % base, fra

[GitHub] [beam] piotr-szuberski opened a new pull request #13354: [BEAM-8569] Add changes note about Hadoop 3 support

piotr-szuberski opened a new pull request #13354: URL: https://github.com/apache/beam/pull/13354 R: @iemejia Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose r

[GitHub] [beam] piotr-szuberski commented on pull request #12611: [BEAM-10139][BEAM-10140] Add cross-language support for Java SpannerIO with python wrapper

piotr-szuberski commented on pull request #12611: URL: https://github.com/apache/beam/pull/12611#issuecomment-728248437 > Looks good, merging now. Thanks for all your work on this @piotr-szuberski :) Thank you too for your reviews! :) ---

[GitHub] [beam] y1chi commented on a change in pull request #13350: [BEAM-11266] Python IO MongoDB: add bucket_auto aggregation option for bundling in Atlas.

y1chi commented on a change in pull request #13350: URL: https://github.com/apache/beam/pull/13350#discussion_r524486089 ## File path: sdks/python/apache_beam/io/mongodbio.py ## @@ -241,6 +275,27 @@ def _get_split_keys(self, desired_chunk_size_in_mb, start_pos, end_pos):

[GitHub] [beam] piotr-szuberski commented on pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

piotr-szuberski commented on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-728249571 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

codecov[bot] edited a comment on pull request #13026: URL: https://github.com/apache/beam/pull/13026#issuecomment-704839820 # [Codecov](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=h1) Report > Merging [#13026](https://codecov.io/gh/apache/beam/pull/13026?src=pr&el=desc) (000ac

[GitHub] [beam] boyuanzz commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

boyuanzz commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524491777 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.powermoc

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13331: [BEAM-11219][Website revamp] Development of All about Apache Beam component

TheNeuralBit commented on a change in pull request #13331: URL: https://github.com/apache/beam/pull/13331#discussion_r524493084 ## File path: website/www/site/assets/icons/extensive-icon.svg ## @@ -0,0 +1,7 @@ +http://www.w3.org/2000/svg"; width="112" height="112" fill="none"

[GitHub] [beam] TheNeuralBit merged pull request #13331: [BEAM-11219][Website revamp] Development of All about Apache Beam component

TheNeuralBit merged pull request #13331: URL: https://github.com/apache/beam/pull/13331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] robertwb commented on a change in pull request #13215: [BEAM-11151] Adds the ToStringFnRunner to Java

robertwb commented on a change in pull request #13215: URL: https://github.com/apache/beam/pull/13215#discussion_r524491784 ## File path: sdks/java/harness/src/main/java/org/apache/beam/fn/harness/ToStringFnRunner.java ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [beam] robertwb commented on a change in pull request #13341: Avoid unnecessary shuffling for single-input elementwise operations.

robertwb commented on a change in pull request #13341: URL: https://github.com/apache/beam/pull/13341#discussion_r524496642 ## File path: sdks/python/apache_beam/dataframe/frames.py ## @@ -1917,9 +1918,12 @@ def func(df, *args, **kwargs): '__i%s__' % base, frame_b

[GitHub] [beam] robertwb merged pull request #13341: Avoid unnecessary shuffling for single-input elementwise operations.

robertwb merged pull request #13341: URL: https://github.com/apache/beam/pull/13341 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] lostluck commented on pull request #13275: [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

lostluck commented on pull request #13275: URL: https://github.com/apache/beam/pull/13275#issuecomment-728260032 Hmmm it seems I was mistaken and this did not automatically run all the post commits. Doing so now. This is an

[GitHub] [beam] lostluck commented on pull request #13275: [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

lostluck commented on pull request #13275: URL: https://github.com/apache/beam/pull/13275#issuecomment-728260568 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] lostluck commented on pull request #13275: [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

lostluck commented on pull request #13275: URL: https://github.com/apache/beam/pull/13275#issuecomment-728260456 Run Release Gradle Build This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13333: Further dataframe batch consolidation.

TheNeuralBit commented on a change in pull request #1: URL: https://github.com/apache/beam/pull/1#discussion_r524496438 ## File path: sdks/python/apache_beam/dataframe/transforms.py ## @@ -223,8 +235,12 @@ def expand(self, pcolls): # Actually evaluate the exp

[GitHub] [beam] lostluck commented on pull request #13275: [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

lostluck commented on pull request #13275: URL: https://github.com/apache/beam/pull/13275#issuecomment-728261076 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] lostluck commented on pull request #13347: [release-2.26.0][BEAM-11264] Add Reshuffle in pd.read_*

lostluck commented on pull request #13347: URL: https://github.com/apache/beam/pull/13347#issuecomment-728262680 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] lostluck commented on pull request #13348: Go redundant type cleanup.

lostluck commented on pull request #13348: URL: https://github.com/apache/beam/pull/13348#issuecomment-728262368 Run Go PostCommit This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] lostluck commented on pull request #13347: [release-2.26.0][BEAM-11264] Add Reshuffle in pd.read_*

lostluck commented on pull request #13347: URL: https://github.com/apache/beam/pull/13347#issuecomment-728262952 Run Python 3.7 PostCommit This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [beam] lostluck commented on pull request #13347: [release-2.26.0][BEAM-11264] Add Reshuffle in pd.read_*

lostluck commented on pull request #13347: URL: https://github.com/apache/beam/pull/13347#issuecomment-728262843 Run Python 3.8 PostCommit This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [beam] lostluck commented on a change in pull request #13272: [BEAM-11207] Metric Extraction via proto RPC API

lostluck commented on a change in pull request #13272: URL: https://github.com/apache/beam/pull/13272#discussion_r523124215 ## File path: sdks/go/pkg/beam/core/runtime/metricsx/metricsx_test.go ## @@ -0,0 +1,166 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [beam] codecov[bot] edited a comment on pull request #12779: [BEAM-10856] Support for NestedValueProvider for Python SDK

codecov[bot] edited a comment on pull request #12779: URL: https://github.com/apache/beam/pull/12779#issuecomment-692856347 # [Codecov](https://codecov.io/gh/apache/beam/pull/12779?src=pr&el=h1) Report > Merging [#12779](https://codecov.io/gh/apache/beam/pull/12779?src=pr&el=desc) (af2c1

[GitHub] [beam] robertwb commented on a change in pull request #13333: Further dataframe batch consolidation.

robertwb commented on a change in pull request #1: URL: https://github.com/apache/beam/pull/1#discussion_r524508497 ## File path: sdks/python/apache_beam/dataframe/transforms.py ## @@ -410,6 +426,35 @@ def _total_memory_usage(frame): float('inf') +class _PreBat

[GitHub] [beam] KevinGG commented on a change in pull request #13335: [BEAM-10921]: Fix BEAM-10921 and underlying issues

KevinGG commented on a change in pull request #13335: URL: https://github.com/apache/beam/pull/13335#discussion_r524516876 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -163,7 +165,8 @@ def __init__(self): # the gRPC server ser

[GitHub] [beam] KevinGG commented on a change in pull request #13335: [BEAM-10921]: Fix BEAM-10921 and underlying issues

KevinGG commented on a change in pull request #13335: URL: https://github.com/apache/beam/pull/13335#discussion_r524516876 ## File path: sdks/python/apache_beam/runners/interactive/interactive_environment.py ## @@ -163,7 +165,8 @@ def __init__(self): # the gRPC server ser

[GitHub] [beam] boyuanzz commented on a change in pull request #13026: [WIP] [BEAM-7003 BEAM-8639 BEAM-8774] Update Kafka dependencies, enable IT test in Postcommit

boyuanzz commented on a change in pull request #13026: URL: https://github.com/apache/beam/pull/13026#discussion_r524517625 ## File path: sdks/java/io/kafka/build.gradle ## @@ -65,26 +76,68 @@ dependencies { testCompile library.java.junit testCompile library.java.powermoc

[GitHub] [beam] chamikaramj commented on pull request #13235: Changing BigQuery insertAll error from INFO level logging to WARNING

chamikaramj commented on pull request #13235: URL: https://github.com/apache/beam/pull/13235#issuecomment-728274702 I think the next step is to try out Beam 2.24.0 or later to see if this is really needed. This is an automat

[GitHub] [beam] boyuanzz commented on a change in pull request #13283: [BEAM-11142] Enable CrossLanguageValidateRunner test for dataflow run…

boyuanzz commented on a change in pull request #13283: URL: https://github.com/apache/beam/pull/13283#discussion_r524523450 ## File path: runners/google-cloud-dataflow-java/build.gradle ## @@ -312,6 +313,36 @@ task validatesRunnerStreaming { )) } +createCrossLanguageValid

[GitHub] [beam] boyuanzz commented on a change in pull request #13283: [BEAM-11142] Enable CrossLanguageValidateRunner test for dataflow run…

boyuanzz commented on a change in pull request #13283: URL: https://github.com/apache/beam/pull/13283#discussion_r524523450 ## File path: runners/google-cloud-dataflow-java/build.gradle ## @@ -312,6 +313,36 @@ task validatesRunnerStreaming { )) } +createCrossLanguageValid

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13211: [BEAM-8106] Separate Java8/11 container image build tasks

TheNeuralBit commented on a change in pull request #13211: URL: https://github.com/apache/beam/pull/13211#discussion_r524528475 ## File path: sdks/java/container/build.gradle ## @@ -86,51 +76,21 @@ licenseReport { renderers = [new JsonReportRenderer()] } -def imageJavaVer

[GitHub] [beam] TheNeuralBit commented on a change in pull request #13211: [BEAM-8106] Separate Java8/11 container image build tasks

TheNeuralBit commented on a change in pull request #13211: URL: https://github.com/apache/beam/pull/13211#discussion_r524529168 ## File path: sdks/java/container/common.gradle ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more co

[GitHub] [beam] kennknowles commented on pull request #13259: Fix Java ValidatesRunner V2 task dependency.

kennknowles commented on pull request #13259: URL: https://github.com/apache/beam/pull/13259#issuecomment-728284902 Makes sense to me. This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] kennknowles merged pull request #13259: Fix Java ValidatesRunner V2 task dependency.

kennknowles merged pull request #13259: URL: https://github.com/apache/beam/pull/13259 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] TheNeuralBit merged pull request #13128: [BEAM-11265] Update quickstart-java.md

TheNeuralBit merged pull request #13128: URL: https://github.com/apache/beam/pull/13128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] boyuanzz commented on pull request #13338: [BEAM-11070] Use self-checkpoint to enfore finalization happens.

boyuanzz commented on pull request #13338: URL: https://github.com/apache/beam/pull/13338#issuecomment-728292692 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message,

  1   2   >