[GitHub] [beam] iemejia commented on issue #10546: [BEAM-9008] Add CassandraIO readAll method

2020-03-31 Thread GitBox
iemejia commented on issue #10546: [BEAM-9008] Add CassandraIO readAll method URL: https://github.com/apache/beam/pull/10546#issuecomment-606445740 I think this is needing a rebase. Please ping me when this is ready for review. I am eager to do so hoping this can make it into Beam 2.21.0

[GitHub] [beam] iemejia commented on issue #10546: [BEAM-9008] Add CassandraIO readAll method

2020-03-31 Thread GitBox
iemejia commented on issue #10546: [BEAM-9008] Add CassandraIO readAll method URL: https://github.com/apache/beam/pull/10546#issuecomment-606446222 I was also thinking about the potential connection issue and I think we can try to tackle it by creating some sort of static connection pool an

[GitHub] [beam] kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606458176 Run Load Tests Python GBK Flink Batch This is an automate

[GitHub] [beam] mxm commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
mxm commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606475180 > Unfortunately, I have no stack traces or error messages. I had to cancel the jobs manually after a long period of inactivity

[GitHub] [beam] jbartok opened a new pull request #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
jbartok opened a new pull request #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273 Hazelcast Jet recently released version 4.0 (https://jet-start.sh/blog/2020/03/02/jet-40-is-released) and this pull request updates Beam's Jet Runner to that versio

[GitHub] [beam] jbartok commented on issue #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
jbartok commented on issue #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273#issuecomment-606480985 R: @mxm This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] alexvanboxel commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO

2020-03-31 Thread GitBox
alexvanboxel commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO URL: https://github.com/apache/beam/pull/11258#discussion_r400705757 ## File path: sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbIO.java #

[GitHub] [beam] iemejia commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO

2020-03-31 Thread GitBox
iemejia commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO URL: https://github.com/apache/beam/pull/11258#discussion_r400785847 ## File path: sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbIO.java ##

[GitHub] [beam] iemejia commented on issue #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO

2020-03-31 Thread GitBox
iemejia commented on issue #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO URL: https://github.com/apache/beam/pull/11258#issuecomment-606521455 Thanks for the review @alexvanboxel You have a point about the javadoc. I will add that. --

[GitHub] [beam] mxm commented on a change in pull request #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
mxm commented on a change in pull request #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273#discussion_r400811820 ## File path: website/src/documentation/runners/jet.md ## @@ -59,24 +59,58 @@ Issue following command in the Beam exa

[GitHub] [beam] aromanenko-dev commented on issue #10973: [BEAM-9476] KinesisIO retry LimitExceededException

2020-03-31 Thread GitBox
aromanenko-dev commented on issue #10973: [BEAM-9476] KinesisIO retry LimitExceededException URL: https://github.com/apache/beam/pull/10973#issuecomment-606566517 @ameihm0912 kindly pinging to prevent this PR get staled This

[GitHub] [beam] piotr-szuberski opened a new pull request #11274: [BEAM-9633] [WIP] Add PubsubIO performance test

2020-03-31 Thread GitBox
piotr-szuberski opened a new pull request #11274: [BEAM-9633] [WIP] Add PubsubIO performance test URL: https://github.com/apache/beam/pull/11274 Add PubsubIO performance test Thank you for your contribution! Follow this checklist to help us incorporat

[GitHub] [beam] jbartok commented on a change in pull request #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
jbartok commented on a change in pull request #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273#discussion_r400846449 ## File path: website/src/documentation/runners/jet.md ## @@ -59,24 +59,58 @@ Issue following command in the Beam

[GitHub] [beam] iemejia commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO

2020-03-31 Thread GitBox
iemejia commented on a change in pull request #11258: [BEAM-9571] Add withMongoClientProvider to MongoDbIO URL: https://github.com/apache/beam/pull/11258#discussion_r400785847 ## File path: sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbIO.java ##

[GitHub] [beam] kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606581424 @mxm I checked two tests from [GBK suite](https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_LoadTests_GBK

[GitHub] [beam] kamilwu edited a comment on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
kamilwu edited a comment on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606582028 This is a log from today's attempt: https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch_PR/70/conso

[GitHub] [beam] kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606582028 This this is a log from today's attempt: https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch_PR/70/console

[GitHub] [beam] kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors.

2020-03-31 Thread GitBox
kamilwu commented on issue #11193: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. URL: https://github.com/apache/beam/pull/11193#issuecomment-606587808 At the moment, I'm considering three options: 1. Downgrade Flink version to 1.9 for all load tests 2. Set execution m

[GitHub] [beam] piotr-szuberski commented on issue #11274: [BEAM-9633] [WIP] Add PubsubIO performance test

2020-03-31 Thread GitBox
piotr-szuberski commented on issue #11274: [BEAM-9633] [WIP] Add PubsubIO performance test URL: https://github.com/apache/beam/pull/11274#issuecomment-606649898 @kamilwu This is an automated message from the Apache Git Servi

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606684667 retest this please This is an automated mes

[GitHub] [beam] lukecwik commented on issue #10786: Enable gRPC support in gcs connector.

2020-03-31 Thread GitBox
lukecwik commented on issue #10786: Enable gRPC support in gcs connector. URL: https://github.com/apache/beam/pull/10786#issuecomment-606685996 retest this please This is an automated message from the Apache Git Service. To re

[GitHub] [beam] lukecwik commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs

2020-03-31 Thread GitBox
lukecwik commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183#issuecomment-606689125 Run Java PreCommit This is an automated message from the Apache Git Ser

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606690268 retest this please This is an automated mes

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-31 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606692628 Run Java PreCommit This is an automated message fr

[GitHub] [beam] lukecwik commented on issue #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines

2020-03-31 Thread GitBox
lukecwik commented on issue #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines URL: https://github.com/apache/beam/pull/11271#issuecomment-606693397 Java Precommit failed due to: ``` 19:54:56 > Task :sdks:java:harness:compileT

[GitHub] [beam] lukecwik commented on issue #10786: [BEAM-8889] Enable gRPC support in gcs connector.

2020-03-31 Thread GitBox
lukecwik commented on issue #10786: [BEAM-8889] Enable gRPC support in gcs connector. URL: https://github.com/apache/beam/pull/10786#issuecomment-606694084 Please re-open this PR when it is ready. This is an automated message

[GitHub] [beam] lukecwik closed pull request #10786: [BEAM-8889] Enable gRPC support in gcs connector.

2020-03-31 Thread GitBox
lukecwik closed pull request #10786: [BEAM-8889] Enable gRPC support in gcs connector. URL: https://github.com/apache/beam/pull/10786 This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] lukecwik commented on a change in pull request #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers

2020-03-31 Thread GitBox
lukecwik commented on a change in pull request #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers URL: https://github.com/apache/beam/pull/11199#discussion_r401005415 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/

[GitHub] [beam] lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606705998 What a terrible choice for the FilterOutputStream implementation. Reading the javadoc they clearly state that everyone w

[GitHub] [beam] lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606708084 Please also fix JAXBCoder.java, as it too uses a FilteredOutputStream: https://github.com/apache/beam/blob/c8e200b157

[GitHub] [beam] lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401020590 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.ja

[GitHub] [beam] lukecwik edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606705998 What a terrible choice for the FilterOutputStream implementation. Reading the javadoc they clearly state that eve

[GitHub] [beam] regadas opened a new pull request #11275: [BEAM-9648]: DirectRunner should return null on timeout

2020-03-31 Thread GitBox
regadas opened a new pull request #11275: [BEAM-9648]: DirectRunner should return null on timeout URL: https://github.com/apache/beam/pull/11275 According to PipelineResult if waitUntilFinish(Duration) is supported it should return null. **Please** add a meaningful description f

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606717579 > I pushed fixes for both changes you requested but it isn't letting me close your changes requested . For t

[GitHub] [beam] ameihm0912 commented on issue #10973: [BEAM-9476] KinesisIO retry LimitExceededException

2020-03-31 Thread GitBox
ameihm0912 commented on issue #10973: [BEAM-9476] KinesisIO retry LimitExceededException URL: https://github.com/apache/beam/pull/10973#issuecomment-606731910 > @ameihm0912 kindly pinging to prevent this PR get staled @aromanenko-dev apologies for the delay here, I'll try to get that

[GitHub] [beam] ibzib merged pull request #11267: [BEAM-9638] Strengthen worker region & zone options tests.

2020-03-31 Thread GitBox
ibzib merged pull request #11267: [BEAM-9638] Strengthen worker region & zone options tests. URL: https://github.com/apache/beam/pull/11267 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] iemejia commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401049373 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnownedOutputStream.java #

[GitHub] [beam] iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606735293 Yes > What a terrible choice for the FilterOutputStream implementation. Reading the javadoc they clearly state tha

[GitHub] [beam] iemejia edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606735293 > What a terrible choice for the FilterOutputStream implementation. Reading the javadoc they clearly state that ev

[GitHub] [beam] lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401020590 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.ja

[GitHub] [beam] lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401052104 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnownedOutputStream.java

[GitHub] [beam] boyuanzz merged pull request #11060: [BEAM-9454] Create Deduplication transform based on user timer/state

2020-03-31 Thread GitBox
boyuanzz merged pull request #11060: [BEAM-9454] Create Deduplication transform based on user timer/state URL: https://github.com/apache/beam/pull/11060 This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] aaltay commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go

2020-03-31 Thread GitBox
aaltay commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go URL: https://github.com/apache/beam/pull/11246#discussion_r401061559 ## File path: sdks/go/container/license_script.sh ## @@ -0,0 +1,25 @@ +# Licensed to the Apache Softwar

[GitHub] [beam] robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-31 Thread GitBox
robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606744041 Regarding resolving a deferred artifact, the returned artifacts not in the list are required to be fetchable

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401073602 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/util/UnownedOutputStream.java

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401075112 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.j

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r401075112 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.j

[GitHub] [beam] lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606756607 retest this please This is an automated message from th

[GitHub] [beam] scwhittle commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
scwhittle commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606756154 Fixed. Sorry for that, I ran tests but must have been on the wrong branch.

[GitHub] [beam] lukemin89 commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukemin89 commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606756553 I want to say I found it in a fancier way, but I just found it by luck. ---

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606757069 retest this please This is an automated mes

[GitHub] [beam] lukemin89 edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukemin89 edited a comment on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606756553 I wish I found it in a fancier way, but I just found it by luck. ---

[GitHub] [beam] reuvenlax commented on issue #10767: Document Beam Schemas

2020-03-31 Thread GitBox
reuvenlax commented on issue #10767: Document Beam Schemas URL: https://github.com/apache/beam/pull/10767#issuecomment-606769541 @aromanenko-dev "dynamic" schemas can mean many things. Schemas support dynamic map fields, that can be used to dynamically extend the schema. -

[GitHub] [beam] lukecwik commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs

2020-03-31 Thread GitBox
lukecwik commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183#issuecomment-606771191 Run Java PreCommit This is an automated message from the Apache Git Ser

[GitHub] [beam] lostluck commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go

2020-03-31 Thread GitBox
lostluck commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go URL: https://github.com/apache/beam/pull/11246#discussion_r401103831 ## File path: sdks/go/container/license_script.sh ## @@ -0,0 +1,25 @@ +# Licensed to the Apache Softw

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606785621 retest this please This is an automated mes

[GitHub] [beam] aaltay commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go

2020-03-31 Thread GitBox
aaltay commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go URL: https://github.com/apache/beam/pull/11246#discussion_r401114775 ## File path: sdks/go/container/license_script.sh ## @@ -0,0 +1,25 @@ +# Licensed to the Apache Softwar

[GitHub] [beam] angoenka commented on a change in pull request #10717: [BEAM-8280] Enable type hint annotations

2020-03-31 Thread GitBox
angoenka commented on a change in pull request #10717: [BEAM-8280] Enable type hint annotations URL: https://github.com/apache/beam/pull/10717#discussion_r401117161 ## File path: sdks/python/apache_beam/typehints/decorators.py ## @@ -138,8 +139,7 @@ def foo((a, b)): _AN

[GitHub] [beam] udim commented on a change in pull request #10717: [BEAM-8280] Enable type hint annotations

2020-03-31 Thread GitBox
udim commented on a change in pull request #10717: [BEAM-8280] Enable type hint annotations URL: https://github.com/apache/beam/pull/10717#discussion_r401118829 ## File path: sdks/python/apache_beam/typehints/decorators.py ## @@ -138,8 +139,7 @@ def foo((a, b)): _ANY_VA

[GitHub] [beam] iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606821403 Mmm seems tests are not running on this one, weird. This

[GitHub] [beam] iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606821653 Now they are! Time to wait to and then merge. This is a

[GitHub] [beam] mxm commented on issue #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
mxm commented on issue #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273#issuecomment-606821680 Btw, there doesn't appear to be a Jenkins job (see .testinfra) for running the validates runner or other integration tests. Do you plan to add one?

[GitHub] [beam] mxm edited a comment on issue #11273: Update Jet Runner to Jet version 4.0

2020-03-31 Thread GitBox
mxm edited a comment on issue #11273: Update Jet Runner to Jet version 4.0 URL: https://github.com/apache/beam/pull/11273#issuecomment-606821680 Btw, there doesn't appear to be a Jenkins job (see .test-infra) for running the validates runner or other integration tests. Do you plan to add on

[GitHub] [beam] iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606822025 > I wish I found it in a fancier way, but I just found it by luck. Curious this looks like something that can be ma

[GitHub] [beam] stale[bot] closed pull request #9009: [BEAM-7699] Propagate null check/comparison to deepEquals

2020-03-31 Thread GitBox
stale[bot] closed pull request #9009: [BEAM-7699] Propagate null check/comparison to deepEquals URL: https://github.com/apache/beam/pull/9009 This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [beam] stale[bot] commented on issue #9009: [BEAM-7699] Propagate null check/comparison to deepEquals

2020-03-31 Thread GitBox
stale[bot] commented on issue #9009: [BEAM-7699] Propagate null check/comparison to deepEquals URL: https://github.com/apache/beam/pull/9009#issuecomment-606833632 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review

[GitHub] [beam] lukecwik merged pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik merged pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096 This is an automated message from the Apache Git Service. To res

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-31 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606839742 https://builds.apache.org/job/beam_PreCommit_Java_Commit/10616/ passed, merging. --

[GitHub] [beam] robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-31 Thread GitBox
robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606845864 Run Java PreCommit This is an automated mes

[GitHub] [beam] robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-31 Thread GitBox
robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606845984 Run JavaPortabilityApi PreCommit This is an

[GitHub] [beam] lukecwik merged pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-31 Thread GitBox
lukecwik merged pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] KevinGG opened a new pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
KevinGG opened a new pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276 1. The problem: when an intended blocking call such as `show`, `collect` or `head` is invoked, the user sometimes doesn't realize the code is

[GitHub] [beam] chamikaramj commented on issue #11254: [BEAM-7961] Refactors X-Lang test pipelines.

2020-03-31 Thread GitBox
chamikaramj commented on issue #11254: [BEAM-7961] Refactors X-Lang test pipelines. URL: https://github.com/apache/beam/pull/11254#issuecomment-606864502 Thanks. This is an automated message from the Apache Git Service. To re

[GitHub] [beam] chamikaramj merged pull request #11254: [BEAM-7961] Refactors X-Lang test pipelines.

2020-03-31 Thread GitBox
chamikaramj merged pull request #11254: [BEAM-7961] Refactors X-Lang test pipelines. URL: https://github.com/apache/beam/pull/11254 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb opened a new pull request #11277: [BEAM-9340] Populate requirement for timer families.

2020-03-31 Thread GitBox
robertwb opened a new pull request #11277: [BEAM-9340] Populate requirement for timer families. URL: https://github.com/apache/beam/pull/11277 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and eas

[GitHub] [beam] KevinGG commented on issue #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
KevinGG commented on issue #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#issuecomment-606868917 `yapf` formatted. `lint` passed locally. R: @aaltay R: @davidyan74 R: @rohdesamuel PTAL, thx! ---

[GitHub] [beam] robertwb commented on issue #11277: [BEAM-9340] Populate requirement for timer families.

2020-03-31 Thread GitBox
robertwb commented on issue #11277: [BEAM-9340] Populate requirement for timer families. URL: https://github.com/apache/beam/pull/11277#issuecomment-606869063 We should consider consolidating timers and timer families at a lower level, at lower priority than getting the protos in shape.

[GitHub] [beam] robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-31 Thread GitBox
robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606872418 Run Java PreCommit This is an automated mes

[GitHub] [beam] lukecwik commented on issue #11277: [BEAM-9340] Populate requirement for timer families.

2020-03-31 Thread GitBox
lukecwik commented on issue #11277: [BEAM-9340] Populate requirement for timer families. URL: https://github.com/apache/beam/pull/11277#issuecomment-606874607 CC: @boyuanzz This looks like a version that covers more then #11266 --

[GitHub] [beam] jaketf edited a comment on issue #11151: [BEAM-9468] Hl7v2 io

2020-03-31 Thread GitBox
jaketf edited a comment on issue #11151: [BEAM-9468] Hl7v2 io URL: https://github.com/apache/beam/pull/11151#issuecomment-604655037 Ok an updates here from an internal thread w/ API team. 1. [Message.List returning message contents is available in beta API with the view parameter.

[GitHub] [beam] jaketf edited a comment on issue #11151: [BEAM-9468] Hl7v2 io

2020-03-31 Thread GitBox
jaketf edited a comment on issue #11151: [BEAM-9468] Hl7v2 io URL: https://github.com/apache/beam/pull/11151#issuecomment-603980265 Open Questions: 1. Should we remove adaptive throttling? - Seems that we're using retries in the client request initializer and right now a "bad rec

[GitHub] [beam] chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs

2020-03-31 Thread GitBox
chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183#issuecomment-606893670 Run Java PreCommit This is an automated message from the Apache Git

[GitHub] [beam] rohdesamuel opened a new pull request #11278: no nesting impl

2020-03-31 Thread GitBox
rohdesamuel opened a new pull request #11278: no nesting impl URL: https://github.com/apache/beam/pull/11278 Change-Id: I4b9150517322301d448c6dc3009a962c293f8bb4 **Please** add a meaningful description for your change here Thank you for your cont

[GitHub] [beam] aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401248323 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfusc

[GitHub] [beam] aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401248341 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfusc

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-31 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606908499 cc: @robertwb This is an automated message from t

[GitHub] [beam] KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401250464 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfus

[GitHub] [beam] KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401251743 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfus

[GitHub] [beam] blcksrx opened a new pull request #11279: [BEAM-9653] remove _AvroSource in favor of using _FastAvroSource

2020-03-31 Thread GitBox
blcksrx opened a new pull request #11279: [BEAM-9653] remove _AvroSource in favor of using _FastAvroSource URL: https://github.com/apache/beam/pull/11279 **Please** add a meaningful description for your change here Thank you for your contribution! Foll

[GitHub] [beam] blcksrx commented on issue #11279: [BEAM-9653] remove _AvroSource in favor of using _FastAvroSource

2020-03-31 Thread GitBox
blcksrx commented on issue #11279: [BEAM-9653] remove _AvroSource in favor of using _FastAvroSource URL: https://github.com/apache/beam/pull/11279#issuecomment-606917166 R: @mxm This is an automated message from the A

[GitHub] [beam] angoenka commented on issue #10870: [BEAM-9263] Bump up sdk dataflow environment major versions

2020-03-31 Thread GitBox
angoenka commented on issue #10870: [BEAM-9263] Bump up sdk dataflow environment major versions URL: https://github.com/apache/beam/pull/10870#issuecomment-606918153 Retest this please This is an automated message from the Ap

[GitHub] [beam] KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
KevinGG commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401251743 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfus

[GitHub] [beam] tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-31 Thread GitBox
tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r401240289 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,143 @@ +#

[GitHub] [beam] tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-31 Thread GitBox
tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400642519 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -63,11 +61,13 @@ d

[GitHub] [beam] tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-31 Thread GitBox
tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r401249019 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,143 @@ +#

[GitHub] [beam] tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-31 Thread GitBox
tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r401176311 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,143 @@ +#

[GitHub] [beam] tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-31 Thread GitBox
tvalentyn commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r401244854 ## File path: sdks/python/container/Dockerfile ## @@ -51,16 +51,20 @@ RUN ln -s /usr/bin/ccache

[GitHub] [beam] pabloem commented on issue #11270: [WIP] Improvements for FnApiRunner

2020-03-31 Thread GitBox
pabloem commented on issue #11270: [WIP] Improvements for FnApiRunner URL: https://github.com/apache/beam/pull/11270#issuecomment-606919524 Run Portable_Python PreCommit This is an automated message from the Apache Git Service

[GitHub] [beam] aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks

2020-03-31 Thread GitBox
aaltay commented on a change in pull request #11276: [BEAM-7923] An indicator of progress in notebooks URL: https://github.com/apache/beam/pull/11276#discussion_r401254017 ## File path: sdks/python/apache_beam/runners/interactive/utils.py ## @@ -142,3 +142,61 @@ def obfusc

[GitHub] [beam] robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-31 Thread GitBox
robertwb commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606925303 The Java PreCommit has a different (unrelated) flake each time it's run. --

  1   2   >