[GitHub] [beam] piotr-szuberski opened a new pull request #11181: [BEAM-9500] Refactor load tests

2020-03-20 Thread GitBox
piotr-szuberski opened a new pull request #11181: [BEAM-9500] Refactor load tests URL: https://github.com/apache/beam/pull/11181 Changed the input to be initialized in the base class instead of the subclasses. Minor: - Replaced creating new TimeMonitors with the one from the supe

[GitHub] [beam] piotr-szuberski commented on issue #11181: [BEAM-9500] Refactor load tests

2020-03-20 Thread GitBox
piotr-szuberski commented on issue #11181: [BEAM-9500] Refactor load tests URL: https://github.com/apache/beam/pull/11181#issuecomment-601761814 I think there is a lot more to do with the part "make LoadTest parameterized instead of a base class." --

[GitHub] [beam] mxm commented on issue #11052: [BEAM-9446] Add missing parallelism and execution mode args.

2020-03-20 Thread GitBox
mxm commented on issue #11052: [BEAM-9446] Add missing parallelism and execution mode args. URL: https://github.com/apache/beam/pull/11052#issuecomment-601768565 @ibzib Could we try to keep the Runner options in case we are not reading them from the job server? We wouldn't have to manually

[GitHub] [beam] suztomo commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
suztomo commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601787116 @je-ik Thank you. It seems Jenkins didn't trigger jobs and "BigQueryIO Streaming Performance Test Java 10 GB" still has

[GitHub] [beam] pabloem merged pull request #11176: [BEAM-7923] Remove the excessive logging from capturable sources property.

2020-03-20 Thread GitBox
pabloem merged pull request #11176: [BEAM-7923] Remove the excessive logging from capturable sources property. URL: https://github.com/apache/beam/pull/11176 This is an automated message from the Apache Git Service. To respo

[beam] branch master updated: Remove the excessive logging from capturable sources property.

2020-03-20 Thread pabloem
This is an automated email from the ASF dual-hosted git repository. pabloem pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new 854a28c Remove the excessive logging from captur

[GitHub] [beam] ibzib commented on issue #11052: [BEAM-9446] Add missing parallelism and execution mode args.

2020-03-20 Thread GitBox
ibzib commented on issue #11052: [BEAM-9446] Add missing parallelism and execution mode args. URL: https://github.com/apache/beam/pull/11052#issuecomment-601814129 > @ibzib Could we try to keep the Runner options in case we are not reading them from the job server? We wouldn't have to manu

[GitHub] [beam] Ardagan opened a new pull request #11182: [DO NOT REVIEW] Heartbeat Transform and MultipleRead

2020-03-20 Thread GitBox
Ardagan opened a new pull request #11182: [DO NOT REVIEW] Heartbeat Transform and MultipleRead URL: https://github.com/apache/beam/pull/11182 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this chec

[GitHub] [beam] TheNeuralBit commented on issue #10767: Document Beam Schemas

2020-03-20 Thread GitBox
TheNeuralBit commented on issue #10767: Document Beam Schemas URL: https://github.com/apache/beam/pull/10767#issuecomment-601828482 > Is "dynamic schemas" already supported in some way? If not, what are the plans for this? I think the only plan that's been discussed is allowing schem

[GitHub] [beam] udim commented on issue #10717: [BEAM-8280] Enable type hint annotations

2020-03-20 Thread GitBox
udim commented on issue #10717: [BEAM-8280] Enable type hint annotations URL: https://github.com/apache/beam/pull/10717#issuecomment-601829383 Run Python PreCommit This is an automated message from the Apache Git Service. To r

[GitHub] [beam] TheNeuralBit commented on a change in pull request #10767: Document Beam Schemas

2020-03-20 Thread GitBox
TheNeuralBit commented on a change in pull request #10767: Document Beam Schemas URL: https://github.com/apache/beam/pull/10767#discussion_r395795896 ## File path: website/src/documentation/programming-guide.md ## @@ -1970,7 +1976,1078 @@ records.apply("WriteToText", See t

[GitHub] [beam] chadrik commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes

2020-03-20 Thread GitBox
chadrik commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes URL: https://github.com/apache/beam/pull/11070#discussion_r395798211 ## File path: website/src/_posts/2020-03-06-python-typing.md ## @@ -0,0 +1,117 @@ +--- +layout: post +tit

[GitHub] [beam] vnorigoog opened a new pull request #11183: add experiment flag use_grpc_for_gcs

2020-03-20 Thread GitBox
vnorigoog opened a new pull request #11183: add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183 Previously, using grpc channel for GCS was controlled by an API change. Changing that to an experiment, so that there is no need to introduce an API change.

[GitHub] [beam] boyuanzz merged pull request #11177: [BEAM-9562] Add Timer to Elements proto representation.

2020-03-20 Thread GitBox
boyuanzz merged pull request #11177: [BEAM-9562] Add Timer to Elements proto representation. URL: https://github.com/apache/beam/pull/11177 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] boyuanzz commented on issue #11177: [BEAM-9562] Add Timer to Elements proto representation.

2020-03-20 Thread GitBox
boyuanzz commented on issue #11177: [BEAM-9562] Add Timer to Elements proto representation. URL: https://github.com/apache/beam/pull/11177#issuecomment-601853060 Thanks for your review! Java PreCommit failures are not related. I'm going to merge it. ---

[GitHub] [beam] udim commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes

2020-03-20 Thread GitBox
udim commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes URL: https://github.com/apache/beam/pull/11070#discussion_r395824054 ## File path: website/src/_posts/2020-03-06-python-typing.md ## @@ -0,0 +1,117 @@ +--- +layout: post +title:

[beam] branch master updated: Add Timer to Elements proto representation.

2020-03-20 Thread boyuanz
This is an automated email from the ASF dual-hosted git repository. boyuanz pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git The following commit(s) were added to refs/heads/master by this push: new dde6dc3 Add Timer to Elements proto representati

[GitHub] [beam] udim commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes

2020-03-20 Thread GitBox
udim commented on a change in pull request #11070: [BEAM-8280] Blog post: Python typing changes URL: https://github.com/apache/beam/pull/11070#discussion_r395826290 ## File path: website/src/_posts/2020-03-06-python-typing.md ## @@ -0,0 +1,117 @@ +--- +layout: post +title:

[GitHub] [beam] robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel.

2020-03-20 Thread GitBox
robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel. URL: https://github.com/apache/beam/pull/11173#issuecomment-601858195 So I spent quite a bit of time refactoring things to do the separate "end all streams for this bundle" proposal that we discussed, an

[GitHub] [beam] lukecwik merged pull request #11162: [BEAM-9339, BEAM-2939] Drop splittable field from ParDoPayload, add splittable dofn requirement to Python SDK.

2020-03-20 Thread GitBox
lukecwik merged pull request #11162: [BEAM-9339, BEAM-2939] Drop splittable field from ParDoPayload, add splittable dofn requirement to Python SDK. URL: https://github.com/apache/beam/pull/11162 This is an automated message

[beam] branch master updated (c728d25 -> cd8a00c)

2020-03-20 Thread lcwik
This is an automated email from the ASF dual-hosted git repository. lcwik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from c728d25 Merge pull request #11177 from boyuanzz/timer_proto new 17b0216 [BEAM-9339, BEAM-2939] Drop splittable fi

[GitHub] [beam] chamikaramj commented on issue #11039: [BEAM-9383] Staging Dataflow artifacts from environment

2020-03-20 Thread GitBox
chamikaramj commented on issue #11039: [BEAM-9383] Staging Dataflow artifacts from environment URL: https://github.com/apache/beam/pull/11039#issuecomment-601864064 Retest this please This is an automated message from the Apa

[GitHub] [beam] youngoli commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation.

2020-03-20 Thread GitBox
youngoli commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation. URL: https://github.com/apache/beam/pull/11179#discussion_r395836071 ## File path: sdks/go/pkg/beam/pcollection.go ## @@ -60,6 +60,12 @@ func (p PCollection) Type() FullType {

[GitHub] [beam] lukecwik commented on issue #11180: [BEAM-9563] Change ToListCombineFn access level to private

2020-03-20 Thread GitBox
lukecwik commented on issue #11180: [BEAM-9563] Change ToListCombineFn access level to private URL: https://github.com/apache/beam/pull/11180#issuecomment-601864250 Run Java PreCommit This is an automated message from the Apa

[GitHub] [beam] chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs

2020-03-20 Thread GitBox
chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183#issuecomment-601864474 LGTM. Thanks. This is an automated message from the Apache Git Servi

[GitHub] [beam] chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs

2020-03-20 Thread GitBox
chamikaramj commented on issue #11183: [BEAM-8889]add experiment flag use_grpc_for_gcs URL: https://github.com/apache/beam/pull/11183#issuecomment-601864385 Retest this please This is an automated message from the Apache Git

[GitHub] [beam] lostluck commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation.

2020-03-20 Thread GitBox
lostluck commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation. URL: https://github.com/apache/beam/pull/11179#discussion_r395838378 ## File path: sdks/go/pkg/beam/pcollection.go ## @@ -60,6 +60,12 @@ func (p PCollection) Type() FullType {

[GitHub] [beam] lostluck commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation.

2020-03-20 Thread GitBox
lostluck commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation. URL: https://github.com/apache/beam/pull/11179#discussion_r395840270 ## File path: sdks/go/pkg/beam/core/graph/fn.go ## @@ -446,23 +444,16 @@ func validateMainInputs(fn *Fn, metho

[GitHub] [beam] robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#issuecomment-601872470 I rebased on top of your PR. This is an automated message from the Apache Gi

[GitHub] [beam] robertwb commented on a change in pull request #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#discussion_r395845584 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/PTransformTra

[GitHub] [beam] youngoli commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation.

2020-03-20 Thread GitBox
youngoli commented on a change in pull request #11179: [BEAM-3301] Bugfix in DoFn validation. URL: https://github.com/apache/beam/pull/11179#discussion_r395850295 ## File path: sdks/go/pkg/beam/core/graph/fn.go ## @@ -446,23 +444,16 @@ func validateMainInputs(fn *Fn, metho

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies

2020-03-20 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies URL: https://github.com/apache/beam/pull/11067#discussion_r395865763 ## File path: licenses/scripts/pull_licenses_py.py ## @@ -0,0 +1,157 @@ +# +# Licensed to the Apache Sof

[GitHub] [beam] Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies

2020-03-20 Thread GitBox
Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies URL: https://github.com/apache/beam/pull/11067#issuecomment-601889129 @tvalentyn , I fixed all your comments about Python. PTAL when you have time. --

[GitHub] [beam] apilloud commented on issue #11041: Use beam join api in sql

2020-03-20 Thread GitBox
apilloud commented on issue #11041: Use beam join api in sql URL: https://github.com/apache/beam/pull/11041#issuecomment-601893513 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601894197 @suztomo do you need to rerun the checks that were not triggered? --

[GitHub] [beam] suztomo commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
suztomo commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601894622 Yes, please This is an automated message from the Apac

[GitHub] [beam] apilloud commented on issue #11041: Use beam join api in sql

2020-03-20 Thread GitBox
apilloud commented on issue #11041: Use beam join api in sql URL: https://github.com/apache/beam/pull/11041#issuecomment-601896403 Run SQL Postcommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] lukecwik commented on issue #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on issue #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#issuecomment-601897399 CC: @ajamato This is an automated message from the Apache Gi

[GitHub] [beam] lukecwik opened a new pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik opened a new pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly an

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601899190 Run Java HadoopFormatIO Performance Test This is an auto

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601899334 Run Spark ValidatesRunner This is an automated message f

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601899388 Run SQL Postcommit This is an automated message from the

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601899121 Run Java PostCommit This is an automated message from th

[GitHub] [beam] je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build

2020-03-20 Thread GitBox
je-ik commented on issue #11168: [BEAM-9542] Limit and clarify the effect of "force" in Java build URL: https://github.com/apache/beam/pull/11168#issuecomment-601899254 Run Dataflow ValidatesRunner This is an automated messag

[GitHub] [beam] robertwb commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally URL: https://github.com/apache/beam/pull/11174#discussion_r395882293 ## File path: sdks/python/apache_beam/transforms/core.py ## @@ -1811,6 +1811,8 @@ def add_input_t

[GitHub] [beam] robertwb commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally URL: https://github.com/apache/beam/pull/11174#discussion_r395883196 ## File path: sdks/python/apache_beam/transforms/core.py ## @@ -1811,6 +1811,8 @@ def add_input_t

[GitHub] [beam] robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395885958 ## File path: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py ## @@ -78,12 +7

[GitHub] [beam] robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395884481 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py ## @@ -269,8 +269,8 @@ def

[GitHub] [beam] lukecwik edited a comment on issue #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik edited a comment on issue #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#issuecomment-601897399 CC: @ajamato @robertwb This is an automated message f

[GitHub] [beam] pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395888936 ## File path: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py ## @@ -78,12 +78

[GitHub] [beam] pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395888936 ## File path: sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py ## @@ -78,12 +78

[GitHub] [beam] pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395889275 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py ## @@ -269,8 +269,8 @@ def v

[GitHub] [beam] Tesio commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test

2020-03-20 Thread GitBox
Tesio commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test URL: https://github.com/apache/beam/pull/10914#issuecomment-601910749 retest this please This is an automated message from the Ap

[GitHub] [beam] robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel.

2020-03-20 Thread GitBox
robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel. URL: https://github.com/apache/beam/pull/11173#issuecomment-601910742 Rebased on #11177 and updated the proto for timers. This is an autom

[GitHub] [beam] KevinGG commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally

2020-03-20 Thread GitBox
KevinGG commented on a change in pull request #11174: [BEAM-7923] Pop failed transform in CombineGlobally URL: https://github.com/apache/beam/pull/11174#discussion_r395890026 ## File path: sdks/python/apache_beam/transforms/core.py ## @@ -1811,6 +1811,8 @@ def add_input_ty

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395888153 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ message

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395887788 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -139,7 +137,7 @@ message Mon

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395888595 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ message

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395889104 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -194,33 +188,25 @@ extend go

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395888786 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -194,33 +188,25 @@ extend go

[GitHub] [beam] aaltay commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test

2020-03-20 Thread GitBox
aaltay commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test URL: https://github.com/apache/beam/pull/10914#issuecomment-601913195 retest this please This is an automated message from the A

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395892463 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ message

[GitHub] [beam] aaltay commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test

2020-03-20 Thread GitBox
aaltay commented on issue #10914: [BEAM-8078] streaming_wordcount_debugging.py is missing a test URL: https://github.com/apache/beam/pull/10914#issuecomment-601913507 @Tesio - unfortunately, due to an ongoing issue, jenkins only starts test when a committer requests it. This should be reso

[GitHub] [beam] ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
ajamato commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395893496 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ message

[GitHub] [beam] robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#discussion_r395897316 ## File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py ## @@ -269,8 +269,8 @@ def

[GitHub] [beam] iemejia commented on issue #10290: [BEAM-8561] Add ThriftIO to support IO for Thrift files

2020-03-20 Thread GitBox
iemejia commented on issue #10290: [BEAM-8561] Add ThriftIO to support IO for Thrift files URL: https://github.com/apache/beam/pull/10290#issuecomment-601918008 We let it like this because we had a regression, if we want to generate and compile the class we need to have thrift installed in

[GitHub] [beam] lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395900505 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ messag

[GitHub] [beam] lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395900617 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -194,33 +188,25 @@ extend g

[GitHub] [beam] robertwb commented on a change in pull request #10760: [BEAM-9545] Dataframe transforms

2020-03-20 Thread GitBox
robertwb commented on a change in pull request #10760: [BEAM-9545] Dataframe transforms URL: https://github.com/apache/beam/pull/10760#discussion_r395900936 ## File path: sdks/python/apache_beam/dataframe/transforms.py ## @@ -0,0 +1,246 @@ +# +# Licensed to the Apache Soft

[GitHub] [beam] robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#issuecomment-601921071 Run Java PreCommit This is an automated message from the Apache Git Service.

[GitHub] [beam] udim commented on issue #10717: [BEAM-8280] Enable type hint annotations

2020-03-20 Thread GitBox
udim commented on issue #10717: [BEAM-8280] Enable type hint annotations URL: https://github.com/apache/beam/pull/10717#issuecomment-601923382 Run Python PreCommit This is an automated message from the Apache Git Service. To r

[GitHub] [beam] pabloem commented on issue #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem commented on issue #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#issuecomment-601923625 Run Python PreCommit This is an automated message from the Apache Git Se

[GitHub] [beam] lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395905193 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ messag

[GitHub] [beam] chamikaramj commented on issue #11185: [BEAM-8019] Some generalizations to support cross-language transforms.

2020-03-20 Thread GitBox
chamikaramj commented on issue #11185: [BEAM-8019] Some generalizations to support cross-language transforms. URL: https://github.com/apache/beam/pull/11185#issuecomment-601927190 cc: @robertwb @lukecwik Still addressing some unit test failures but sharing for any early comments. -

[GitHub] [beam] chamikaramj opened a new pull request #11185: [BEAM-8019] Some generalizations to support cross-language transforms.

2020-03-20 Thread GitBox
chamikaramj opened a new pull request #11185: [BEAM-8019] Some generalizations to support cross-language transforms. URL: https://github.com/apache/beam/pull/11185 These are needed for for runners that need to build a Python object graph from a runner API proto with external transforms (fo

[GitHub] [beam] iemejia commented on issue #11099: [BEAM-9420] Configurable timeout for blocking kafka API call(s)

2020-03-20 Thread GitBox
iemejia commented on issue #11099: [BEAM-9420] Configurable timeout for blocking kafka API call(s) URL: https://github.com/apache/beam/pull/11099#issuecomment-601928283 @aromanenko-dev maybe since he has been maintaing KafkaIo for a while. --

[GitHub] [beam] lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395908901 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ messag

[GitHub] [beam] robertwb commented on issue #11174: [BEAM-7923] Pop failed transform when error is raised

2020-03-20 Thread GitBox
robertwb commented on issue #11174: [BEAM-7923] Pop failed transform when error is raised URL: https://github.com/apache/beam/pull/11174#issuecomment-601929637 retest this please This is an automated message from the Apache G

[GitHub] [beam] robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel.

2020-03-20 Thread GitBox
robertwb commented on issue #11173: [BEAM-9558] Add explicit end bit for data channel. URL: https://github.com/apache/beam/pull/11173#issuecomment-601930536 Run Java PreCommit This is an automated message from the Apache Git

[GitHub] [beam] lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo.

2020-03-20 Thread GitBox
lukecwik commented on a change in pull request #11184: [WIP][BEAM-4374] Update protos related to MonitoringInfo. URL: https://github.com/apache/beam/pull/11184#discussion_r395908901 ## File path: model/pipeline/src/main/proto/metrics.proto ## @@ -229,101 +215,127 @@ messag

[GitHub] [beam] pabloem commented on a change in pull request #11086: [BEAM-8910] Make custom BQ source read from Avro

2020-03-20 Thread GitBox
pabloem commented on a change in pull request #11086: [BEAM-8910] Make custom BQ source read from Avro URL: https://github.com/apache/beam/pull/11086#discussion_r395910855 ## File path: sdks/python/apache_beam/io/gcp/bigquery_read_it_test.py ## @@ -236,11 +251,12 @@ def cr

[GitHub] [beam] aaltay commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies

2020-03-20 Thread GitBox
aaltay commented on a change in pull request #11156: [BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies URL: https://github.com/apache/beam/pull/11156#discussion_r395911140 ## File path: buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy #

[GitHub] [beam] youngoli merged pull request #11179: [BEAM-3301] Bugfix in DoFn validation.

2020-03-20 Thread GitBox
youngoli merged pull request #11179: [BEAM-3301] Bugfix in DoFn validation. URL: https://github.com/apache/beam/pull/11179 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[beam] branch master updated (cd8a00c -> 2f25e62)

2020-03-20 Thread danoliveira
This is an automated email from the ASF dual-hosted git repository. danoliveira pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from cd8a00c Merge pull request #11162 from lukecwik/proto3 add 78141f2 [BEAM-3301] Bugfix in DoFn validation.

[GitHub] [beam] iemejia commented on issue #11008: Update comment to tell user this is not secure

2020-03-20 Thread GitBox
iemejia commented on issue #11008: Update comment to tell user this is not secure URL: https://github.com/apache/beam/pull/11008#issuecomment-601937789 I filled https://issues.apache.org/jira/browse/BEAM-9564 to remove this risky option since it seems we agree on this. I will open a PR for

[GitHub] [beam] iemejia closed pull request #11008: Update comment to tell user this is not secure

2020-03-20 Thread GitBox
iemejia closed pull request #11008: Update comment to tell user this is not secure URL: https://github.com/apache/beam/pull/11008 This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] pabloem commented on issue #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem commented on issue #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153#issuecomment-601938785 Run Python PreCommit This is an automated message from the Apache Git Se

[GitHub] [beam] iemejia opened a new pull request #11186: [BEAM-9564] Remove insecure ssl options from MongoDBIO

2020-03-20 Thread GitBox
iemejia opened a new pull request #11186: [BEAM-9564] Remove insecure ssl options from MongoDBIO URL: https://github.com/apache/beam/pull/11186 These changes are not backwards compatible but this is intended to solve the potential security issues and also because MongoDBIO does not have st

[GitHub] [beam] robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#issuecomment-601946329 Run Java PreCommit This is an automated message from the Apache Git

[GitHub] [beam] robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#issuecomment-601946285 This passes locally. org.apache.beam.sdk.transforms.ParDoLifecycleTest.testTeardownCalledAfterExceptionInFinishBundleStateful seems unrelate

[GitHub] [beam] robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java.

2020-03-20 Thread GitBox
robertwb commented on issue #11165: [BEAM-9340] Populate requirements for Java. URL: https://github.com/apache/beam/pull/11165#issuecomment-601947065 Run Java PreCommit This is an automated message from the Apache Git Service.

[GitHub] [beam] Hannah-Jiang opened a new pull request #11187: optionally import grpc

2020-03-20 Thread GitBox
Hannah-Jiang opened a new pull request #11187: optionally import grpc URL: https://github.com/apache/beam/pull/11187 R: @aaltay Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: -

[GitHub] [beam] aaltay commented on issue #11174: [BEAM-7923] Pop failed transform when error is raised

2020-03-20 Thread GitBox
aaltay commented on issue #11174: [BEAM-7923] Pop failed transform when error is raised URL: https://github.com/apache/beam/pull/11174#issuecomment-601949155 retest this please This is an automated message from the Apache Git

[GitHub] [beam] aaltay commented on a change in pull request #11187: optionally import grpc

2020-03-20 Thread GitBox
aaltay commented on a change in pull request #11187: optionally import grpc URL: https://github.com/apache/beam/pull/11187#discussion_r395928394 ## File path: sdks/python/apache_beam/runners/direct/test_stream_impl.py ## @@ -29,13 +29,20 @@ import itertools -import grp

[GitHub] [beam] aaltay commented on issue #11187: optionally import grpc

2020-03-20 Thread GitBox
aaltay commented on issue #11187: optionally import grpc URL: https://github.com/apache/beam/pull/11187#issuecomment-601949594 @Hannah-Jiang feel free to self merge, once you address my comment and tests pass. This is an auto

[GitHub] [beam] robertwb merged pull request #11173: [BEAM-9558] Add explicit end bit for data channel.

2020-03-20 Thread GitBox
robertwb merged pull request #11173: [BEAM-9558] Add explicit end bit for data channel. URL: https://github.com/apache/beam/pull/11173 This is an automated message from the Apache Git Service. To respond to the message, plea

[beam] branch master updated (2f25e62 -> 0389f54)

2020-03-20 Thread robertwb
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/beam.git. from 2f25e62 Merge pull request #11179: [BEAM-3301] Bugfix in DoFn validation. add eb05a91 [BEAM-9558] Add an expl

[beam] 01/01: Merge pull request #11173 [BEAM-9558] Add explicit end bit for data channel.

2020-03-20 Thread robertwb
This is an automated email from the ASF dual-hosted git repository. robertwb pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/beam.git commit 0389f54ccd5c018f75c883bafa6be959503a9b58 Merge: 2f25e62 c15a7e2 Author: Robert Bradshaw AuthorDate: Fri Mar 20 16:23:37

[GitHub] [beam] robertwb commented on issue #11110: [BEAM-9398] runtime_type_check: support setup

2020-03-20 Thread GitBox
robertwb commented on issue #0: [BEAM-9398] runtime_type_check: support setup URL: https://github.com/apache/beam/pull/0#issuecomment-601952041 Run Python PreCommit This is an automated message from the Apache Git Ser

[GitHub] [beam] pabloem merged pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner

2020-03-20 Thread GitBox
pabloem merged pull request #11153: [BEAM-9537] Adding a new module for FnApiRunner URL: https://github.com/apache/beam/pull/11153 This is an automated message from the Apache Git Service. To respond to the message, please l

<    1   2   3   >