[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606432397 Run Java PreCommit This is an automated message fr

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11243: [BEAM-9136]Add licenses for dependencies for Java

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r400602453 ## File path: sdks/java/container/build.gradle ## @@ -68,6 +73,25 @@ golang { } } +// thi

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11243: [BEAM-9136]Add licenses for dependencies for Java

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r400602453 ## File path: sdks/java/container/build.gradle ## @@ -68,6 +73,25 @@ golang { } } +// thi

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606382795 Run Java PreCommit This is an automated message fr

[GitHub] [beam] chamikaramj commented on issue #11254: [BEAM-7961] Refactors X-Lang test pipelines.

2020-03-30 Thread GitBox
chamikaramj commented on issue #11254: [BEAM-7961] Refactors X-Lang test pipelines. URL: https://github.com/apache/beam/pull/11254#issuecomment-606378924 Run Python PreCommit This is an automated message from the Apache Git S

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606369776 Run Java PreCommit This is an automated message fr

[GitHub] [beam] lukecwik commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
lukecwik commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606368223 Run Java PreCommit This is an automated message fr

[GitHub] [beam] lukecwik commented on issue #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines

2020-03-30 Thread GitBox
lukecwik commented on issue #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines URL: https://github.com/apache/beam/pull/11271#issuecomment-606366808 retest this please ---

[GitHub] [beam] lukecwik commented on a change in pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines URL: https://github.com/apache/beam/pull/11271#discussion_r400610628 ## File path: sdks/go/pkg/beam/core/runtime/exec/combine.

[GitHub] [beam] lukecwik commented on a change in pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines URL: https://github.com/apache/beam/pull/11271#discussion_r400609084 ## File path: runners/core-construction-java/src/main/ja

[GitHub] [beam] youngoli commented on issue #11257: [BEAM-9642] Create runtime invokers for SDF methods.

2020-03-30 Thread GitBox
youngoli commented on issue #11257: [BEAM-9642] Create runtime invokers for SDF methods. URL: https://github.com/apache/beam/pull/11257#issuecomment-606366621 R: @lostluck Ready for an actual review. It should be much more optimized now. I have a call function so the code won't rep

[GitHub] [beam] boyuanzz commented on a change in pull request #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers

2020-03-30 Thread GitBox
boyuanzz commented on a change in pull request #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers URL: https://github.com/apache/beam/pull/11199#discussion_r400607853 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400606139 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400604823 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400606909 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400604546 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400604009 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400603582 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606356720 Double checked python side. Since we don't have timer family support in python sdk, we should be fine for now.

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606356448 Run Java PreCommit This is an automated message fr

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11246: [BEAM-9136]Add licenses for dependencies for Go URL: https://github.com/apache/beam/pull/11246#discussion_r400600122 ## File path: sdks/go/container/license_script.sh ## @@ -0,0 +1,25 @@ +# Licensed to the Apache S

[GitHub] [beam] boyuanzz commented on issue #11060: [BEAM-9454] Create Deduplication transform based on user timer/state

2020-03-30 Thread GitBox
boyuanzz commented on issue #11060: [BEAM-9454] Create Deduplication transform based on user timer/state URL: https://github.com/apache/beam/pull/11060#issuecomment-606349734 Run Python PreCommit This is an automated message

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400473578 ## File path: sdks/python/container/license_scripts/manual_licenses/enum34/LICENSE ## @@ -0,0

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400542085 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,153 @@

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400500406 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,153 @@

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400486395 ## File path: sdks/python/container/license_scripts/pull_licenses_py.py ## @@ -0,0 +1,153 @@

[GitHub] [beam] robinyqiu opened a new pull request #11272: Support ZetaSQL DATE type as a Beam LogicalType

2020-03-30 Thread GitBox
robinyqiu opened a new pull request #11272: Support ZetaSQL DATE type as a Beam LogicalType URL: https://github.com/apache/beam/pull/11272 This PR adds support of all ZetaSQL ([BigQuery Standard SQL](https://cloud.google.com/bigquery/docs/reference/standard-sql/timestamp_functions)) DATE

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400583972 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400570316 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400566768 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400585676 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400567481 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400572321 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400584277 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400566563 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400571617 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
robertwb commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400583853 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] udim commented on issue #11175: [BEAM-9529] Remove datastore.v1, googledatastore

2020-03-30 Thread GitBox
udim commented on issue #11175: [BEAM-9529] Remove datastore.v1, googledatastore URL: https://github.com/apache/beam/pull/11175#issuecomment-606333889 Run Python2_PVR_Flink PreCommit This is an automated message from the Apach

[GitHub] [beam] chamikaramj commented on issue #11175: [BEAM-9529] Remove datastore.v1, googledatastore

2020-03-30 Thread GitBox
chamikaramj commented on issue #11175: [BEAM-9529] Remove datastore.v1, googledatastore URL: https://github.com/apache/beam/pull/11175#issuecomment-606326635 LGTM. Thanks. This is an automated message from the Apache Git Serv

[GitHub] [beam] lukemin89 commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
lukemin89 commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606317443 Thanks for the review! I was looking to improve my 10+TB GBK steps and happened to find this. I just decided to

[GitHub] [beam] acrites opened a new pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines

2020-03-30 Thread GitBox
acrites opened a new pull request #11271: [BEAM-9624] Adds Convert to Accumulators operator for use in combiner lifting for streaming pipelines URL: https://github.com/apache/beam/pull/11271 The Convert To Accumulators operation is defined here: https://s.apache.org/beam-runner-api-c

[GitHub] [beam] apilloud merged pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest

2020-03-30 Thread GitBox
apilloud merged pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest URL: https://github.com/apache/beam/pull/11268 This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [beam] pabloem opened a new pull request #11270: [WIP] Improvements for FnApiRunner

2020-03-30 Thread GitBox
pabloem opened a new pull request #11270: [WIP] Improvements for FnApiRunner URL: https://github.com/apache/beam/pull/11270 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us i

[GitHub] [beam] ibzib opened a new pull request #11269: [BEAM-9199] Require Dataflow --region in Python SDK.

2020-03-30 Thread GitBox
ibzib opened a new pull request #11269: [BEAM-9199] Require Dataflow --region in Python SDK. URL: https://github.com/apache/beam/pull/11269 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checkl

[GitHub] [beam] apilloud commented on issue #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest

2020-03-30 Thread GitBox
apilloud commented on issue #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest URL: https://github.com/apache/beam/pull/11268#issuecomment-606306157 retest this please This is an automated message from the Apache Git

[GitHub] [beam] robinyqiu commented on a change in pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest

2020-03-30 Thread GitBox
robinyqiu commented on a change in pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest URL: https://github.com/apache/beam/pull/11268#discussion_r400555614 ## File path: sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/Z

[GitHub] [beam] robinyqiu opened a new pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest

2020-03-30 Thread GitBox
robinyqiu opened a new pull request #11268: [BEAM-9179] Clean up code in ZetaSQLDialectSpecTest URL: https://github.com/apache/beam/pull/11268 Minor code cleanup: making tests more readable; grouping related tests together; renaming tests; un-ignore working tests; removing duplicate tests;

[GitHub] [beam] pabloem commented on issue #11229: [BEAM-9608] Increasing scope of context managers for FnApiRunner

2020-03-30 Thread GitBox
pabloem commented on issue #11229: [BEAM-9608] Increasing scope of context managers for FnApiRunner URL: https://github.com/apache/beam/pull/11229#issuecomment-606298444 @robertwb thoughts? This is an automated message from t

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r400545743 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.j

[GitHub] [beam] scwhittle commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-30 Thread GitBox
scwhittle commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606294220 I pushed fixes for both changes you requested but it isn't letting me close your changes requested . For t

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r400543326 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.j

[GitHub] [beam] lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
lukemin89 commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r400543326 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.j

[GitHub] [beam] scwhittle commented on a change in pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-30 Thread GitBox
scwhittle commented on a change in pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#discussion_r400541708 ## File path: runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/

[GitHub] [beam] ibzib opened a new pull request #11267: [BEAM-9638] Strengthen worker region & zone options tests.

2020-03-30 Thread GitBox
ibzib opened a new pull request #11267: [BEAM-9638] Strengthen worker region & zone options tests. URL: https://github.com/apache/beam/pull/11267 These tests were too loose before, see the jira for details. Thank you for your contribution! Follow this

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606282119 > Did you want to update the validation in [Python](https://github.com/apache/beam/blob/33d78c56dcffe73aaff540ee360

[GitHub] [beam] lukecwik commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
lukecwik commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606281271 Did you want to update the validation in [Python](https://github.com/apache/beam/blob/33d78c56dcffe73aaff540ee360d8

[GitHub] [beam] boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz commented on issue #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266#issuecomment-606275711 Run Dataflow ValidatesRunner This is an automated

[GitHub] [beam] boyuanzz opened a new pull request #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful.

2020-03-30 Thread GitBox
boyuanzz opened a new pull request #11266: [BEAM-9636] Using TimerFamily should also be considered as stateful. URL: https://github.com/apache/beam/pull/11266 **Please** add a meaningful description for your change here Thank you for your contribution!

[GitHub] [beam] udim commented on issue #11241: [BEAM-5422] Document DynamicDestinations.getTable uniqueness requirement

2020-03-30 Thread GitBox
udim commented on issue #11241: [BEAM-5422] Document DynamicDestinations.getTable uniqueness requirement URL: https://github.com/apache/beam/pull/11241#issuecomment-606274211 Run Java PreCommit This is an automated message fr

[GitHub] [beam] stale[bot] commented on issue #10709: [DO NOT MERGE] Run Python RC Validation Tests

2020-03-30 Thread GitBox
stale[bot] commented on issue #10709: [DO NOT MERGE] Run Python RC Validation Tests URL: https://github.com/apache/beam/pull/10709#issuecomment-606267717 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If y

[GitHub] [beam] youngoli commented on a change in pull request #11257: [BEAM-3301] Create runtime invokers for SDF methods.

2020-03-30 Thread GitBox
youngoli commented on a change in pull request #11257: [BEAM-3301] Create runtime invokers for SDF methods. URL: https://github.com/apache/beam/pull/11257#discussion_r400511146 ## File path: sdks/go/pkg/beam/core/runtime/exec/sdf_invokers.go ## @@ -0,0 +1,131 @@ +// Licens

[GitHub] [beam] tvalentyn commented on a change in pull request #11265: Fix a Py2/3 incompatibility in profiler.

2020-03-30 Thread GitBox
tvalentyn commented on a change in pull request #11265: Fix a Py2/3 incompatibility in profiler. URL: https://github.com/apache/beam/pull/11265#discussion_r400511384 ## File path: sdks/python/apache_beam/utils/profiler.py ## @@ -88,7 +88,11 @@ def __exit__(self, *args):

[GitHub] [beam] lukecwik commented on a change in pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#discussion_r400507824 ## File path: runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/b

[GitHub] [beam] lukecwik merged pull request #10980: Address leftover styling comments from PR/10892

2020-03-30 Thread GitBox
lukecwik merged pull request #10980: Address leftover styling comments from PR/10892 URL: https://github.com/apache/beam/pull/10980 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-30 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606248199 Run Spotless PreCommit This is an automated

[GitHub] [beam] lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream

2020-03-30 Thread GitBox
lukecwik commented on issue #11096: [BEAM-9399] Change the redirection of System.err to be a custom PrintStream URL: https://github.com/apache/beam/pull/11096#issuecomment-606248141 Run Java PreCommit This is an automated mes

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400470967 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400468512 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400476127 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400489288 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400478381 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400467076 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400480966 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400467966 ## File path: model/job-management/src/main/proto/beam_artifact_api.proto

[GitHub] [beam] lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#discussion_r400490551 ## File path: sdks/python/apache_beam/runners/portability/artifact_service.py

[GitHub] [beam] robertwb commented on issue #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers

2020-03-30 Thread GitBox
robertwb commented on issue #11199: [BEAM-9562][WIP] Update Timer encoding with respect of dynamic timers URL: https://github.com/apache/beam/pull/11199#issuecomment-606244698 The proto changes look fine. To not overlap work, I will let Luke take over this PR, but if there is anything spec

[GitHub] [beam] iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
iemejia commented on issue #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#issuecomment-606243244 CC: @lukecwik you may be interested on taking a quick look since it seems you authored `UnownedOutputStream` --

[GitHub] [beam] iemejia commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
iemejia commented on a change in pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263#discussion_r400485597 ## File path: sdks/java/core/src/test/java/org/apache/beam/sdk/util/UnownedOutputStreamTest.jav

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11265: Fix a Py2/3 incompatibility in profiler.

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11265: Fix a Py2/3 incompatibility in profiler. URL: https://github.com/apache/beam/pull/11265#discussion_r400483969 ## File path: sdks/python/apache_beam/utils/profiler.py ## @@ -88,7 +88,11 @@ def __exit__(self, *args):

[GitHub] [beam] Hannah-Jiang commented on issue #11265: Fix a Py2/3 incompatibility in profiler.

2020-03-30 Thread GitBox
Hannah-Jiang commented on issue #11265: Fix a Py2/3 incompatibility in profiler. URL: https://github.com/apache/beam/pull/11265#issuecomment-606237227 LGTM This is an automated message from the Apache Git Service. To respond t

[GitHub] [beam] mxm commented on a change in pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server.

2020-03-30 Thread GitBox
mxm commented on a change in pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server. URL: https://github.com/apache/beam/pull/11189#discussion_r400476144 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -285,10 +289,

[GitHub] [beam] tvalentyn commented on issue #11265: Fix a Py2/3 incompatibility in profiler.

2020-03-30 Thread GitBox
tvalentyn commented on issue #11265: Fix a Py2/3 incompatibility in profiler. URL: https://github.com/apache/beam/pull/11265#issuecomment-606230369 R: @Hannah-Jiang This is an automated message from the Apache Git Service. To

[GitHub] [beam] tvalentyn opened a new pull request #11265: Fix a Py2/3 incompatibility in profiler.

2020-03-30 Thread GitBox
tvalentyn opened a new pull request #11265: Fix a Py2/3 incompatibility in profiler. URL: https://github.com/apache/beam/pull/11265 Pre-Commit Tests Status (on master branch) --- |Jav

[GitHub] [beam] aaltay merged pull request #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
aaltay merged pull request #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262 This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python

2020-03-30 Thread GitBox
Hannah-Jiang commented on a change in pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#discussion_r400467345 ## File path: sdks/python/container/license_scripts/dep_urls_py.yaml ## @@ -0,0 +1,116 @@ +#

[GitHub] [beam] lukecwik commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606217331 Run JavaPortabilityApi PreCommit This is an

[GitHub] [beam] lukecwik commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service.

2020-03-30 Thread GitBox
lukecwik commented on issue #11203: [BEAM-9577] Define and implement dependency-aware artifact staging service. URL: https://github.com/apache/beam/pull/11203#issuecomment-606217265 Run Portable_Python PreCommit This is an au

[GitHub] [beam] lukecwik commented on a change in pull request #11205: [BEAM-9578] Enumerating artifacts is too expensive in Java

2020-03-30 Thread GitBox
lukecwik commented on a change in pull request #11205: [BEAM-9578] Enumerating artifacts is too expensive in Java URL: https://github.com/apache/beam/pull/11205#discussion_r400458940 ## File path: model/pipeline/src/main/proto/beam_runner_api.proto ## @@ -1146,33 +1146,41

[GitHub] [beam] robertwb opened a new pull request #11264: [BEAM-9496] Add to_dataframe and to_pcollection APIs.

2020-03-30 Thread GitBox
robertwb opened a new pull request #11264: [BEAM-9496] Add to_dataframe and to_pcollection APIs. URL: https://github.com/apache/beam/pull/11264 This may be a more natural interface for mixing pandas-style code with Beam. It does have the downside of being inefficient if dataframe

[GitHub] [beam] lukecwik merged pull request #11231: [BEAM-4374] Shortids for the Go SDK

2020-03-30 Thread GitBox
lukecwik merged pull request #11231: [BEAM-4374] Shortids for the Go SDK URL: https://github.com/apache/beam/pull/11231 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [beam] ibzib merged pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server.

2020-03-30 Thread GitBox
ibzib merged pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server. URL: https://github.com/apache/beam/pull/11189 This is an automated message from the Apache Git Service. To respond to th

[GitHub] [beam] ibzib commented on a change in pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server.

2020-03-30 Thread GitBox
ibzib commented on a change in pull request #11189: [BEAM-9446] Retain unknown arguments when using uber jar job server. URL: https://github.com/apache/beam/pull/11189#discussion_r400448482 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -285,10 +28

[GitHub] [beam] davidyan74 commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
davidyan74 commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262#issuecomment-606204900 Thank you. I tried the changes in this PR in the notebook and it fixed the issue. --

[GitHub] [beam] lukemin89 opened a new pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream

2020-03-30 Thread GitBox
lukemin89 opened a new pull request #11263: [BEAM-9325] Override proper write method in UnownedOutputStream URL: https://github.com/apache/beam/pull/11263 org.apache.beam.sdk.util.UnownedOutputStream does not override the method `public void write(byte b[], int off, int len) throws IOExc

[GitHub] [beam] aaltay commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
aaltay commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262#issuecomment-606188524 retest this please This is an automated message from the Apache Git Servi

[GitHub] [beam] KevinGG edited a comment on issue #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
KevinGG edited a comment on issue #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262#issuecomment-606187708 > How do you test this? Since the change is in the string template? We have the integration test in the container image rele

[GitHub] [beam] KevinGG commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
KevinGG commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262#issuecomment-606187708 > How do you test this? Since the change is in the string template? We have the integration test in the container image release pro

[GitHub] [beam] KevinGG commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
KevinGG commented on issue #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262#issuecomment-606179630 yapf formatted lint passed locally R: @aaltay R: @davidyan74 R: @rohdesamuel PTAL, thx! ---

[GitHub] [beam] KevinGG opened a new pull request #11262: [BEAM-7923] Fix datatable on notebook reloading

2020-03-30 Thread GitBox
KevinGG opened a new pull request #11262: [BEAM-7923] Fix datatable on notebook reloading URL: https://github.com/apache/beam/pull/11262 1. The problem: when a user reloads the browser window of a jupyter notebook page, the javascripts of all output areas are executed in an arbi

[GitHub] [beam] boyuanzz commented on issue #11216: [BEAM-9562] Remove TimerSpec from Proto

2020-03-30 Thread GitBox
boyuanzz commented on issue #11216: [BEAM-9562] Remove TimerSpec from Proto URL: https://github.com/apache/beam/pull/11216#issuecomment-606161054 > @boyuanzz @lukecwik Run Dataflow ValidatesRunner (beam_PostCommit_Java_ValidatesRunner_Dataflow) started failing after this PR: > https://b

  1   2   >