[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] mravi commented on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
mravi commented on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702524362 Run PythonFormatter PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] mravi commented on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
mravi commented on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702516384 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] codecov[bot] edited a comment on pull request #12883: [BEAM-7746] Add more typing to metrics

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12883: URL: https://github.com/apache/beam/pull/12883#issuecomment-699197950 # [Codecov](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=h1) Report > Merging [#12883](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12883: [BEAM-7746] Add more typing to metrics

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12883: URL: https://github.com/apache/beam/pull/12883#issuecomment-699197950 # [Codecov](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=h1) Report > Merging [#12883](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12883: [BEAM-7746] Add more typing to metrics

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12883: URL: https://github.com/apache/beam/pull/12883#issuecomment-699197950 # [Codecov](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=h1) Report > Merging [#12883](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12883: [BEAM-7746] Add more typing to metrics

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12883: URL: https://github.com/apache/beam/pull/12883#issuecomment-699197950 # [Codecov](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=h1) Report > Merging [#12883](https://codecov.io/gh/apache/beam/pull/12883?src=pr&el=desc) into

[GitHub] [beam] chadrik commented on pull request #12881: [BEAM-7746] Get mypy passing on runners.worker

2020-10-01 Thread GitBox
chadrik commented on pull request #12881: URL: https://github.com/apache/beam/pull/12881#issuecomment-702501121 Done. On Thu, Oct 1, 2020 at 5:10 PM Robert Bradshaw wrote: > Yes, let's fix ProcessBundleProgressMetadataResponse, drop some TODOs, > enable what checks we ca

[GitHub] [beam] codecov[bot] edited a comment on pull request #12881: [BEAM-7746] Get mypy passing on runners.worker

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12881: URL: https://github.com/apache/beam/pull/12881#issuecomment-696540479 # [Codecov](https://codecov.io/gh/apache/beam/pull/12881?src=pr&el=h1) Report > Merging [#12881](https://codecov.io/gh/apache/beam/pull/12881?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] kennknowles merged pull request #12970: [BEAM-8024] Add dataflow and flink runner to JPMS test

2020-10-01 Thread GitBox
kennknowles merged pull request #12970: URL: https://github.com/apache/beam/pull/12970 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tudorm commented on a change in pull request #12994: Add support for gracefully aborting workers.

2020-10-01 Thread GitBox
tudorm commented on a change in pull request #12994: URL: https://github.com/apache/beam/pull/12994#discussion_r498598992 ## File path: runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/util/common/worker/MapTaskExecutorTest.java #

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12924: [BEAM-10124] Add ContextualTextIO

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12924: URL: https://github.com/apache/beam/pull/12924#issuecomment-698040036 # [Codecov](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=h1) Report > Merging [#12924](https://codecov.io/gh/apache/beam/pull/12924?src=pr&el=desc) into

[GitHub] [beam] kennknowles commented on a change in pull request #12994: Add support for gracefully aborting workers.

2020-10-01 Thread GitBox
kennknowles commented on a change in pull request #12994: URL: https://github.com/apache/beam/pull/12994#discussion_r498595635 ## File path: runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/util/common/worker/MapTaskExecutorTest.java

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] kennknowles opened a new pull request #12997: [BEAM-10991] Adjust CHANGES.md to indicate rollback of fix for BEAM-8543

2020-10-01 Thread GitBox
kennknowles opened a new pull request #12997: URL: https://github.com/apache/beam/pull/12997 In reverting the fix for BEAM-8543 I also reverted the edit to CHANGES.md. Instead, CHANGES.md should have forward information about the known issues. This PR adds it. I have also added this to my

[GitHub] [beam] tudorm commented on pull request #12994: Add support for gracefully aborting workers.

2020-10-01 Thread GitBox
tudorm commented on pull request #12994: URL: https://github.com/apache/beam/pull/12994#issuecomment-702493438 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] kennknowles commented on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid

2020-10-01 Thread GitBox
kennknowles commented on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702488953 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [beam] kennknowles commented on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid

2020-10-01 Thread GitBox
kennknowles commented on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702488814 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [beam] codecov[bot] edited a comment on pull request #12971: [BEAM-10988] Partition dataframes according to size estimates.

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12971: URL: https://github.com/apache/beam/pull/12971#issuecomment-701054283 # [Codecov](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=h1) Report > Merging [#12971](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=desc) into

[GitHub] [beam] tvalentyn merged pull request #12790: [BEAM-10862] Handle empty tfrecord files within a glob

2020-10-01 Thread GitBox
tvalentyn merged pull request #12790: URL: https://github.com/apache/beam/pull/12790 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] tvalentyn commented on pull request #12790: [BEAM-10862] Handle empty tfrecord files within a glob

2020-10-01 Thread GitBox
tvalentyn commented on pull request #12790: URL: https://github.com/apache/beam/pull/12790#issuecomment-702479951 LGTM. I think CodeCov complains because TFRecord tests don't run since TensforFlow is not installed on Jenkins. Verified that these tests pass on Google-internal test in

[GitHub] [beam] aaltay commented on a change in pull request #12894: read from Pub/Sub and run as stream

2020-10-01 Thread GitBox
aaltay commented on a change in pull request #12894: URL: https://github.com/apache/beam/pull/12894#discussion_r498582933 ## File path: website/www/site/content/en/get-started/wordcount-example.md ## @@ -1207,7 +1210,7 @@ each element in the `PCollection`. {{< /highlight >}}

[GitHub] [beam] codecov[bot] edited a comment on pull request #12971: [BEAM-10988] Partition dataframes according to size estimates.

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12971: URL: https://github.com/apache/beam/pull/12971#issuecomment-701054283 # [Codecov](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=h1) Report > Merging [#12971](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=desc) into

[GitHub] [beam] aaltay commented on pull request #12282: [WIP][BEAM-7819] Python - Include message_id and publish_time in PubsubMessage

2020-10-01 Thread GitBox
aaltay commented on pull request #12282: URL: https://github.com/apache/beam/pull/12282#issuecomment-702476177 Closing the PR based on the latest comment. @tiny-dancer thank you for working on this up to this point. This is

[GitHub] [beam] aaltay closed pull request #12282: [WIP][BEAM-7819] Python - Include message_id and publish_time in PubsubMessage

2020-10-01 Thread GitBox
aaltay closed pull request #12282: URL: https://github.com/apache/beam/pull/12282 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [beam] aaltay commented on pull request #12867: Pub/Sub Lite getSplitBacklog implementation

2020-10-01 Thread GitBox
aaltay commented on pull request #12867: URL: https://github.com/apache/beam/pull/12867#issuecomment-702475859 Is this still work in progress? Do you need help? This is an automated message from the Apache Git Service. To res

[GitHub] [beam] aaltay commented on pull request #12953: Minor programming guide improvements

2020-10-01 Thread GitBox
aaltay commented on pull request #12953: URL: https://github.com/apache/beam/pull/12953#issuecomment-702475145 R: @rosetn This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] aaltay commented on pull request #12781: [BEAM-10859] Allow different project for BQ source temp dataset

2020-10-01 Thread GitBox
aaltay commented on pull request #12781: URL: https://github.com/apache/beam/pull/12781#issuecomment-702475288 Is this still a work in progress? This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [beam] codecov[bot] edited a comment on pull request #12971: [BEAM-10988] Partition dataframes according to size estimates.

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12971: URL: https://github.com/apache/beam/pull/12971#issuecomment-701054283 # [Codecov](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=h1) Report > Merging [#12971](https://codecov.io/gh/apache/beam/pull/12971?src=pr&el=desc) into

[GitHub] [beam] aaltay commented on pull request #12996: [BEAM-10545] Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
aaltay commented on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702471021 > R: @aaltay > PTAL. > > Let's talk offline to see if we need to create release branch and tags for a release. Change LGTM. I am assuming you are planning to relea

[GitHub] [beam] robertwb commented on a change in pull request #12705: [BEAM-10720][WIP] Implement StringMethods

2020-10-01 Thread GitBox
robertwb commented on a change in pull request #12705: URL: https://github.com/apache/beam/pull/12705#discussion_r498571169 ## File path: sdks/python/apache_beam/dataframe/frames.py ## @@ -905,7 +905,48 @@ def checked_callable_index(df): preserves_partition_by=part

[GitHub] [beam] codecov[bot] edited a comment on pull request #12996: [BEAM-10545] Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702463583 # [Codecov](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=h1) Report > Merging [#12996](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] KevinGG commented on pull request #12996: [BEAM-10545] Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
KevinGG commented on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702468895 R: @aaltay PTAL. Let's talk offline to see if we need to create release branch and tags for a release. ---

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12996: Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702463583 # [Codecov](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=h1) Report > Merging [#12996](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12996: Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702463583 # [Codecov](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=h1) Report > Merging [#12996](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12996: Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
codecov[bot] commented on pull request #12996: URL: https://github.com/apache/beam/pull/12996#issuecomment-702463583 # [Codecov](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=h1) Report > Merging [#12996](https://codecov.io/gh/apache/beam/pull/12996?src=pr&el=desc) into [master

[GitHub] [beam] robertwb merged pull request #12804: Fix translations.optimize_pipeline() failure on empty pipelines

2020-10-01 Thread GitBox
robertwb merged pull request #12804: URL: https://github.com/apache/beam/pull/12804 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb merged pull request #12887: [BEAM-10935] Replace @abstractproperty with @abstractmethod

2020-10-01 Thread GitBox
robertwb merged pull request #12887: URL: https://github.com/apache/beam/pull/12887 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb commented on a change in pull request #12883: [BEAM-7746] Add more typing to metrics

2020-10-01 Thread GitBox
robertwb commented on a change in pull request #12883: URL: https://github.com/apache/beam/pull/12883#discussion_r498569928 ## File path: sdks/python/apache_beam/metrics/metric.py ## @@ -101,23 +121,26 @@ def gauge(namespace, name): class DelegatingCounter(Counter): """

[GitHub] [beam] robertwb commented on pull request #12881: [BEAM-7746] Get mypy passing on runners.worker

2020-10-01 Thread GitBox
robertwb commented on pull request #12881: URL: https://github.com/apache/beam/pull/12881#issuecomment-702459845 Yes, let's fix `ProcessBundleProgressMetadataResponse`, drop some TODOs, enable what checks we can, and get this merged. --

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] KevinGG opened a new pull request #12996: Advance the jupyterlab sidepanel extension version for first release

2020-10-01 Thread GitBox
KevinGG opened a new pull request #12996: URL: https://github.com/apache/beam/pull/12996 Advance the version from 0.1.0 to 1.0.0 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] robertwb commented on pull request #12995: [BEAM-2914] Add portable merging window support to Python.

2020-10-01 Thread GitBox
robertwb commented on pull request #12995: URL: https://github.com/apache/beam/pull/12995#issuecomment-702450119 R: @angoenka This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] robertwb opened a new pull request #12995: [BEAM-2914] Add portable merging window support to Python.

2020-10-01 Thread GitBox
robertwb opened a new pull request #12995: URL: https://github.com/apache/beam/pull/12995 This adds support to the Python SDK and the Python ULR. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly a

[GitHub] [beam] codecov[bot] edited a comment on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set ear

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mi

2020-10-01 Thread GitBox
codecov[bot] commented on pull request #12993: URL: https://github.com/apache/beam/pull/12993#issuecomment-702448732 # [Codecov](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=h1) Report > Merging [#12993](https://codecov.io/gh/apache/beam/pull/12993?src=pr&el=desc) into [releas

[GitHub] [beam] tudorm commented on pull request #12994: Add support for gracefully aborting workers.

2020-10-01 Thread GitBox
tudorm commented on pull request #12994: URL: https://github.com/apache/beam/pull/12994#issuecomment-702445980 R: @kennknowles This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [beam] tudorm opened a new pull request #12994: Add support for gracefully aborting workers.

2020-10-01 Thread GitBox
tudorm opened a new pull request #12994: URL: https://github.com/apache/beam/pull/12994 On complete_work_status received (by the work progress updater thread), abort the corresponding map task executor thread. This relies on Thread.interrupt() to unblock the worker thread if blocked and r

[GitHub] [beam] rohdesamuel commented on pull request #12984: [BEAM-10994] Add ability for HotKeyLogger to log a key

2020-10-01 Thread GitBox
rohdesamuel commented on pull request #12984: URL: https://github.com/apache/beam/pull/12984#issuecomment-702443622 Run Java_Examples_Dataflow_Java11 PreCommit This is an automated message from the Apache Git Service. To resp

[GitHub] [beam] kennknowles opened a new pull request #12993: [BEAM-10991] Cherrypick #12980 to 2.25.0: Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid

2020-10-01 Thread GitBox
kennknowles opened a new pull request #12993: URL: https://github.com/apache/beam/pull/12993 This change is a serious regression in 2.24.0 so should be cherrypicked back to 2.25.0 Thank you for your contribution! Follow this checklist to help us incor

[GitHub] [beam] kennknowles merged pull request #12980: [BEAM-10991] Revert "[BEAM-8543] Dataflow streaming timers are not strictly time ordered when set earlier mid-bundle (#11924)"

2020-10-01 Thread GitBox
kennknowles merged pull request #12980: URL: https://github.com/apache/beam/pull/12980 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] lukecwik commented on a change in pull request #12881: [BEAM-7746] Get mypy passing on runners.worker

2020-10-01 Thread GitBox
lukecwik commented on a change in pull request #12881: URL: https://github.com/apache/beam/pull/12881#discussion_r498545227 ## File path: sdks/python/apache_beam/runners/worker/sdk_worker.py ## @@ -622,10 +659,12 @@ def process_bundle_progress_metadata_request(self,

[GitHub] [beam] chadrik commented on pull request #12881: [BEAM-7746] Get mypy passing on runners.worker

2020-10-01 Thread GitBox
chadrik commented on pull request #12881: URL: https://github.com/apache/beam/pull/12881#issuecomment-702430284 We're getting merge conflicts. I'm thinking of just ignoring the remaining errors, so that we can just get this merged. I can add a FIXME referencing BEAM-7746. Thoughts?

[GitHub] [beam] lukecwik merged pull request #12983: [BEAM-10994] Add hotKeyLoggingEnabled to the Java DataflowPipelineOptions

2020-10-01 Thread GitBox
lukecwik merged pull request #12983: URL: https://github.com/apache/beam/pull/12983 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] lukecwik commented on pull request #12603: [WIP][BEAM-10670] Make SparkRunner opt-out for using an SDF powered Read transform.

2020-10-01 Thread GitBox
lukecwik commented on pull request #12603: URL: https://github.com/apache/beam/pull/12603#issuecomment-702422811 @iemejia Since streaming is effectively broken due to lack of support for watermark holds. What do you think about enabling SDF for Spark and it only working in batch? I'

[GitHub] [beam] lukecwik commented on pull request #12603: [WIP][BEAM-10670] Make SparkRunner opt-out for using an SDF powered Read transform.

2020-10-01 Thread GitBox
lukecwik commented on pull request #12603: URL: https://github.com/apache/beam/pull/12603#issuecomment-702422068 > The phenomenon of microbatches producing results early I noticed it too in the past when trying to enable the Read.Unbounded tests. I could not understand why, and I thought i

[GitHub] [beam] lukecwik commented on pull request #12983: [BEAM-10994] Add hotKeyLoggingEnabled to the Java DataflowPipelineOptions

2020-10-01 Thread GitBox
lukecwik commented on pull request #12983: URL: https://github.com/apache/beam/pull/12983#issuecomment-702421331 R: @lukecwik This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] lukecwik commented on a change in pull request #12983: [BEAM-10994] Add hotKeyLoggingEnabled to the Java DataflowPipelineOptions

2020-10-01 Thread GitBox
lukecwik commented on a change in pull request #12983: URL: https://github.com/apache/beam/pull/12983#discussion_r498526932 ## File path: runners/google-cloud-dataflow-java/src/test/java/org/apache/beam/runners/dataflow/options/DataflowPipelineOptionsTest.java ## @@ -243,5 +24

[GitHub] [beam] robinyqiu merged pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
robinyqiu merged pull request #12991: URL: https://github.com/apache/beam/pull/12991 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] pabloem commented on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
pabloem commented on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702406799 I believe failures now are due to the missing Dataflow worker containers This is an automated message from the Ap

[GitHub] [beam] codecov[bot] edited a comment on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702339339 # [Codecov](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=h1) Report > Merging [#12991](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
codecov[bot] commented on pull request #12992: URL: https://github.com/apache/beam/pull/12992#issuecomment-702403670 # [Codecov](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=h1) Report > Merging [#12992](https://codecov.io/gh/apache/beam/pull/12992?src=pr&el=desc) into [master

[GitHub] [beam] ibzib merged pull request #12989: [BEAM-9026] Avoid side effects from positional value provider arguments.

2020-10-01 Thread GitBox
ibzib merged pull request #12989: URL: https://github.com/apache/beam/pull/12989 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] mravi opened a new pull request #12992: [BEAM-10848] Initialize gauge to 0

2020-10-01 Thread GitBox
mravi opened a new pull request #12992: URL: https://github.com/apache/beam/pull/12992 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] robinyqiu commented on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
robinyqiu commented on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702394534 Run Python_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [beam] y1chi commented on pull request #11720: [BEAM-9729] Ignore bundle registration requests.

2020-10-01 Thread GitBox
y1chi commented on pull request #11720: URL: https://github.com/apache/beam/pull/11720#issuecomment-702393496 @robertwb friendly ping, should we merge this PR This is an automated message from the Apache Git Service. To respo

[GitHub] [beam] robinyqiu commented on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
robinyqiu commented on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702389561 @pabloem Could you take a look at the failing tests? Are they relevant? This is an automated message from the A

[GitHub] [beam] kennknowles merged pull request #12988: Remove redundant Dataflow Runner Harness precommits

2020-10-01 Thread GitBox
kennknowles merged pull request #12988: URL: https://github.com/apache/beam/pull/12988 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] codecov[bot] edited a comment on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702339339 # [Codecov](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=h1) Report > Merging [#12991](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=desc) into

[GitHub] [beam] lukecwik merged pull request #12874: [BEAM-10930] Use dense JSON responses for BigQueryIO interactions

2020-10-01 Thread GitBox
lukecwik merged pull request #12874: URL: https://github.com/apache/beam/pull/12874 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] tvalentyn commented on pull request #12964: [BEAM-10639] Integration test that exercises --setup_file flag on Dataflow

2020-10-01 Thread GitBox
tvalentyn commented on pull request #12964: URL: https://github.com/apache/beam/pull/12964#issuecomment-702353755 Also checked the worker startup logs for 2020-09-30_06_48_59-6954590719325885463, which look reasonable: ``` 2020-09-30 06:55:19.396 PDT Executing: /usr/local/bin/pip i

[GitHub] [beam] codecov[bot] edited a comment on pull request #12990: [BEAM-9547] Implement dot method for dataframes.

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12990: URL: https://github.com/apache/beam/pull/12990#issuecomment-702271768 # [Codecov](https://codecov.io/gh/apache/beam/pull/12990?src=pr&el=h1) Report > Merging [#12990](https://codecov.io/gh/apache/beam/pull/12990?src=pr&el=desc) into

[GitHub] [beam] udim commented on pull request #12989: [BEAM-9026] Avoid side effects from positional value provider arguments.

2020-10-01 Thread GitBox
udim commented on pull request #12989: URL: https://github.com/apache/beam/pull/12989#issuecomment-702351583 Looks like a good workaround. I also commented on the bug This is an automated message from the Apache Git Service.

[GitHub] [beam] rohdesamuel commented on pull request #12983: [BEAM-10994] Add hotKeyLoggingEnabled to the Java DataflowPipelineOptions

2020-10-01 Thread GitBox
rohdesamuel commented on pull request #12983: URL: https://github.com/apache/beam/pull/12983#issuecomment-702347773 > If this is a feature that will always be on then do you want to use experiments instead or do you want customers to have control over this always? Because this will e

[GitHub] [beam] yichuan66 edited a comment on pull request #12987: Yichuan66/fix typo

2020-10-01 Thread GitBox
yichuan66 edited a comment on pull request #12987: URL: https://github.com/apache/beam/pull/12987#issuecomment-702344005 > Thanks! Happy to review PRs or help you find an appropriate reviewer :) Thanks Brian! I had a question on how I should learn the Go SDK code efficiently because

[GitHub] [beam] codecov[bot] edited a comment on pull request #12954: [DO NOT MERGE] Update IO dependencies

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12954: URL: https://github.com/apache/beam/pull/12954#issuecomment-699904807 # [Codecov](https://codecov.io/gh/apache/beam/pull/12954?src=pr&el=h1) Report > Merging [#12954](https://codecov.io/gh/apache/beam/pull/12954?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12991: [cherry-pick][BEAM-10950] Passing project properly in BQSource

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12991: URL: https://github.com/apache/beam/pull/12991#issuecomment-702339339 # [Codecov](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=h1) Report > Merging [#12991](https://codecov.io/gh/apache/beam/pull/12991?src=pr&el=desc) into

[GitHub] [beam] robertwb commented on pull request #12990: [BEAM-9547] Implement dot method for dataframes.

2020-10-01 Thread GitBox
robertwb commented on pull request #12990: URL: https://github.com/apache/beam/pull/12990#issuecomment-702344216 R: @apilloud This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] yichuan66 commented on pull request #12987: Yichuan66/fix typo

2020-10-01 Thread GitBox
yichuan66 commented on pull request #12987: URL: https://github.com/apache/beam/pull/12987#issuecomment-702344005 > Thanks! Happy to review PRs or help you find an appropriate reviewer :) Thanks Brian! I had a question on how I should learn the Go SDK code efficiently because I want

[GitHub] [beam] tvalentyn merged pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report

2020-10-01 Thread GitBox
tvalentyn merged pull request #12927: URL: https://github.com/apache/beam/pull/12927 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12954: [DO NOT MERGE] Update IO dependencies

2020-10-01 Thread GitBox
codecov[bot] edited a comment on pull request #12954: URL: https://github.com/apache/beam/pull/12954#issuecomment-699904807 # [Codecov](https://codecov.io/gh/apache/beam/pull/12954?src=pr&el=h1) Report > Merging [#12954](https://codecov.io/gh/apache/beam/pull/12954?src=pr&el=desc) into

[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report

2020-10-01 Thread GitBox
tvalentyn commented on a change in pull request #12927: URL: https://github.com/apache/beam/pull/12927#discussion_r498461732 ## File path: sdks/python/apache_beam/io/avroio.py ## @@ -627,11 +627,19 @@ def write_record(self, writer, value): writer.append(value) +class _

  1   2   >