[GitHub] [beam] youngoli commented on a change in pull request #12743: [DO NOT MERGE] Update Beam website to release 2.24.0.

youngoli commented on a change in pull request #12743: URL: https://github.com/apache/beam/pull/12743#discussion_r490569708 ## File path: website/www/site/content/en/get-started/downloads.md ## @@ -87,16 +87,24 @@ versions denoted `0.x.y`. ## Releases +### 2.24.0 (2020-07-

[GitHub] [beam] youngoli commented on pull request #12743: Update Beam website to release 2.24.0.

youngoli commented on pull request #12743: URL: https://github.com/apache/beam/pull/12743#issuecomment-694507294 This should be safe to merge, but I'm just waiting a bit until the dist.apache link is good This is an automate

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr&el=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490571910 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,72 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-07-29 00:00:01 -0800 +# Upd

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490572359 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,77 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-09-17 00:00:01 -0800 +categ

[GitHub] [beam] udim commented on a change in pull request #12745: Add a blog post for Apache Beam 2.24.0.

udim commented on a change in pull request #12745: URL: https://github.com/apache/beam/pull/12745#discussion_r490571910 ## File path: website/www/site/content/en/blog/beam-2.24.0.md ## @@ -0,0 +1,72 @@ +--- +title: "Apache Beam 2.24.0" +date: 2020-07-29 00:00:01 -0800 +# Upd

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] KevinGG commented on a change in pull request #12704: [BEAM-10603] Implement the new Large Source Recording API.

KevinGG commented on a change in pull request #12704: URL: https://github.com/apache/beam/pull/12704#discussion_r490593309 ## File path: sdks/python/apache_beam/runners/interactive/display/pcoll_visualization.py ## @@ -407,13 +392,7 @@ def _display_dataframe(self, data, update

[GitHub] [beam] robertwb commented on pull request #12491: Avoid re-encoding row types.

robertwb commented on pull request #12491: URL: https://github.com/apache/beam/pull/12491#issuecomment-694536569 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=desc) into

[GitHub] [beam] ibzib commented on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

ibzib commented on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-694540273 > I'm a little unsure about the overhead this adds in terms of the number of options. Would it make sense to instead use the following format? For example: > > ``` > --env

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=desc) into

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694547267 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-694547392 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] boyuanzz merged pull request #12678: [BEAM-10703] Add a step property for shardable states during Dataflow graph translation (Java)

boyuanzz merged pull request #12678: URL: https://github.com/apache/beam/pull/12678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr&el=desc) into

[GitHub] [beam] robertwb commented on pull request #12844: [BEAM-10894] Support for more pandas formats.

robertwb commented on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694559264 Rebased on https://github.com/apache/beam/pull/12841 This is an automated message from the Apache Git Service. T

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=desc) into

[GitHub] [beam] robertwb merged pull request #12832: [BEAM-10906] Add basic ToRows transform.

robertwb merged pull request #12832: URL: https://github.com/apache/beam/pull/12832 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb merged pull request #12491: Avoid re-encoding row types.

robertwb merged pull request #12491: URL: https://github.com/apache/beam/pull/12491 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr&el=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12844: [BEAM-10894] Support for more pandas formats.

codecov[bot] commented on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694568117 # [Codecov](https://codecov.io/gh/apache/beam/pull/12844?src=pr&el=h1) Report > Merging [#12844](https://codecov.io/gh/apache/beam/pull/12844?src=pr&el=desc) into [master

[GitHub] [beam] codecov[bot] edited a comment on pull request #12844: [BEAM-10894] Support for more pandas formats.

codecov[bot] edited a comment on pull request #12844: URL: https://github.com/apache/beam/pull/12844#issuecomment-694568117 # [Codecov](https://codecov.io/gh/apache/beam/pull/12844?src=pr&el=h1) Report > Merging [#12844](https://codecov.io/gh/apache/beam/pull/12844?src=pr&el=desc) into

[GitHub] [beam] robertwb commented on a change in pull request #12727: [BEAM-10844] Add experiment option prebuild_sdk_container to prebuild python sdk container with dependencies.

robertwb commented on a change in pull request #12727: URL: https://github.com/apache/beam/pull/12727#discussion_r490632829 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -994,6 +994,23 @@ def _add_argparse_args(cls, parser): 'staged in the

[GitHub] [beam] robertwb commented on pull request #12819: [BEAM-9561] Initial framework for testing pandas website docs.

robertwb commented on pull request #12819: URL: https://github.com/apache/beam/pull/12819#issuecomment-694586974 Same test_flatten_no_pcollections failure. This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] jayendra13 commented on a change in pull request #12596: [BEAM-10498] [WIP] Eliminate nullability errors from :sdks:java:extensions:sql:zetasql

jayendra13 commented on a change in pull request #12596: URL: https://github.com/apache/beam/pull/12596#discussion_r490648800 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/SqlAnalyzer.java ## @@ -218,6 +218,8 @@ SimpleC

[GitHub] [beam] udim commented on a change in pull request #12805: [BEAM-10867] Add file generation to GcsPath

udim commented on a change in pull request #12805: URL: https://github.com/apache/beam/pull/12805#discussion_r490652676 ## File path: sdks/java/extensions/google-cloud-platform-core/src/test/java/org/apache/beam/sdk/extensions/gcp/util/gcsfs/GcsPathTest.java ## @@ -175,6 +175,

[GitHub] [beam] udim commented on pull request #12805: [BEAM-10867] Add file generation to GcsPath

udim commented on pull request #12805: URL: https://github.com/apache/beam/pull/12805#issuecomment-694605629 > Based on an offline chat, seems like we are adding support for GCS versioning here. It'll be good if we can can have a short doc and/or an email to to dev list to make sure we agr

[GitHub] [beam] lukecwik commented on pull request #12603: [WIP][BEAM-10670] Make SparkRunner opt-out for using an SDF powered Read transform.

lukecwik commented on pull request #12603: URL: https://github.com/apache/beam/pull/12603#issuecomment-694659139 @iemejia I have updated the code and added a `SparkProcessedKeyedElements` using `updateStateByKey` to evaluate a splittable DoFn. I based the logic off of the `SparkGroupAlsoBy

<    1   2