[GitHub] [beam] youngoli merged pull request #12845: [BEAM-6928] Update changelog with WriteToBigQuery changed requirements.

2020-09-15 Thread GitBox
youngoli merged pull request #12845: URL: https://github.com/apache/beam/pull/12845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] youngoli commented on a change in pull request #12588: [BEAM-7009] Add Go SDK Standard Coders yaml tests.

2020-09-15 Thread GitBox
youngoli commented on a change in pull request #12588: URL: https://github.com/apache/beam/pull/12588#discussion_r489139814 ## File path: sdks/go/test/regression/coders/fromyaml/fromyaml.go ## @@ -0,0 +1,415 @@ +// Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [beam] youngoli commented on a change in pull request #12588: [BEAM-7009] Add Go SDK Standard Coders yaml tests.

2020-09-15 Thread GitBox
youngoli commented on a change in pull request #12588: URL: https://github.com/apache/beam/pull/12588#discussion_r489138265 ## File path: sdks/go/pkg/beam/core/runtime/exec/coder.go ## @@ -434,6 +600,267 @@ func convertIfNeeded(v interface{}, allocated *FullValue) *FullValue

[GitHub] [beam] youngoli commented on a change in pull request #12588: [BEAM-7009] Add Go SDK Standard Coders yaml tests.

2020-09-15 Thread GitBox
youngoli commented on a change in pull request #12588: URL: https://github.com/apache/beam/pull/12588#discussion_r489137571 ## File path: sdks/go/pkg/beam/core/runtime/exec/coder.go ## @@ -81,24 +82,82 @@ func MakeElementEncoder(c *coder.Coder) ElementEncoder {

[GitHub] [beam] udim commented on a change in pull request #12832: Add basic ToRows transform.

2020-09-15 Thread GitBox
udim commented on a change in pull request #12832: URL: https://github.com/apache/beam/pull/12832#discussion_r489114329 ## File path: sdks/python/apache_beam/transforms/core.py ## @@ -2519,6 +2520,45 @@ def expand(self, pcoll): (*(key + value +class

[GitHub] [beam] udim commented on a change in pull request #12702: [BEAM-10603] Fix BCJ termination conditions.

2020-09-15 Thread GitBox
udim commented on a change in pull request #12702: URL: https://github.com/apache/beam/pull/12702#discussion_r489117331 ## File path: sdks/python/apache_beam/runners/interactive/background_caching_job.py ## @@ -87,11 +87,10 @@ def _should_end_condition_checker(self):

[GitHub] [beam] mouyang commented on pull request #12730: [BEAM-10532] convert NUMERIC field in TableSchema

2020-09-15 Thread GitBox
mouyang commented on pull request #12730: URL: https://github.com/apache/beam/pull/12730#issuecomment-693114519 > Seems like this will be covered by #12711 > > cc: @robinyqiu It does. This has additional unit tests not in that PR.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12822: URL: https://github.com/apache/beam/pull/12822#issuecomment-690950662 # [Codecov](https://codecov.io/gh/apache/beam/pull/12822?src=pr=h1) Report > Merging [#12822](https://codecov.io/gh/apache/beam/pull/12822?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12822: URL: https://github.com/apache/beam/pull/12822#issuecomment-690950662 # [Codecov](https://codecov.io/gh/apache/beam/pull/12822?src=pr=h1) Report > Merging [#12822](https://codecov.io/gh/apache/beam/pull/12822?src=pr=desc) into

[GitHub] [beam] udim edited a comment on pull request #12832: Add basic ToRows transform.

2020-09-15 Thread GitBox
udim edited a comment on pull request #12832: URL: https://github.com/apache/beam/pull/12832#issuecomment-693106765 Please add a JIRA to the title This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] udim commented on pull request #12832: Add basic ToRows transform.

2020-09-15 Thread GitBox
udim commented on pull request #12832: URL: https://github.com/apache/beam/pull/12832#issuecomment-693106765 Please add a JIRA to the description This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] ihji commented on a change in pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ihji commented on a change in pull request #12822: URL: https://github.com/apache/beam/pull/12822#discussion_r489094641 ## File path: sdks/python/apache_beam/io/gcp/bigquery.py ## @@ -1210,20 +1236,32 @@ def process(self, element, *schema_side_inputs): return

[GitHub] [beam] codecov[bot] edited a comment on pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12822: URL: https://github.com/apache/beam/pull/12822#issuecomment-690950662 # [Codecov](https://codecov.io/gh/apache/beam/pull/12822?src=pr=h1) Report > Merging [#12822](https://codecov.io/gh/apache/beam/pull/12822?src=pr=desc) into

[GitHub] [beam] emilymye commented on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
emilymye commented on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-693100689 Will update once I've run the x-lang example This is an automated message from the Apache Git Service. To

[GitHub] [beam] codecov[bot] edited a comment on pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12505: URL: https://github.com/apache/beam/pull/12505#issuecomment-684502100 # [Codecov](https://codecov.io/gh/apache/beam/pull/12505?src=pr=h1) Report > Merging [#12505](https://codecov.io/gh/apache/beam/pull/12505?src=pr=desc) into

[GitHub] [beam] rosetn edited a comment on pull request #12835: [BEAM-10889] Add a note about BatchElements on GroupIntoBatches pages.

2020-09-15 Thread GitBox
rosetn edited a comment on pull request #12835: URL: https://github.com/apache/beam/pull/12835#issuecomment-693100165 Ideally, we should have an entry in the Transforms catalog for BatchElements, but I can understand needing this reference now. FYI @davidcavazos

[GitHub] [beam] emilymye commented on a change in pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
emilymye commented on a change in pull request #12505: URL: https://github.com/apache/beam/pull/12505#discussion_r489092740 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/Environments.java ## @@ -357,4 +354,14 @@ private

[GitHub] [beam] rosetn commented on pull request #12835: [BEAM-10889] Add a note about BatchElements on GroupIntoBatches pages.

2020-09-15 Thread GitBox
rosetn commented on pull request #12835: URL: https://github.com/apache/beam/pull/12835#issuecomment-693100165 Ideally, we should have an entry in the Transforms catalog for BatchElements, but I can understand needing this reference now.

[GitHub] [beam] pabloem commented on pull request #12721: [BEAM-10871] Add deidentify for FhirIO connector

2020-09-15 Thread GitBox
pabloem commented on pull request #12721: URL: https://github.com/apache/beam/pull/12721#issuecomment-693099477 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] apilloud commented on a change in pull request #12643: [BEAM-10438] Update SupportedZetaSqlBuiltinFunctions and support math functions

2020-09-15 Thread GitBox
apilloud commented on a change in pull request #12643: URL: https://github.com/apache/beam/pull/12643#discussion_r489084327 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/bigquery/BeamBigQuerySqlDialect.java ## @@ -87,16

[GitHub] [beam] ihji removed a comment on pull request #12852: [BEAM-10890] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ihji removed a comment on pull request #12852: URL: https://github.com/apache/beam/pull/12852#issuecomment-693091570 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] ihji commented on pull request #12852: [BEAM-10890] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ihji commented on pull request #12852: URL: https://github.com/apache/beam/pull/12852#issuecomment-693091533 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [beam] ihji commented on a change in pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ihji commented on a change in pull request #12822: URL: https://github.com/apache/beam/pull/12822#discussion_r48908 ## File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py ## @@ -538,6 +539,13 @@ def _insert_all_rows( try: response =

[GitHub] [beam] apilloud commented on a change in pull request #12596: [BEAM-10498] [WIP] Eliminate nullability errors from :sdks:java:extensions:sql:zetasql

2020-09-15 Thread GitBox
apilloud commented on a change in pull request #12596: URL: https://github.com/apache/beam/pull/12596#discussion_r489077798 ## File path: sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/translation/SqlOperators.java ## @@ -132,6

[GitHub] [beam] monicadsong commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r489074298 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] boyuanzz commented on a change in pull request #12836: [BEAM-10887] Expose clear() to FnApiTimer.

2020-09-15 Thread GitBox
boyuanzz commented on a change in pull request #12836: URL: https://github.com/apache/beam/pull/12836#discussion_r489074544 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/state/Timer.java ## @@ -81,6 +81,9 @@ */ void setRelative(); + /** Clears the

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] robertwb commented on pull request #12819: [BEAM-9561] Initial framework for testing pandas website docs.

2020-09-15 Thread GitBox
robertwb commented on pull request #12819: URL: https://github.com/apache/beam/pull/12819#issuecomment-693021255 The apache_beam.transforms.ptransform_test.PTransformTest.test_flatten_no_pcollections failure looks unrelated.

[GitHub] [beam] apilloud merged pull request #12818: [BEAM-10875] Support NUMERIC in spanner schema parser

2020-09-15 Thread GitBox
apilloud merged pull request #12818: URL: https://github.com/apache/beam/pull/12818 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] chamikaramj commented on pull request #12730: [BEAM-10532] convert NUMERIC field in TableSchema

2020-09-15 Thread GitBox
chamikaramj commented on pull request #12730: URL: https://github.com/apache/beam/pull/12730#issuecomment-693011358 Seems like this will be covered by https://github.com/apache/beam/pull/12711 cc: @robinyqiu This is

[GitHub] [beam] chamikaramj commented on pull request #12767: Remove experimental annotations for BQ storage API source

2020-09-15 Thread GitBox
chamikaramj commented on pull request #12767: URL: https://github.com/apache/beam/pull/12767#issuecomment-693010464 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] chamikaramj commented on pull request #12767: Remove experimental annotations for BQ storage API source

2020-09-15 Thread GitBox
chamikaramj commented on pull request #12767: URL: https://github.com/apache/beam/pull/12767#issuecomment-693010547 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-693007091 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] ibzib merged pull request #12820: [BEAM-9575] Only copy the Spark runner jar, not whatever other jars h…

2020-09-15 Thread GitBox
ibzib merged pull request #12820: URL: https://github.com/apache/beam/pull/12820 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] ihji opened a new pull request #12852: [BEAM-10890] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ihji opened a new pull request #12852: URL: https://github.com/apache/beam/pull/12852 …ests for Java SDK Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] amaliujia commented on a change in pull request #12843: [BEAM-10895] Support UNNEST an (possibly nested) array field of an struct column

2020-09-15 Thread GitBox
amaliujia commented on a change in pull request #12843: URL: https://github.com/apache/beam/pull/12843#discussion_r488992286 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnnestRel.java ## @@ -111,31 +112,41 @@ public

[GitHub] [beam] amaliujia commented on a change in pull request #12843: [BEAM-10895] Support UNNEST an (possibly nested) array field of an struct column

2020-09-15 Thread GitBox
amaliujia commented on a change in pull request #12843: URL: https://github.com/apache/beam/pull/12843#discussion_r488992286 ## File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamUnnestRel.java ## @@ -111,31 +112,41 @@ public

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] ibzib commented on pull request #12820: [BEAM-9575] Only copy the Spark runner jar, not whatever other jars h…

2020-09-15 Thread GitBox
ibzib commented on pull request #12820: URL: https://github.com/apache/beam/pull/12820#issuecomment-692979803 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] rohdesamuel commented on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-15 Thread GitBox
rohdesamuel commented on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692976118 R: @pabloem This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12799: [BEAM-10603] Add record_pipeline, clear to RM and fix duration limiter

2020-09-15 Thread GitBox
codecov[bot] commented on pull request #12799: URL: https://github.com/apache/beam/pull/12799#issuecomment-692960218 # [Codecov](https://codecov.io/gh/apache/beam/pull/12799?src=pr=h1) Report > Merging [#12799](https://codecov.io/gh/apache/beam/pull/12799?src=pr=desc) into

[GitHub] [beam] robertwb commented on pull request #12819: [BEAM-9561] Initial framework for testing pandas website docs.

2020-09-15 Thread GitBox
robertwb commented on pull request #12819: URL: https://github.com/apache/beam/pull/12819#issuecomment-692945434 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] robertwb commented on a change in pull request #12782: Overriding Dataflow Native BQSource.

2020-09-15 Thread GitBox
robertwb commented on a change in pull request #12782: URL: https://github.com/apache/beam/pull/12782#discussion_r488902538 ## File path: sdks/python/apache_beam/io/gcp/bigquery_test.py ## @@ -168,7 +168,8 @@ def test_invalid_json_neg_inf(self): @unittest.skipIf(HttpError is

[GitHub] [beam] robertwb commented on pull request #12782: Overriding Dataflow Native BQSource.

2020-09-15 Thread GitBox
robertwb commented on pull request #12782: URL: https://github.com/apache/beam/pull/12782#issuecomment-692921764 Run Portable_Python PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] TheNeuralBit commented on pull request #12705: [BEAM-10720][WIP] Implement StringMethods

2020-09-15 Thread GitBox
TheNeuralBit commented on pull request #12705: URL: https://github.com/apache/beam/pull/12705#issuecomment-692919644 > Can you rebase this? Done This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] monicadsong closed pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong closed pull request #12851: URL: https://github.com/apache/beam/pull/12851 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tvalentyn commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
tvalentyn commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488878863 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] codecov[bot] edited a comment on pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12851: URL: https://github.com/apache/beam/pull/12851#issuecomment-692881256 # [Codecov](https://codecov.io/gh/apache/beam/pull/12851?src=pr=h1) Report > Merging [#12851](https://codecov.io/gh/apache/beam/pull/12851?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] ibzib commented on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
ibzib commented on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692885360 (Filed BEAM-10901 for test flakes.) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] pabloem commented on pull request #12721: [BEAM-10871] Add deidentify for FhirIO connector

2020-09-15 Thread GitBox
pabloem commented on pull request #12721: URL: https://github.com/apache/beam/pull/12721#issuecomment-692884656 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] monicadsong commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488863704 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] monicadsong commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488865232 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] monicadsong commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488863704 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] robertwb commented on pull request #12705: [BEAM-10720][WIP] Implement StringMethods

2020-09-15 Thread GitBox
robertwb commented on pull request #12705: URL: https://github.com/apache/beam/pull/12705#issuecomment-692882655 Can you rebase this? This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] monicadsong commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488863704 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] robertwb merged pull request #12787: [BEAM-10641] Add eliminate_common_key_with_none graph optimizer

2020-09-15 Thread GitBox
robertwb merged pull request #12787: URL: https://github.com/apache/beam/pull/12787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] ibzib commented on a change in pull request #12827: [BEAM-10885] Add Avro support to Kafka table provider

2020-09-15 Thread GitBox
ibzib commented on a change in pull request #12827: URL: https://github.com/apache/beam/pull/12827#discussion_r488862816 ## File path: sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider/kafka/BeamKafkaTableTest.java ## @@ -41,27 +40,49 @@

[GitHub] [beam] codecov[bot] commented on pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
codecov[bot] commented on pull request #12851: URL: https://github.com/apache/beam/pull/12851#issuecomment-692881256 # [Codecov](https://codecov.io/gh/apache/beam/pull/12851?src=pr=h1) Report > Merging [#12851](https://codecov.io/gh/apache/beam/pull/12851?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692353567 # [Codecov](https://codecov.io/gh/apache/beam/pull/12576?src=pr=h1) Report > Merging [#12576](https://codecov.io/gh/apache/beam/pull/12576?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] codecov[bot] edited a comment on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] mxm commented on pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
mxm commented on pull request #12576: URL: https://github.com/apache/beam/pull/12576#issuecomment-692877960 Thanks! I'll have another look tomorrow but generally looks good to me. This is an automated message from the

[GitHub] [beam] codecov[bot] edited a comment on pull request #12819: [BEAM-9561] Initial framework for testing pandas website docs.

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12819: URL: https://github.com/apache/beam/pull/12819#issuecomment-691336298 # [Codecov](https://codecov.io/gh/apache/beam/pull/12819?src=pr=h1) Report > Merging [#12819](https://codecov.io/gh/apache/beam/pull/12819?src=pr=desc) into

[GitHub] [beam] codecov[bot] commented on pull request #12806: [BEAM-10869] Make WriteToPubsub output serialized PubsubMessage proto bytes when using runner v2

2020-09-15 Thread GitBox
codecov[bot] commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692876818 # [Codecov](https://codecov.io/gh/apache/beam/pull/12806?src=pr=h1) Report > Merging [#12806](https://codecov.io/gh/apache/beam/pull/12806?src=pr=desc) into

[GitHub] [beam] tvalentyn commented on a change in pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
tvalentyn commented on a change in pull request #12851: URL: https://github.com/apache/beam/pull/12851#discussion_r488853550 ## File path: sdks/python/apache_beam/transforms/stats_test.py ## @@ -89,6 +90,12 @@ def setUp(self): None, 0.1,

[GitHub] [beam] boyuanzz commented on pull request #12806: [BEAM-10869] Use PubsubMessagePayloadOnlyCoder when writing to Pubsub.

2020-09-15 Thread GitBox
boyuanzz commented on pull request #12806: URL: https://github.com/apache/beam/pull/12806#issuecomment-692865584 r: @chamikaramj This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] codecov[bot] edited a comment on pull request #12779: [BEAM-10856] Support for NestedValueProvider for Python SDK

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12779: URL: https://github.com/apache/beam/pull/12779#issuecomment-692856347 # [Codecov](https://codecov.io/gh/apache/beam/pull/12779?src=pr=h1) Report > Merging [#12779](https://codecov.io/gh/apache/beam/pull/12779?src=pr=desc) into

[GitHub] [beam] robertwb merged pull request #12812: [BEAM-10873] Stronger testing of dataframe partitioning declarations.

2020-09-15 Thread GitBox
robertwb merged pull request #12812: URL: https://github.com/apache/beam/pull/12812 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] robertwb commented on pull request #12812: [BEAM-10873] Stronger testing of dataframe partitioning declarations.

2020-09-15 Thread GitBox
robertwb commented on pull request #12812: URL: https://github.com/apache/beam/pull/12812#issuecomment-692861888 The github actions failures look completely unrelated and the tests all pass on jenkins (and locally). This

[GitHub] [beam] ibzib commented on a change in pull request #12576: [BEAM-10671] Add environment configuration fields as first-class pipeline options.

2020-09-15 Thread GitBox
ibzib commented on a change in pull request #12576: URL: https://github.com/apache/beam/pull/12576#discussion_r488839465 ## File path: sdks/python/apache_beam/options/pipeline_options.py ## @@ -1044,7 +1044,12 @@ def _add_argparse_args(cls, parser): 'form {"os":

[GitHub] [beam] monicadsong commented on pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong commented on pull request #12851: URL: https://github.com/apache/beam/pull/12851#issuecomment-692856430 R: @tvalentyn This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] codecov[bot] commented on pull request #12779: [BEAM-10856] Support for NestedValueProvider for Python SDK

2020-09-15 Thread GitBox
codecov[bot] commented on pull request #12779: URL: https://github.com/apache/beam/pull/12779#issuecomment-692856347 # [Codecov](https://codecov.io/gh/apache/beam/pull/12779?src=pr=h1) Report > Merging [#12779](https://codecov.io/gh/apache/beam/pull/12779?src=pr=desc) into

[GitHub] [beam] monicadsong opened a new pull request #12851: [BEAM-10900] add ability for ApproximateUniqueCombineFn to handle numpy input data

2020-09-15 Thread GitBox
monicadsong opened a new pull request #12851: URL: https://github.com/apache/beam/pull/12851 If element for ApproximateUniqueCombineFn.add_input() is numpy type, convert to python type. Thank you for your contribution! Follow this checklist to help us

[GitHub] [beam] tvalentyn commented on pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn commented on pull request #12811: URL: https://github.com/apache/beam/pull/12811#issuecomment-692850902 Thanks, @samatix ! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] tvalentyn merged pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn merged pull request #12811: URL: https://github.com/apache/beam/pull/12811 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tvalentyn commented on a change in pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn commented on a change in pull request #12811: URL: https://github.com/apache/beam/pull/12811#discussion_r488824939 ## File path: sdks/python/apache_beam/runners/portability/stager_test.py ## @@ -448,7 +448,8 @@ def test_sdk_location_remote_source_file(self,

[GitHub] [beam] lukecwik commented on a change in pull request #12617: [BEAM-10670] Update Samza to be opt-out for SplittableDoFn powering the Read transform.

2020-09-15 Thread GitBox
lukecwik commented on a change in pull request #12617: URL: https://github.com/apache/beam/pull/12617#discussion_r488825043 ## File path: runners/samza/src/main/java/org/apache/beam/runners/samza/SamzaRunner.java ## @@ -106,6 +107,7 @@ public PortablePipelineResult

[GitHub] [beam] amaliujia commented on pull request #12843: [BEAM-10895] Support UNNEST an (possibly nested) array field of an struct column

2020-09-15 Thread GitBox
amaliujia commented on pull request #12843: URL: https://github.com/apache/beam/pull/12843#issuecomment-692848083 Thanks @robinyqiu will take a look soon. This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] kw2542 commented on a change in pull request #12617: [BEAM-10670] Update Samza to be opt-out for SplittableDoFn powering the Read transform.

2020-09-15 Thread GitBox
kw2542 commented on a change in pull request #12617: URL: https://github.com/apache/beam/pull/12617#discussion_r488820872 ## File path: runners/samza/src/main/java/org/apache/beam/runners/samza/SamzaRunner.java ## @@ -106,6 +107,7 @@ public PortablePipelineResult

[GitHub] [beam] tvalentyn commented on pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn commented on pull request #12811: URL: https://github.com/apache/beam/pull/12811#issuecomment-692845652 Actually found the previous green PostCommit result: https://ci-beam.apache.org/job/beam_PostCommit_Python36_PR/93/

[GitHub] [beam] tvalentyn commented on pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn commented on pull request #12811: URL: https://github.com/apache/beam/pull/12811#issuecomment-692844750 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [beam] tvalentyn commented on pull request #12811: [BEAM-10705] Fixes a bug when passing whl files in --sdk_location from https locations

2020-09-15 Thread GitBox
tvalentyn commented on pull request #12811: URL: https://github.com/apache/beam/pull/12811#issuecomment-692844451 > able to reproduce the last two issues (they seem to be random). Any ideas on what went wrong? Those seem to be flaky tests :(. Found this in the logs: ```

[GitHub] [beam] chamikaramj commented on a change in pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
chamikaramj commented on a change in pull request #12505: URL: https://github.com/apache/beam/pull/12505#discussion_r488818825 ## File path: website/www/site/content/en/documentation/runtime/environments.md ## @@ -116,8 +116,8 @@ By default, no licenses/notices are added to

[GitHub] [beam] tvalentyn commented on pull request #12637: [BEAM-10768] Don't assert the order in which elements are received.

2020-09-15 Thread GitBox
tvalentyn commented on pull request #12637: URL: https://github.com/apache/beam/pull/12637#issuecomment-692843176 seeing this flake in precommits, thanks for fixing it. This is an automated message from the Apache Git

[GitHub] [beam] tvalentyn edited a comment on pull request #12779: [BEAM-10856] Support for NestedValueProvider for Python SDK

2020-09-15 Thread GitBox
tvalentyn edited a comment on pull request #12779: URL: https://github.com/apache/beam/pull/12779#issuecomment-692833522 > A user has written a feature that they would find useful, and that will not change the experience for other users (if anything, it should improve it). The feature

[GitHub] [beam] TheNeuralBit commented on a change in pull request #12505: [BEAM-8106] Add version to java container image name

2020-09-15 Thread GitBox
TheNeuralBit commented on a change in pull request #12505: URL: https://github.com/apache/beam/pull/12505#discussion_r488809349 ## File path: sdks/java/container/Dockerfile-java11 ## @@ -16,6 +16,10 @@ # limitations under the License.

[GitHub] [beam] codecov[bot] edited a comment on pull request #12841: [BEAM-10894] Basic CSV reading and writing.

2020-09-15 Thread GitBox
codecov[bot] edited a comment on pull request #12841: URL: https://github.com/apache/beam/pull/12841#issuecomment-692395796 # [Codecov](https://codecov.io/gh/apache/beam/pull/12841?src=pr=h1) Report > Merging [#12841](https://codecov.io/gh/apache/beam/pull/12841?src=pr=desc) into

[GitHub] [beam] ajamato commented on a change in pull request #12822: [BEAM-10880] Log error counts to debug BigQuery streaming insert requ…

2020-09-15 Thread GitBox
ajamato commented on a change in pull request #12822: URL: https://github.com/apache/beam/pull/12822#discussion_r488813424 ## File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py ## @@ -538,6 +539,13 @@ def _insert_all_rows( try: response =

[GitHub] [beam] timclemons opened a new pull request #12850: [BEAM-10481] Ensure registration of the accumulator occurs.

2020-09-15 Thread GitBox
timclemons opened a new pull request #12850: URL: https://github.com/apache/beam/pull/12850 Move registration of accumulator so as to include those recovered from checkpoints. Prior to this, if the accumulator was recovered from a checkpoint it would not be registered, resulting in a

  1   2   >