[GitHub] [beam] chamikaramj commented on pull request #12168: Move Kafka directly to io submodule.

2020-07-02 Thread GitBox
chamikaramj commented on pull request #12168: URL: https://github.com/apache/beam/pull/12168#issuecomment-653358890 LGTM. Thanks Robert. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] codeBehindMe commented on pull request #12170: [BEAM-10166] Improve execution timer errors with more concise error messages from DoFns

2020-07-02 Thread GitBox
codeBehindMe commented on pull request #12170: URL: https://github.com/apache/beam/pull/12170#issuecomment-653358006 R: @lostluck This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] purbanow commented on pull request #12149: [BEAM-9897] Add cross-language support to SnowflakeIO.Read

2020-07-02 Thread GitBox
purbanow commented on pull request #12149: URL: https://github.com/apache/beam/pull/12149#issuecomment-653357937 @chamikaramj sounds good to me :) Thanks This is an automated message from the Apache Git Service. To respond

[GitHub] [beam] codeBehindMe opened a new pull request #12170: [BEAM-10166] Improve execution timer errors with more concise error messages from DoFns

2020-07-02 Thread GitBox
codeBehindMe opened a new pull request #12170: URL: https://github.com/apache/beam/pull/12170 The Go SDK uses errors returned by DoFns to signal failures to process bundles, and terminate bundle processing. However, if the preceding DoFn uses emitters, rather than error returns, the code

[GitHub] [beam] pabloem merged pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem merged pull request #12158: URL: https://github.com/apache/beam/pull/12158 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [beam] pabloem commented on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem commented on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653322920 Thanks @ettirapp - this is a great start! : ) This is an automated message from the Apache Git Service. To

[GitHub] [beam] pabloem removed a comment on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem removed a comment on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653322676 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] kennknowles commented on pull request #12162: [BEAM-10402] Enable checker framework, disabled everywhere since every module has errors. Eliminate some errors (but no complete modules)

2020-07-02 Thread GitBox
kennknowles commented on pull request #12162: URL: https://github.com/apache/beam/pull/12162#issuecomment-653295836 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] kennknowles edited a comment on pull request #12162: [BEAM-10402] Enable checker framework, disabled everywhere since every module has errors. Eliminate some errors (but no complete mo

2020-07-02 Thread GitBox
kennknowles edited a comment on pull request #12162: URL: https://github.com/apache/beam/pull/12162#issuecomment-653252161 @TheNeuralBit I finally got to it. This is an automated message from the Apache Git Service. To

[GitHub] [beam] amaliujia closed pull request #12068: [Do Not Merge][Proof of Concept] Support SQL UDTVF

2020-07-02 Thread GitBox
amaliujia closed pull request #12068: URL: https://github.com/apache/beam/pull/12068 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] amaliujia commented on pull request #12169: [SQL][ZetaSQL] Support Pure SQL user-defined table-valued function.

2020-07-02 Thread GitBox
amaliujia commented on pull request #12169: URL: https://github.com/apache/beam/pull/12169#issuecomment-653274247 R: @ibzib This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [beam] amaliujia opened a new pull request #12169: [SQL][ZetaSQL] Support Pure SQL user-defined table-valued function.

2020-07-02 Thread GitBox
amaliujia opened a new pull request #12169: URL: https://github.com/apache/beam/pull/12169 This PR adds the support of ``` "CREATE TABLE FUNCTION CustomerRange(MinID INT64, MaxID INT64)\n" " AS\n" "SELECT *\n" "FROM KeyValue\n" "WHERE key >= MinId AND key <=

[GitHub] [beam] pabloem commented on pull request #12084: [BEAM-10317] Python - Update BigQueryIO to tag BigQuery Jobs with the Dataflow Job ID

2020-07-02 Thread GitBox
pabloem commented on pull request #12084: URL: https://github.com/apache/beam/pull/12084#issuecomment-653268082 Run Python 2 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] robertwb commented on a change in pull request #12086: [BEAM-10322] allow only single assignment to producing stages by pcol…

2020-07-02 Thread GitBox
robertwb commented on a change in pull request #12086: URL: https://github.com/apache/beam/pull/12086#discussion_r449317183 ## File path: sdks/python/apache_beam/runners/portability/fn_api_runner/execution.py ## @@ -375,6 +375,8 @@ def get_all_side_inputs(): for o in

[GitHub] [beam] pabloem commented on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem commented on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653267257 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] robertwb commented on pull request #12086: [BEAM-10322] allow only single assignment to producing stages by pcol…

2020-07-02 Thread GitBox
robertwb commented on pull request #12086: URL: https://github.com/apache/beam/pull/12086#issuecomment-653267148 Could you clarify what the bug that you're trying to fix here is? This is an automated message from the Apache

[GitHub] [beam] tvalentyn merged pull request #11709: Throw an error when main output has the same tag as side outputs

2020-07-02 Thread GitBox
tvalentyn merged pull request #11709: URL: https://github.com/apache/beam/pull/11709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] tvalentyn commented on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn commented on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653260583 Run Python 3.8 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] rjbordon commented on pull request #12148: Minor fix on documentation to remove a duplicated word 'to'

2020-07-02 Thread GitBox
rjbordon commented on pull request #12148: URL: https://github.com/apache/beam/pull/12148#issuecomment-653255800 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] tvalentyn removed a comment on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn removed a comment on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653131533 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] tvalentyn commented on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn commented on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653252897 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] kennknowles commented on pull request #12162: [BEAM-10402] Enable checker framework, disabled everywhere since every module has errors. Eliminate some errors (but no complete modules)

2020-07-02 Thread GitBox
kennknowles commented on pull request #12162: URL: https://github.com/apache/beam/pull/12162#issuecomment-653252161 @TheNeuralBit I finally got to it in my free time. This is an automated message from the Apache Git Service.

[GitHub] [beam] tweise commented on a change in pull request #12133: [BEAM-10385] Integrate SQL expansion into Flink job server

2020-07-02 Thread GitBox
tweise commented on a change in pull request #12133: URL: https://github.com/apache/beam/pull/12133#discussion_r449298751 ## File path: sdks/python/apache_beam/transforms/sql.py ## @@ -67,10 +66,9 @@ class SqlTransform(ExternalTransform): """ URN =

[GitHub] [beam] robertwb edited a comment on pull request #12168: Move Kafka directly to io submodule.

2020-07-02 Thread GitBox
robertwb edited a comment on pull request #12168: URL: https://github.com/apache/beam/pull/12168#issuecomment-653228545 (The high line count is just due to the file move; the content required no editing. This is visible by looking at the commits individually.)

[GitHub] [beam] robertwb commented on pull request #12168: Move Kafka directly to io submodule.

2020-07-02 Thread GitBox
robertwb commented on pull request #12168: URL: https://github.com/apache/beam/pull/12168#issuecomment-653228545 (The high line count is just due to the file move; the content required no editing.) This is an automated

[GitHub] [beam] robertwb commented on pull request #12168: Move Kafka directly to io submodule.

2020-07-02 Thread GitBox
robertwb commented on pull request #12168: URL: https://github.com/apache/beam/pull/12168#issuecomment-653228312 R: @chamikaramj CC: @tvalentyn This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] robertwb opened a new pull request #12168: Move Kafka directly to io submodule.

2020-07-02 Thread GitBox
robertwb opened a new pull request #12168: URL: https://github.com/apache/beam/pull/12168 It'd be good to get this into the next release before people start using it (and we add a lot of documentation). Stubs left in the old location for backwards compatibility.

[GitHub] [beam] kennknowles commented on a change in pull request #12162: [BEAM-10402] Enable checker framework, disabled everywhere since every module has errors. Eliminate some errors (but no comple

2020-07-02 Thread GitBox
kennknowles commented on a change in pull request #12162: URL: https://github.com/apache/beam/pull/12162#discussion_r449273320 ## File path: sdks/java/core/src/main/java/org/apache/beam/sdk/metrics/MetricFiltering.java ## @@ -19,6 +19,7 @@ import java.util.Set; import

[GitHub] [beam] pabloem commented on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem commented on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653225448 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [beam] robertwb commented on a change in pull request #12133: [BEAM-10385] Integrate SQL expansion into Flink job server

2020-07-02 Thread GitBox
robertwb commented on a change in pull request #12133: URL: https://github.com/apache/beam/pull/12133#discussion_r449266867 ## File path: sdks/python/apache_beam/transforms/sql.py ## @@ -67,10 +66,9 @@ class SqlTransform(ExternalTransform): """ URN =

[GitHub] [beam] tvalentyn commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
tvalentyn commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r449254651 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,234 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] tvalentyn commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
tvalentyn commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r449254651 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,234 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] tvalentyn commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
tvalentyn commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r449178144 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,234 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] jaketf commented on pull request #11959: refactor HCLS IO ITs to support stores in other projects

2020-07-02 Thread GitBox
jaketf commented on pull request #11959: URL: https://github.com/apache/beam/pull/11959#issuecomment-653207804 ultimately we need to make sure service-[project_number]@gcp-sa-healthcare.iam.gserviceaccount.com has Storage Object Viewer on the buckets we are importing from.

[GitHub] [beam] jaketf edited a comment on pull request #11959: refactor HCLS IO ITs to support stores in other projects

2020-07-02 Thread GitBox
jaketf edited a comment on pull request #11959: URL: https://github.com/apache/beam/pull/11959#issuecomment-653188130 I was able to reproduce this failure. It occurs when using the import method and [falling back to using the temp

[GitHub] [beam] jaketf commented on pull request #11959: refactor HCLS IO ITs to support stores in other projects

2020-07-02 Thread GitBox
jaketf commented on pull request #11959: URL: https://github.com/apache/beam/pull/11959#issuecomment-653188130 I was able to reproduce this failure. It occurs when using the import method and [falling back to using the temp

[GitHub] [beam] pabloem commented on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem commented on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653182742 (that's a code phrase to run the tests) This is an automated message from the Apache Git Service. To respond to

[GitHub] [beam] pabloem commented on pull request #12158: [BEAM-10378] Adding Azure IO module

2020-07-02 Thread GitBox
pabloem commented on pull request #12158: URL: https://github.com/apache/beam/pull/12158#issuecomment-653182672 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] jaketf commented on pull request #11959: refactor HCLS IO ITs to support stores in other projects

2020-07-02 Thread GitBox
jaketf commented on pull request #11959: URL: https://github.com/apache/beam/pull/11959#issuecomment-653182234 My focus has been shifted to other customers. Taking a look at why this test was failing now, will update this thread with a commit to fix or findings.

[GitHub] [beam] amaliujia merged pull request #12052: Update walkthrough.md

2020-07-02 Thread GitBox
amaliujia merged pull request #12052: URL: https://github.com/apache/beam/pull/12052 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [beam] lostluck commented on pull request #12061: [BEAM-4634] Add options to allow BigQuery StandardSQL queries.

2020-07-02 Thread GitBox
lostluck commented on pull request #12061: URL: https://github.com/apache/beam/pull/12061#issuecomment-653175129 I'm fine with it either way for the reasons Robert (Bradshaw) describes. Given that standard is the default for the actual go bigquery API, defaulting to legacy would be

[GitHub] [beam] pabloem commented on pull request #12084: [BEAM-10317] Python - Update BigQueryIO to tag BigQuery Jobs with the Dataflow Job ID

2020-07-02 Thread GitBox
pabloem commented on pull request #12084: URL: https://github.com/apache/beam/pull/12084#issuecomment-653168951 Run Python 3.5 PostCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] robertwb commented on pull request #12061: [BEAM-4634] Add options to allow BigQuery StandardSQL queries.

2020-07-02 Thread GitBox
robertwb commented on pull request #12061: URL: https://github.com/apache/beam/pull/12061#issuecomment-653167957 (1) is not (yet) a concern for Go, but good (if unfortunate) point about (2). Something that could mitigate this is to make it required. On Thu, Jul 2, 2020 at 11:25 AM

[GitHub] [beam] bamnet commented on pull request #12061: [BEAM-4634] Add options to allow BigQuery StandardSQL queries.

2020-07-02 Thread GitBox
bamnet commented on pull request #12061: URL: https://github.com/apache/beam/pull/12061#issuecomment-653157515 I'm very new to Beam so don't have strong thoughts here, but two things jump out: 1) It's a breaking change. 2) LegacySQL appears to be the default for Python and Java.

[GitHub] [beam] robertwb commented on pull request #12061: [BEAM-4634] Add options to allow BigQuery StandardSQL queries.

2020-07-02 Thread GitBox
robertwb commented on pull request #12061: URL: https://github.com/apache/beam/pull/12061#issuecomment-653150195 Given that Go is still experimental, any thoughts on switching the default to non-legacy? On Tue, Jun 23, 2020 at 5:31 PM Robert Burke wrote: >

[GitHub] [beam] lgajowy commented on a change in pull request #12117: [BEAM-10343] Add dispositions for SnowflakeIO.write

2020-07-02 Thread GitBox
lgajowy commented on a change in pull request #12117: URL: https://github.com/apache/beam/pull/12117#discussion_r449152583 ## File path: sdks/java/io/snowflake/src/main/java/org/apache/beam/sdk/io/snowflake/SnowflakeIO.java ## @@ -155,7 +157,7 @@ * items.apply( *

[GitHub] [beam] robertwb commented on pull request #12075: Remove usage of jdk-internal NotNull annotation

2020-07-02 Thread GitBox
robertwb commented on pull request #12075: URL: https://github.com/apache/beam/pull/12075#issuecomment-653147904 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] aromanenko-dev merged pull request #12131: [BEAM-10335] add STS assume role to AwsModule

2020-07-02 Thread GitBox
aromanenko-dev merged pull request #12131: URL: https://github.com/apache/beam/pull/12131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [beam] tvalentyn edited a comment on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn edited a comment on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653132163 @kamilwu I anticipate that some of the postcommit tests may be failing, possibly due to outdated IO libraries. If so, you could help identify, isolate and investigate

[GitHub] [beam] tvalentyn commented on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn commented on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653132163 @kamilwu I anticipate that some of the postcommit tests may be failing, possibly due to outdated IO libraries. If so, you could help identify, isolate and investigate

[GitHub] [beam] tvalentyn commented on pull request #11788: [BEAM-9785] Add Python 3.8 postcommit tests

2020-07-02 Thread GitBox
tvalentyn commented on pull request #11788: URL: https://github.com/apache/beam/pull/11788#issuecomment-653131533 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] tvalentyn commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
tvalentyn commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r449165261 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,231 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] tvalentyn commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
tvalentyn commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r449160887 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -170,26 +174,50 @@ if [[ $confirmation = "y" ]]; then git clone

[GitHub] [beam] kamilwu commented on pull request #12167: [BEAM-9889] Populate local instance of InfluxDB with data

2020-07-02 Thread GitBox
kamilwu commented on pull request #12167: URL: https://github.com/apache/beam/pull/12167#issuecomment-653117466 R: @Ardagan Will you find a moment to make a code review? cc: @iemejia This is an automated message from

[GitHub] [beam] kamilwu opened a new pull request #12167: [BEAM-9889] Populate local instance of InfluxDB with data

2020-07-02 Thread GitBox
kamilwu opened a new pull request #12167: URL: https://github.com/apache/beam/pull/12167 The goal of these changes is to make the process of creating and modifying Grafana dashboards easier by populating InfluxDB database with real data. Two workflows have been implemented: * a

[GitHub] [beam] lgajowy commented on pull request #12117: [BEAM-10343] Add dispositions for SnowflakeIO.write

2020-07-02 Thread GitBox
lgajowy commented on pull request #12117: URL: https://github.com/apache/beam/pull/12117#issuecomment-653116162 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] kamilwu commented on a change in pull request #12065: [BEAM-10306] Add latency measurements to Python Flink load test

2020-07-02 Thread GitBox
kamilwu commented on a change in pull request #12065: URL: https://github.com/apache/beam/pull/12065#discussion_r449141423 ## File path: sdks/python/apache_beam/testing/load_tests/pardo_test.py ## @@ -147,10 +151,18 @@ def process(self, element, state=state_param):

[GitHub] [beam] aromanenko-dev commented on a change in pull request #12131: [BEAM-10335] add STS assume role to AwsModule

2020-07-02 Thread GitBox
aromanenko-dev commented on a change in pull request #12131: URL: https://github.com/apache/beam/pull/12131#discussion_r449112432 ## File path: sdks/java/io/amazon-web-services/build.gradle ## @@ -18,8 +18,14 @@ import groovy.json.JsonOutput * limitations under the License.

[GitHub] [beam] mxm commented on pull request #12137: [BEAM-10392] Attempt to fix/enable detection of rabbitmq:test stuckness.

2020-07-02 Thread GitBox
mxm commented on pull request #12137: URL: https://github.com/apache/beam/pull/12137#issuecomment-653102269 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [beam] lazylynx commented on pull request #12115: [BEAM-7672] dynamically setup acceptable wheel specs according to installed python version

2020-07-02 Thread GitBox
lazylynx commented on pull request #12115: URL: https://github.com/apache/beam/pull/12115#issuecomment-653101559 @tvalentyn Updated. PTAL when tests are passed. This is an automated message from the Apache Git Service. To

[GitHub] [beam] lazylynx commented on a change in pull request #12115: [BEAM-7672] dynamically setup acceptable wheel specs according to installed python version

2020-07-02 Thread GitBox
lazylynx commented on a change in pull request #12115: URL: https://github.com/apache/beam/pull/12115#discussion_r449128227 ## File path: sdks/python/container/boot.go ## @@ -127,6 +129,10 @@ func main() { // Guard from concurrent artifact retrieval and installation,

[GitHub] [beam] lazylynx commented on a change in pull request #12115: [BEAM-7672] dynamically setup acceptable wheel specs according to installed python version

2020-07-02 Thread GitBox
lazylynx commented on a change in pull request #12115: URL: https://github.com/apache/beam/pull/12115#discussion_r449126596 ## File path: sdks/python/container/boot.go ## @@ -170,6 +176,28 @@ func main() { log.Fatalf("Python exited: %v", execx.Execute("python",

[GitHub] [beam] mxm merged pull request #12148: Minor fix on documentation to remove a duplicated word 'to'

2020-07-02 Thread GitBox
mxm merged pull request #12148: URL: https://github.com/apache/beam/pull/12148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [beam] aromanenko-dev commented on a change in pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on a change in pull request #12063: URL: https://github.com/apache/beam/pull/12063#discussion_r449044147 ## File path: runners/spark/src/main/java/org/apache/beam/runners/spark/metrics/MetricsAccumulator.java ## @@ -58,13 +57,13 @@ public static void

[GitHub] [beam] aromanenko-dev commented on a change in pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on a change in pull request #12063: URL: https://github.com/apache/beam/pull/12063#discussion_r449044147 ## File path: runners/spark/src/main/java/org/apache/beam/runners/spark/metrics/MetricsAccumulator.java ## @@ -58,13 +57,13 @@ public static void

[GitHub] [beam] aromanenko-dev commented on a change in pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on a change in pull request #12063: URL: https://github.com/apache/beam/pull/12063#discussion_r449044147 ## File path: runners/spark/src/main/java/org/apache/beam/runners/spark/metrics/MetricsAccumulator.java ## @@ -58,13 +57,13 @@ public static void

[GitHub] [beam] aromanenko-dev commented on a change in pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on a change in pull request #12063: URL: https://github.com/apache/beam/pull/12063#discussion_r449044147 ## File path: runners/spark/src/main/java/org/apache/beam/runners/spark/metrics/MetricsAccumulator.java ## @@ -58,13 +57,13 @@ public static void

[GitHub] [beam] aromanenko-dev commented on pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on pull request #12063: URL: https://github.com/apache/beam/pull/12063#issuecomment-653015794 Run Spark ValidatesRunner This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] aromanenko-dev commented on pull request #12063: [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI

2020-07-02 Thread GitBox
aromanenko-dev commented on pull request #12063: URL: https://github.com/apache/beam/pull/12063#issuecomment-653014984 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r448983103 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,231 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] piotr-szuberski commented on pull request #12145: [BEAM-10136] [BEAM-10135] Add JdbcIO for cross-language with python wrapper

2020-07-02 Thread GitBox
piotr-szuberski commented on pull request #12145: URL: https://github.com/apache/beam/pull/12145#issuecomment-652985646 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r448974933 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,231 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r448974075 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,231 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r448971722 ## File path: release/src/main/scripts/download_github_actions_artifacts.py ## @@ -0,0 +1,231 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [beam] TobKed opened a new pull request #12166: [BEAM-10404] Cancel queued/running GitHub Action builds on second push to PR

2020-07-02 Thread GitBox
TobKed opened a new pull request #12166: URL: https://github.com/apache/beam/pull/12166 We need to Cancel builds to PR too. similar change in Apache Airflow https://github.com/apache/airflow/pull/9513 Thank you for your contribution! Follow

[GitHub] [beam] mxm commented on pull request #12065: [BEAM-10306] Add latency measurements to Python Flink load test

2020-07-02 Thread GitBox
mxm commented on pull request #12065: URL: https://github.com/apache/beam/pull/12065#issuecomment-652972556 Run PythonFormatter PreCommit This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [beam] mxm commented on pull request #12065: [BEAM-10306] Add latency measurements to Python Flink load test

2020-07-02 Thread GitBox
mxm commented on pull request #12065: URL: https://github.com/apache/beam/pull/12065#issuecomment-652972679 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] kamilwu commented on pull request #12146: [BEAM-10383] Update Snowflake JDBC dependency

2020-07-02 Thread GitBox
kamilwu commented on pull request #12146: URL: https://github.com/apache/beam/pull/12146#issuecomment-652950416 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] TobKed commented on a change in pull request #12150: [BEAM-10398] Use GitHub Actions in wheels release process for Python

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12150: URL: https://github.com/apache/beam/pull/12150#discussion_r448928301 ## File path: release/src/main/scripts/build_release_candidate.sh ## @@ -170,26 +174,50 @@ if [[ $confirmation = "y" ]]; then git clone ${GIT_REPO_URL}

[GitHub] [beam] TobKed commented on a change in pull request #12049: [BEAM-10399] Periodic clear of GCS wheels staging bucket

2020-07-02 Thread GitBox
TobKed commented on a change in pull request #12049: URL: https://github.com/apache/beam/pull/12049#discussion_r448909493 ## File path: .github/workflows/clear_staging_bucket.yml ## @@ -0,0 +1,67 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [beam] lgajowy commented on pull request #12117: [BEAM-10343] Add dispositions for SnowflakeIO.write

2020-07-02 Thread GitBox
lgajowy commented on pull request #12117: URL: https://github.com/apache/beam/pull/12117#issuecomment-652925396 @purbanow I will try to take a look at that today or tomorrow. If I won't leave a review by then, please try to ping somebody else (that will mean that I did not have enough

[GitHub] [beam] ihji commented on pull request #12165: fix formatting for pull request post-commit status table

2020-07-02 Thread GitBox
ihji commented on pull request #12165: URL: https://github.com/apache/beam/pull/12165#issuecomment-652921247 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [beam] ihji opened a new pull request #12165: fix formatting for pull request post-commit status table

2020-07-02 Thread GitBox
ihji opened a new pull request #12165: URL: https://github.com/apache/beam/pull/12165 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] kamilwu commented on pull request #12146: [BEAM-10383] Update Snowflake JDBC dependency

2020-07-02 Thread GitBox
kamilwu commented on pull request #12146: URL: https://github.com/apache/beam/pull/12146#issuecomment-652890299 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [beam] mxm commented on a change in pull request #12133: [BEAM-10385] Integrate SQL expansion into Flink job server

2020-07-02 Thread GitBox
mxm commented on a change in pull request #12133: URL: https://github.com/apache/beam/pull/12133#discussion_r448859883 ## File path: sdks/python/apache_beam/transforms/sql.py ## @@ -67,10 +66,9 @@ class SqlTransform(ExternalTransform): """ URN =

[GitHub] [beam] ihji opened a new pull request #12164: [BEAM-10397] add missing environment in windowing strategy for Dataflow

2020-07-02 Thread GitBox
ihji opened a new pull request #12164: URL: https://github.com/apache/beam/pull/12164 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose

[GitHub] [beam] kkucharc commented on pull request #12151: [BEAM-9896] Add streaming for SnowflakeIO.Write to Java SDK

2020-07-02 Thread GitBox
kkucharc commented on pull request #12151: URL: https://github.com/apache/beam/pull/12151#issuecomment-652863614 @pabloem would you find time to take a look? Or maybe you can suggest someone? (Java PreCommit is failing is not connected with those changes.)

[GitHub] [beam] je-ik commented on pull request #12155: [BEAM-10400] Fix race condition in DirectRunner watermark update

2020-07-02 Thread GitBox
je-ik commented on pull request #12155: URL: https://github.com/apache/beam/pull/12155#issuecomment-652846468 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [beam] stale[bot] commented on pull request #11589: WIP: [BEAM-9623] Support for SQL TableProviders in Python SqlTransform

2020-07-02 Thread GitBox
stale[bot] commented on pull request #11589: URL: https://github.com/apache/beam/pull/11589#issuecomment-652832568 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

[GitHub] [beam] robinyqiu opened a new pull request #12163: [BEAM-9712] Refactor ZetaSQL planner code and support setting default timezone

2020-07-02 Thread GitBox
robinyqiu opened a new pull request #12163: URL: https://github.com/apache/beam/pull/12163 Enable setting default timezone option on ZetaSQL analyzer. Default timezone is used both at - query analysis time (e.g. `CAST` literal to `TIMESTAMP`), and - expression execution time (e.g.