[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021171 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions( //

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021225 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions( //

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198021157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions( //

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92323/ Test FAILed. ---

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #92323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92323/testReport)** for PR 21102 at commit

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198019979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -174,3 +209,25 @@ object JDBCOptions {

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198019710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -150,6 +183,7 @@ class JDBCOptions( }

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198018682 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,32 @@ class JDBCOptions( //

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92322/ Test FAILed. ---

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #92322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92322/testReport)** for PR 21103 at commit

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r198018360 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -174,3 +209,25 @@ object JDBCOptions {

[GitHub] spark issue #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bug

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21641 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bug

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21641 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/473/

[GitHub] spark issue #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bug

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21641 **[Test build #92324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92324/testReport)** for PR 21641 at commit

[GitHub] spark pull request #21641: [SPARK-24658][SQL] Remove workaround for ANTLR bu...

2018-06-25 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/21641 [SPARK-24658][SQL] Remove workaround for ANTLR bug ## What changes were proposed in this pull request? Issue antlr/antlr4#781 has already been fixed, so the workaround of extracting the

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21590 @gatorsmile @maropu I have hopefully addressed the comments. Please take a look when you get a chance. --- - To

[GitHub] spark issue #21627: [SPARK-24484][MLLIB]Power Iteration Clustering is giving...

2018-06-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21627 Jenkins, test this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92321/ Test PASSed. ---

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21640 **[Test build #92321 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92321/testReport)** for PR 21640 at commit

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-25 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21628 @HyukjinKwon Thanks for your help! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21570: [SPARK-24564][TEST] Add test suite for RecordBinaryCompa...

2018-06-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21570 ping @JoshRosen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-06-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21061 cc @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92320/ Test PASSed. ---

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #92320 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92320/testReport)** for PR 20345 at commit

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN ...

2018-06-25 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r198004697 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerSource.scala --- @@ -0,0 +1,35 @@ +/* + *

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92318/ Test PASSed. ---

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21590 **[Test build #92318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92318/testReport)** for PR 21590 at commit

[GitHub] spark issue #18900: [SPARK-21687][SQL] Spark SQL should set createTime for H...

2018-06-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18900 we should also update `CatalogTablePartition.toLinkedHashMap`, following `CatalogTable.toLinkedHashMap` Otherwise LGTM ---

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-25 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r198000610 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3236,49 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92319/ Test PASSed. ---

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21611: [SPARK-24569][SQL] Aggregator with output type Option sh...

2018-06-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21611 does this affect normal dataset schema? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21590 **[Test build #92319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92319/testReport)** for PR 21590 at commit

[GitHub] spark pull request #21611: [SPARK-24569][SQL] Aggregator with output type Op...

2018-06-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21611#discussion_r197997909 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala --- @@ -43,12 +43,17 @@ import

[GitHub] spark issue #21639: [SPARK-24631][tests] Avoid cross-job pollution in TestUt...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21639 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92312/ Test PASSed. ---

[GitHub] spark issue #21639: [SPARK-24631][tests] Avoid cross-job pollution in TestUt...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21639 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21639: [SPARK-24631][tests] Avoid cross-job pollution in TestUt...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21639 **[Test build #92312 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92312/testReport)** for PR 21639 at commit

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yea, that's all true. I admit what you and @jerryshao did makes sense in a way. If we failed to replace the Hive fork to 2.3.x and keep the current fork, I got it that's the last resort that

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92317/ Test PASSed. ---

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21631 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPruning en...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21631 **[Test build #92317 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92317/testReport)** for PR 21631 at commit

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #92323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92323/testReport)** for PR 21102 at commit

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #92322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92322/testReport)** for PR 21103 at commit

[GitHub] spark pull request #21620: [SPARK-24636][SQL] Type coercion of arrays for ar...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21620 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21628: [SPARK-23776][DOC] Update instructions for runnin...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21628 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21495 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-25 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r197992151 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -660,6 +660,30 @@ class

[GitHub] spark issue #21620: [SPARK-24636][SQL] Type coercion of arrays for array_joi...

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21620 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92316/ Test FAILed. ---

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21495 LGTM too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21640 **[Test build #92316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92316/testReport)** for PR 21640 at commit

[GitHub] spark issue #21628: [SPARK-23776][DOC] Update instructions for running PySpa...

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21628 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-25 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21623 cc @gszadovszky @nandorKollar --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-06-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21495 OK, I'm going to merge it. We can fix the following issues if exists. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21617: [SPARK-24634][SS] Add a new metric regarding numb...

2018-06-25 Thread HeartSaVioR
Github user HeartSaVioR closed the pull request at: https://github.com/apache/spark/pull/21617 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21617: [SPARK-24634][SS] Add a new metric regarding number of r...

2018-06-25 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21617 Abandoning the patch. While I think the JIRA issue is still valid, looks like we should address watermark issue to have correct number of late events. Thanks for reviewing @jose-torres

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r197989956 --- Diff: python/pyspark/sql/column.py --- @@ -514,6 +514,17 @@ def isin(self, *cols): desc_nulls_first =

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21560 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21560 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92310/ Test PASSed. ---

[GitHub] spark issue #21560: [SPARK-24386][SS] coalesce(1) aggregates in continuous p...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21560 **[Test build #92310 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92310/testReport)** for PR 21560 at commit

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92315/ Test FAILed. ---

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #92315 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92315/testReport)** for PR 21482 at commit

[GitHub] spark pull request #21623: [SPARK-24638][SQL] StringStartsWith support push ...

2018-06-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21623#discussion_r197988392 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +272,29 @@ private[parquet]

[GitHub] spark pull request #21638: [SPARK-22357][CORE] SparkContext.binaryFiles igno...

2018-06-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21638#discussion_r197988075 --- Diff: core/src/main/scala/org/apache/spark/input/PortableDataStream.scala --- @@ -45,7 +45,8 @@ private[spark] abstract class StreamFileInputFormat[T]

[GitHub] spark pull request #21617: [SPARK-24634][SS] Add a new metric regarding numb...

2018-06-25 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21617#discussion_r197986093 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -48,12 +49,13 @@ class StateOperatorProgress private[sql](

[GitHub] spark issue #21611: [SPARK-24569][SQL] Aggregator with output type Option sh...

2018-06-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21611 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21560: [SPARK-24386][SS] coalesce(1) aggregates in conti...

2018-06-25 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21560#discussion_r197985638 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousCoalesceRDD.scala --- @@ -0,0 +1,108 @@ +/*

[GitHub] spark pull request #21617: [SPARK-24634][SS] Add a new metric regarding numb...

2018-06-25 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21617#discussion_r197984227 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -48,12 +49,13 @@ class StateOperatorProgress private[sql](

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21638 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92309/ Test PASSed. ---

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21638 **[Test build #92309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92309/testReport)** for PR 21638 at commit

[GitHub] spark pull request #21617: [SPARK-24634][SS] Add a new metric regarding numb...

2018-06-25 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/spark/pull/21617#discussion_r197981651 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -48,12 +49,13 @@ class StateOperatorProgress private[sql](

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21640 **[Test build #92321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92321/testReport)** for PR 21640 at commit

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21640: [SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21640 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/472/

[GitHub] spark pull request #21617: [SPARK-24634][SS] Add a new metric regarding numb...

2018-06-25 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/21617#discussion_r197980605 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -48,12 +49,13 @@ class StateOperatorProgress private[sql](

[GitHub] spark issue #21530: [spark][pyspark][sql] Correcting code for Documentation ...

2018-06-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21530 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21637: [SPARK-24324][PYTHON][FOLLOWUP] Grouped Map positional c...

2018-06-25 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21637 Thanks @HyukjinKwon , merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #92320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92320/testReport)** for PR 20345 at commit

[GitHub] spark pull request #21637: [SPARK-24324][PYTHON][FOLLOWUP] Grouped Map posit...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21637 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21616: [SPARK-24552][core] Use unique id instead of atte...

2018-06-25 Thread vanzin
Github user vanzin closed the pull request at: https://github.com/apache/spark/pull/21616 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21616: [SPARK-24552][core] Use unique id instead of attempt num...

2018-06-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21616 I should have checked first, but this doesn't merge to 2.1, and it doesn't look like 2.1 is affected anyway. There seems to be just one code path in 2.1 that hits this path, and it already uses a

[GitHub] spark issue #21616: [SPARK-24552][core] Use unique id instead of attempt num...

2018-06-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21616 Merged to 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21558: [SPARK-24552][SQL] Use task ID instead of attempt...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21558 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21606: [SPARK-24552][core][SQL] Use task ID instead of a...

2018-06-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21606 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21615: [SPARK-24552][core][sql] Use unique id instead of...

2018-06-25 Thread vanzin
Github user vanzin closed the pull request at: https://github.com/apache/spark/pull/21615 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21615: [SPARK-24552][core][sql] Use unique id instead of attemp...

2018-06-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21615 Merged to 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21606: [SPARK-24552][core][SQL] Use task ID instead of attempt ...

2018-06-25 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21606 Given the deafening silence, I'll merge the PRs myself, given there's a bunch of +1s from others. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN ...

2018-06-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r197975990 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -771,6 +784,7 @@ object ApplicationMaster

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN ...

2018-06-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r197976701 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerSource.scala --- @@ -0,0 +1,35 @@ +/* + *

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN ...

2018-06-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r197976137 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -67,6 +68,14 @@ private[spark] class

[GitHub] spark pull request #21631: [SPARK-24645][SQL] Skip parsing when csvColumnPru...

2018-06-25 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21631#discussion_r197973962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -183,11 +183,19 @@ class UnivocityParser(

[GitHub] spark pull request #21468: [SPARK-22151] : PYTHONPATH not picked up from the...

2018-06-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21468#discussion_r197971721 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -811,10 +811,18 @@ private[spark] class Client(

  1   2   3   4   5   >