[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-01 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 btw, can you update the title like `[SPARK-24695][SQL]...` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21679: SPARK-24695: To add support to return Calendar interval ...

2018-07-01 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21679 Since `CalendarInterval` is an internal class, I think users are not intended to use the class directly... --- - To unsubscribe,

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21648 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199396025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet] class

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21648 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92519/ Test PASSed. ---

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-07-01 Thread cxzl25
Github user cxzl25 commented on the issue: https://github.com/apache/spark/pull/21656 @maropu I have added a unit test. Can you trigger a test for this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21682 **[Test build #92519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92519/testReport)** for PR 21682 at commit [`e9d5625`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92518/ Test PASSed. ---

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21648 **[Test build #92518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92518/testReport)** for PR 21648 at commit [`1f40375`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21691: Branch 2.2

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21691 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21691: Branch 2.2

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21691 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21682 @maropu We accidentally dropped the pushdown of these data types when we refactored the file formats. You can check the change history and find the PR. ---

[GitHub] spark issue #21691: Branch 2.2

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21691 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21682 cc @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21682: [SPARK-24706][SQL] ByteType and ShortType support...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21682#discussion_r199390298 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -42,6 +42,14 @@ private[parquet] class

[GitHub] spark pull request #21691: Branch 2.2

2018-07-01 Thread xianbin
GitHub user xianbin opened a pull request: https://github.com/apache/spark/pull/21691 Branch 2.2 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch w

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199389588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala --- @@ -182,18 +182,20 @@ private[parquet

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199368095 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -47,16 +47,25 @@ import org.apache.

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199364935 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -71,9 +80,22 @@ private[parquet] cl

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199389252 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -71,9 +80,22 @@ private[parquet] cl

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199356283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -47,16 +47,25 @@ import org.apache.

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199354841 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -417,11 +417,12 @@ class ParquetFile

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r199365004 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -47,16 +47,25 @@ import org.apache.

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199388576 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21682 @HyukjinKwon btw, why we don't support pushdown for these types? Any historical reason? --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199387596 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-07-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21495 Sure. https://issues.apache.org/jira/browse/SPARK-24715 is created now. --- - To unsubscribe, e-mail: reviews-unsubscr...@s

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199386837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -129,7 +129,7 @@ case class CaseWhen(

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-07-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r199384074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3237,50 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-07-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r199384249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3236,13 +3237,50 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199386302 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/613/

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/613/ --- ---

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing the output data type of...

2018-07-01 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r199385955 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ConditionalExpressionSuite.scala --- @@ -113,6 +113,35 @@ class ConditionalE

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199385685 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +@pro

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/613/ --- -

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-07-01 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/20611 This PR is open from many days, we need this fix in our version code also, can someone please review and provide me suggestions if any. ---

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199385323 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #92520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92520/testReport)** for PR 21669 at commit [`000120f`](https://github.com/apache/spark/commit/00

[GitHub] spark pull request #21659: [SPARK-24530][PYTHON] Add a control to force Pyth...

2018-07-01 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21659#discussion_r199384118 --- Diff: python/docs/Makefile --- @@ -1,19 +1,35 @@ # Makefile for Sphinx documentation # +ifndef SPHINXPYTHON +SPHINXBUILD ?= sphin

[GitHub] spark pull request #21659: [SPARK-24530][PYTHON] Add a control to force Pyth...

2018-07-01 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/21659#discussion_r199384289 --- Diff: python/docs/Makefile --- @@ -1,19 +1,35 @@ # Makefile for Sphinx documentation # +ifndef SPHINXPYTHON +SPHINXBUILD ?= sphin

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21690 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21690 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21690 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21690: [SPARK-24713]AppMatser of spark streaming kafka OOM if t...

2018-07-01 Thread yuanboliu
Github user yuanboliu commented on the issue: https://github.com/apache/spark/pull/21690 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark pull request #21690: [SPARK-24713]AppMatser of spark streaming kafka O...

2018-07-01 Thread yuanboliu
GitHub user yuanboliu opened a pull request: https://github.com/apache/spark/pull/21690 [SPARK-24713]AppMatser of spark streaming kafka OOM if there are hund… We have hundreds of kafka topics need to be consumed in one application. The application master will throw OOM exception a

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199380206 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +@pro

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199379979 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199379810 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +@pro

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199379854 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +@pro

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199379417 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199378189 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +@pro

[GitHub] spark issue #21495: [SPARK-24418][Build] Upgrade Scala to 2.11.12 and 2.12.6

2018-07-01 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21495 @dongjoon-hyun can you please create a JIRA to track this issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21648 I roughly double checked most of primitive types like Boolean, Long and String work via Py4J, and some codes related with it in Py4J. Some types like `Option[...]` will probably just return the

[GitHub] spark issue #21535: [SPARK-23596][SQL][WIP] Test interpreted path on Dataset...

2018-07-01 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21535 If we only test interpreted path on selected test suites, e.g., `org.apache.spark.sql.catalyst.encoders.*`, I think we won't increase too much test time. I feel that we should not make it different i

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21682 **[Test build #92519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92519/testReport)** for PR 21682 at commit [`e9d5625`](https://github.com/apache/spark/commit/e9

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r199374528 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -20,6 +20,8 @@ package org.apache.sp

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/612/

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21682 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-01 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r199374458 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -551,6 +551,36 @@ object TypeCoercion {

[GitHub] spark issue #21682: [SPARK-24706][SQL] ByteType and ShortType support pushdo...

2018-07-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21682 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/611/

[GitHub] spark pull request #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21073#discussion_r199373569 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -17,6 +17,7 @@ package org.apache.spar

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21648 **[Test build #92518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92518/testReport)** for PR 21648 at commit [`1f40375`](https://github.com/apache/spark/commit/1f

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199373295 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark issue #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to manage ...

2018-07-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21648 LGTM for the changes. I'm just wondering if all config types are supported by the PY4J method call. Do we need some tests for that? --- -

[GitHub] spark pull request #21648: [SPARK-24665][PySpark] Use SQLConf in PySpark to ...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21648#discussion_r199370180 --- Diff: python/pyspark/sql/context.py --- @@ -93,6 +93,11 @@ def _ssql_ctx(self): """ return self._jsqlContext +

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21675 ping @mcteo mind addressing https://github.com/apache/spark/pull/21675#discussion_r199316192? I think it's good to address before merging it in. --- --

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/21688 I'll see if I get a chance to look at this this week, but with the holiday it may have to wait a week. --- - To unsubscribe, e

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92517/ Test PASSed. ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92517/testReport)** for PR 21688 at commit [`132fce9`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92517/testReport)** for PR 21688 at commit [`132fce9`](https://github.com/apache/spark/commit/13

[GitHub] spark issue #21689: [Minor][ML]Minor correction in the powerIterationSuite

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21689 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21689: [Minor][ML]Minor correction in the powerIterationSuite

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21689 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21689: [Minor][ML]Minor correction in the powerIterationSuite

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21689 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21689: Minor correction in the powerIterationSuite

2018-07-01 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/21689 Minor correction in the powerIterationSuite ## What changes were proposed in this pull request? Currently the power iteration clustering test in ml maps the results to the labels 0 and

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92516/testReport)** for PR 21688 at commit [`6a0622e`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92516/ Test FAILed. ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92516/testReport)** for PR 21688 at commit [`6a0622e`](https://github.com/apache/spark/commit/6a

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92515/testReport)** for PR 21688 at commit [`d79eb80`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92515/ Test FAILed. ---

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21688 **[Test build #92515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92515/testReport)** for PR 21688 at commit [`d79eb80`](https://github.com/apache/spark/commit/d7

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21688 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread pgandhi999
Github user pgandhi999 commented on the issue: https://github.com/apache/spark/pull/21688 @ajbozarth @vanzin @tgravescs Hello, After our last discussion in PR https://github.com/apache/spark/pull/19270, I have made some changes to the datatables, to fetch the data and perform sort,

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21688 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-07-01 Thread pgandhi999
GitHub user pgandhi999 opened a pull request: https://github.com/apache/spark/pull/21688 [SPARK-21809] : Change Stage Page to use datatables to support sorting columns and searching Support column sort, pagination and search for Stage Page using jQuery DataTable and REST API. Befor

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92513/ Test PASSed. ---

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21687 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing the output data type of CaseWh...

2018-07-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21687 **[Test build #92513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92513/testReport)** for PR 21687 at commit [`7104063`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #21671: [SPARK-24682] [SQL] from_json / to_json now handl...

2018-07-01 Thread patrickmcgloin
Github user patrickmcgloin commented on a diff in the pull request: https://github.com/apache/spark/pull/21671#discussion_r199356020 --- Diff: python/pyspark/sql/functions.py --- @@ -2163,9 +2163,9 @@ def json_tuple(col, *fields): @since(2.1) def from_json(col, schema, opt

  1   2   >