[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20492 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20483: [SPARK-23312][SQL] add a config to turn off vectorized c...

2018-02-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20483 Thank you for adding this. I will look at the performance regression. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20274 The [Pre-build spark](https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc2-bin/) contains `kubernetes-model-2.0.0.jar`. but the below command will not contain this jar: ```

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20400 **[Test build #87061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87061/testReport)** for PR 20400 at commit

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20400 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87061/ Test PASSed. ---

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20400 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-02-04 Thread zjffdu
Github user zjffdu commented on the issue: https://github.com/apache/spark/pull/13599 ping @holdenk @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20274 An interesting discovery: if `SPARK_HOME/jars` missing `kubernetes-model-2.0.0.jar`, the silent mode is broken. --- - To

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20492 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87059/ Test PASSed. ---

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20492 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20492 **[Test build #87059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87059/testReport)** for PR 20492 at commit

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20400 **[Test build #87061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87061/testReport)** for PR 20400 at commit

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20400 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/584/

[GitHub] spark issue #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), unbounded...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20400 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20400: [SPARK-23084][PYTHON]Add unboundedPreceding(), un...

2018-02-04 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20400#discussion_r165893442 --- Diff: python/pyspark/sql/window.py --- @@ -208,20 +236,27 @@ def rangeBetween(self, start, end): and "5" means the five off after the

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87057/ Test PASSed. ---

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87057 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87057/testReport)** for PR 20487 at commit

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 Thanks @hvanhovell , the major changes is `SQLQueryTestSuite.scala`: ```scala private def listTestCases(): Seq[TestCase] = { -listFilesRecursively(new File(inputFilePath)).map {

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20505 **[Test build #87060 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87060/testReport)** for PR 20505 at commit

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87060/ Test FAILed. ---

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20505 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20502: [SPARK-23330][WebUI] Spark UI SQL executions page...

2018-02-04 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20502#discussion_r165890435 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -179,7 +179,7 @@ private[ui] abstract class

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20504 @wangyum this is a PR of 24K lines! How are we going to review this properly? Can you provide some guidelines (major changes etc)? ---

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20505 **[Test build #87060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87060/testReport)** for PR 20505 at commit

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20505 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20505: [SPARK-23251][SQL] Add checks for collection elem...

2018-02-04 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20505#discussion_r165889177 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLImplicits.scala --- @@ -165,11 +165,15 @@ abstract class SQLImplicits extends

[GitHub] spark issue #17422: [SPARK-20087] Attach accumulators / metrics to 'TaskKill...

2018-02-04 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/17422 All right then, I will take it over. Of course the credit should go to @noodle-fb. We can discuss whether this behaviour is desirable or not in the JIRA or the new PR. ---

[GitHub] spark issue #20449: [SPARK-23040][CORE]: Returns interruptible iterator for ...

2018-02-04 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/20449 ping @cloud-fan and @jiangxb1987. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165884501 --- Diff: pom.xml --- @@ -185,6 +185,10 @@ 2.8 1.8 1.0.0 +

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165883566 --- Diff: python/pyspark/sql/tests.py --- @@ -2186,7 +2186,7 @@ def test_replace(self): # replace with subset specified with one column

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165884365 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1587,6 +1600,52 @@ def replace(self, to_replace, value=None, subset=None): |null| null|null|

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165883051 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1557,6 +1557,9 @@ def replace(self, to_replace, value=None, subset=None): For example, if

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165884027 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1923,6 +1923,9 @@ def toPandas(self): 02 Alice 15Bob

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165883842 --- Diff: pom.xml --- @@ -185,6 +185,10 @@ 2.8 1.8 1.0.0 +

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165883676 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1923,6 +1923,9 @@ def toPandas(self): 02 Alice 15Bob

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165879157 --- Diff: pom.xml --- @@ -185,6 +185,10 @@ 2.8 1.8 1.0.0 +

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87054/ Test PASSed. ---

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20504 **[Test build #87054 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87054/testReport)** for PR 20504 at commit

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18555 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87053/ Test PASSed. ---

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #87053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87053/testReport)** for PR 18555 at commit

[GitHub] spark pull request #16099: [SPARK-18665][SQL] set statement state to "ERROR"...

2018-02-04 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/16099#discussion_r165878592 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperation.scala --- @@ -241,6 +241,8 @@

[GitHub] spark issue #16099: [SPARK-18665][SQL] set statement state to "ERROR" after ...

2018-02-04 Thread cenyuhai
Github user cenyuhai commented on the issue: https://github.com/apache/spark/pull/16099 @gatorsmile two years passed... I don't know what to say. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87058/ Test PASSed. ---

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #87058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87058/testReport)** for PR 20164 at commit

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20473 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16099: [SPARK-18665][SQL] set statement state to "ERROR"...

2018-02-04 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/16099#discussion_r165877094 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperation.scala --- @@ -241,6 +241,8 @@

[GitHub] spark pull request #16099: [SPARK-18665][SQL] set statement state to "ERROR"...

2018-02-04 Thread BruceXu1991
Github user BruceXu1991 commented on a diff in the pull request: https://github.com/apache/spark/pull/16099#discussion_r165876866 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperation.scala --- @@ -241,6 +241,8 @@

[GitHub] spark pull request #20502: [SPARK-23330][WebUI] Spark UI SQL executions page...

2018-02-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20502#discussion_r165876103 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -179,7 +179,7 @@ private[ui] abstract class

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 Will merge this one in few days if there's no more comments. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20492 **[Test build #87059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87059/testReport)** for PR 20492 at commit

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20492 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/583/

[GitHub] spark issue #20492: [SPARK-23310][CORE] Turn off read ahead input stream for...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20492 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20492: [SPARK-23310][CORE] Turn off read ahead input str...

2018-02-04 Thread sitalkedia
Github user sitalkedia commented on a diff in the pull request: https://github.com/apache/spark/pull/20492#discussion_r165874317 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java --- @@ -77,7 +77,7 @@ public

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/582/

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20164 **[Test build #87058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87058/testReport)** for PR 20164 at commit

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20473 Will merge this one if there's no more comments in few days. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87057/testReport)** for PR 20487 at commit

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20164 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/581/

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165873671 --- Diff: python/setup.py --- @@ -100,6 +100,11 @@ def _supports_symlinks(): file=sys.stderr) exit(-1) +# If

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165873582 --- Diff: python/pyspark/sql/tests.py --- @@ -2794,7 +2792,6 @@ def count_bucketed_cols(names, table="pyspark_bucket"): def

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165873632 --- Diff: pom.xml --- @@ -185,6 +185,10 @@ 2.8 1.8 1.0.0 +

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-04 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/20164 @WeichenXu123 Yes, my concern is that it is confusing if the transform failure is caused by column conflict by a ‘invisible’ column. @srowen Agree that it is not perfect if we

[GitHub] spark pull request #20502: [SPARK-23330][WebUI] Spark UI SQL executions page...

2018-02-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20502#discussion_r165872846 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/AllExecutionsPage.scala --- @@ -179,7 +179,7 @@ private[ui] abstract class

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-04 Thread lucio-yz
Github user lucio-yz commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r165872418 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +1002,22 @@ private[spark] object RandomForest extends

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87056/ Test PASSed. ---

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20473 **[Test build #87056 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87056/testReport)** for PR 20473 at commit

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87055/ Test FAILed. ---

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20473 **[Test build #87055 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87055/testReport)** for PR 20473 at commit

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20473 **[Test build #87056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87056/testReport)** for PR 20473 at commit

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/580/

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20473#discussion_r165869377 --- Diff: python/run-tests.py --- @@ -151,6 +152,67 @@ def parse_opts(): return opts +def _check_dependencies(python_exec,

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/579/

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20473 **[Test build #87055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87055/testReport)** for PR 20473 at commit

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/578/

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20504 **[Test build #87054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87054/testReport)** for PR 20504 at commit

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87052/ Test FAILed. ---

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20504 **[Test build #87052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87052/testReport)** for PR 20504 at commit

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support a...

2018-02-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20504 After SPARK-21646, `hive/binaryComparison.sql.out`, `hive/decimalPrecision.sql.out` and `hive/promoteStrings.sql.out` seems like this:

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18555 **[Test build #87053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87053/testReport)** for PR 18555 at commit

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165865310 --- Diff: python/pyspark/sql/utils.py --- @@ -115,18 +115,30 @@ def toJArray(gateway, jtype, arr): def require_minimum_pandas_version():

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-04 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165865284 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1923,6 +1923,9 @@ def toPandas(self): 02 Alice 15Bob

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20505 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20505: [SPARK-23251][SQL] Add checks for collection element Enc...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20505 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20505: [SPARK-23251][SQL] Add checks for collection elem...

2018-02-04 Thread michalsenkyr
GitHub user michalsenkyr opened a pull request: https://github.com/apache/spark/pull/20505 [SPARK-23251][SQL] Add checks for collection element Encoders Implicit methods of `SQLImplicits` providing Encoders for collections did not check for Encoders for their elements. This

[GitHub] spark pull request #19054: [SPARK-18067] Avoid shuffling child if join keys ...

2018-02-04 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/19054#discussion_r165861581 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala --- @@ -220,45 +220,99 @@ case class

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support t...

2018-02-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20504 **[Test build #87052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87052/testReport)** for PR 20504 at commit

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support t...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/577/

[GitHub] spark issue #20504: [SPARK-23332][SQL] Update SQLQueryTestSuite to support t...

2018-02-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20504 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   >