[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20487 Ah, yup. There are few tests for old Pandas which were tested only when Pandas version was lower, and I rewrote them to be tested when both Pandas version is lower and missing. Let me clarify

[GitHub] spark issue #20515: [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use datetime.da...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20515 **[Test build #87094 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87094/testReport)** for PR 20515 at commit

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87091/ Test FAILed. ---

[GitHub] spark issue #20515: [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use datetime.da...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/615/

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20515: [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use datetime.da...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20515 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20495 **[Test build #87091 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87091/testReport)** for PR 20495 at commit

[GitHub] spark issue #20515: [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use datetime.da...

2018-02-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20515 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20515: [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use date...

2018-02-05 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20515 [SPARK-23290][SQL][PYTHON][BACKPORT-2.3] Use datetime.date for date type when converting Spark DataFrame to Pandas DataFrame. ## What changes were proposed in this pull request? This is a

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18555 Seems fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20495: [SPARK-23327] [SQL] Update the description and te...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20495#discussion_r166205775 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1655,15 +1655,17 @@ case class Left(str:

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20487 looks like this PR doesn't skip the "old Pandas" tests, but rewrite them? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...

2018-02-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20473 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20473 Thank you @felixcheung, @yhuai, @ueshin and @BryanCutler for reviewing this. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20473 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20506 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20506 @ueshin can you send a new PR for 2.3? it conflicts, thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20506 LGTM, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19340: [SPARK-22119][ML] Add cosine distance to KMeans

2018-02-05 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/19340 @mgaido91 agree that it is better to normalize centers --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20493 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20493: [SPARK-23326][WEBUI]schedulerDelay should return 0 when ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20493 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/20493#discussion_r166197592 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusUtilsSuite.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #20493: [SPARK-23326][WEBUI]schedulerDelay should return 0 when ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20493 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20493#discussion_r166197254 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusUtilsSuite.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20495: [SPARK-23327] [SQL] Update the description and te...

2018-02-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20495#discussion_r166196777 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala --- @@ -1655,15 +1655,17 @@ case class

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87092/ Test PASSed. ---

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20506 **[Test build #87092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87092/testReport)** for PR 20506 at commit

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87087/ Test PASSed. ---

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87087/testReport)** for PR 20477 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20513 **[Test build #87093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87093/testReport)** for PR 20513 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/614/

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20513 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-05 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18555 cc @HyukjinKwon, --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87088/ Test FAILed. ---

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20513 **[Test build #87088 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87088/testReport)** for PR 20513 at commit

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 After went through the changes here, I think we only need to update 2 nodes to include table name in `nodeName`: hive table scan and in-memory table scan. ---

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87086/ Test PASSed. ---

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20485 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20485 **[Test build #87086 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87086/testReport)** for PR 20485 at commit

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r166193218 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -169,10 +171,12 @@ case class LogicalRDD( case class

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r166193203 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala --- @@ -103,6 +103,8 @@ case class ExternalRDDScanExec[T](

[GitHub] spark pull request #20448: [SPARK-23203][SQL] make DataSourceV2Relation immu...

2018-02-05 Thread cloud-fan
Github user cloud-fan closed the pull request at: https://github.com/apache/spark/pull/20448 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/613/

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20506 **[Test build #87092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87092/testReport)** for PR 20506 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20513 LGTM pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20513: [SPARK-23312][SQL][followup] add a config to turn...

2018-02-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20513#discussion_r166192327 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -61,6 +61,9 @@ case class

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r166192233 --- Diff: python/pyspark/sql/tests.py --- @@ -4062,18 +4062,42 @@ def test_vectorized_udf_unsupported_types(self): with

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r166191974 --- Diff: python/pyspark/sql/types.py --- @@ -1694,6 +1694,21 @@ def from_arrow_schema(arrow_schema): for field in arrow_schema])

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 After more thoughts, I feel it's reasonable to include table information in the node name. The UI displays `nodeName` in the plan graph, and displays `simpleString` in a pop-up window

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20507 also cc @cloud-fan @gatorsmile @sameeragarwal --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20419 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87084/ Test PASSed. ---

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20419 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20419: [SPARK-23032][SQL][FOLLOW-UP]Add codegenStageId in comme...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20419 **[Test build #87084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87084/testReport)** for PR 20419 at commit

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/612/

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20495 **[Test build #87091 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87091/testReport)** for PR 20495 at commit

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20495 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20506 @HyukjinKwon SGTM! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r166189478 --- Diff: python/pyspark/sql/tests.py --- @@ -4062,18 +4062,42 @@ def test_vectorized_udf_unsupported_types(self): with

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r166189014 --- Diff: python/pyspark/sql/types.py --- @@ -1694,6 +1694,21 @@ def from_arrow_schema(arrow_schema): for field in arrow_schema])

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87085/ Test FAILed. ---

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20495 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20495 **[Test build #87085 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87085/testReport)** for PR 20495 at commit

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-05 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/20514 LGTM, thanks for fixing this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20506 I originally thought similarly but after another look into this again, it seems it would rather be better to keep it consistent with what Pandas does for now. FYI, seems `datetime.date` ->

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20510 **[Test build #87090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87090/testReport)** for PR 20510 at commit

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/611/

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20514 **[Test build #87089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87089/testReport)** for PR 20514 at commit

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20514 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/610/

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20514 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issue...

2018-02-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20514 cc @sitalkedia @sameeragarwal --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20514: [SPARK-23310][CORE][FOLLOWUP] Fix Java style chec...

2018-02-05 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20514 [SPARK-23310][CORE][FOLLOWUP] Fix Java style check issues. ## What changes were proposed in this pull request? This is a follow-up of #20492 which broke lint-java checks. This pr fixes

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20513 **[Test build #87088 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87088/testReport)** for PR 20513 at commit

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20513 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/609/

[GitHub] spark pull request #20513: [SPARK-23312][SQL][followup] add a config to turn...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20513#discussion_r166184445 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -61,6 +61,9 @@ case class

[GitHub] spark issue #20513: [SPARK-23312][SQL][followup] add a config to turn off ve...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20513 @sameeragarwal @kiszk @viirya @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20513: [SPARK-23312][SQL][followup] add a config to turn...

2018-02-05 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20513 [SPARK-23312][SQL][followup] add a config to turn off vectorized cache reader ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/20483 tried to

[GitHub] spark pull request #20508: [SPARK-23335][SQL] Should not convert to double w...

2018-02-05 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/20508#discussion_r166182782 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -327,6 +327,14 @@ object TypeCoercion {

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20493#discussion_r166181600 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusUtilsSuite.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20493#discussion_r166181455 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusUtils.scala --- @@ -17,16 +17,23 @@ package org.apache.spark.status

[GitHub] spark issue #20511: [SPARK-23340][BUILD] Update ORC to 1.4.2

2018-02-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20511 Thank you for review, @gatorsmile and @HyukjinKwon . Sure, this is for Apache Spark 2.4. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 The failure is due to flaky test suite. ``` org.apache.spark.sql.hive.client.HiveClientSuites.(It is not a test it is a sbt.testing.NestedSuiteSelector) ``` jenkins, retest this

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r166179612 --- Diff: python/pyspark/sql/dataframe.py --- @@ -2020,8 +2021,6 @@ def _to_corrected_pandas_type(dt): return np.int32 elif type(dt)

[GitHub] spark issue #19340: [SPARK-22119][ML] Add cosine distance to KMeans

2018-02-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19340 @mgaido91 what do you think about the right follow-up here? as in your comment just above? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87087/testReport)** for PR 20477 at commit

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/608/

[GitHub] spark pull request #20477: [SPARK-23303][SQL] improve the explain result for...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20477#discussion_r166175748 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExec.scala --- @@ -36,11 +38,14 @@ import

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/607/

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20485 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20485 **[Test build #87086 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87086/testReport)** for PR 20485 at commit

[GitHub] spark issue #20485: [SPARK-23315][SQL] failed to get output from canonicaliz...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20485 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

  1   2   3   4   >