[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-01 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20473 @HyukjinKwon Good catch! Yeah, we should use it there. Could you fix it please? Thanks! --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...

2018-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20473#discussion_r165573545 --- Diff: python/run-tests.py --- @@ -151,6 +152,68 @@ def parse_opts(): return opts +def _check_dependencies(python_exec, modul

[GitHub] spark issue #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyArrow ar...

2018-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20473 Will double check and open a PR tonight .. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20446: [SPARK-23254][ML] Add user guide entry for DataFr...

2018-02-01 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20446#discussion_r165573866 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/SummarizerExample.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Soft

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20446 **[Test build #86975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86975/testReport)** for PR 20446 at commit [`fc9622b`](https://github.com/apache/spark/commit/fc

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165574190 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165573955 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165574082 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/515/ Test

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20479 Ah, I moved both of them into `FileBasedDataSourceSuite`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20479 Thank you for review, @HyukjinKwon , @viirya , and @gatorsmile . Now, there is no redundancy. --- - To unsubscribe, e-m

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20446 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/516/ Test

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20446: [SPARK-23254][ML] Add user guide entry for DataFr...

2018-02-01 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/20446#discussion_r165575680 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/SummarizerExample.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ignoreMiss...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20479 **[Test build #86976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86976/testReport)** for PR 20479 at commit [`4512d72`](https://github.com/apache/spark/commit/45

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165575849 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ign...

2018-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20479#discussion_r165575853 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -92,4 +96,39 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165575817 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20446 **[Test build #86975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86975/testReport)** for PR 20446 at commit [`fc9622b`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20446 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20446: [SPARK-23254][ML] Add user guide entry for DataFrame mul...

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86975/ Test PASSed. ---

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165576328 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,66 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ign...

2018-02-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20479#discussion_r165576607 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -92,4 +96,39 @@ class FileBasedDataSourceSuite extends Qu

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/517/ Test

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #86977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86977/testReport)** for PR 20484 at commit [`fc5b395`](https://github.com/apache/spark/commit/fc

[GitHub] spark pull request #20479: [SPARK-23305][SQL][TEST] Add `spark.sql.files.ign...

2018-02-01 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20479#discussion_r165577600 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -92,4 +96,39 @@ class FileBasedDataSourceSuite extends Quer

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #86978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86978/testReport)** for PR 20484 at commit [`7b3b0a4`](https://github.com/apache/spark/commit/7b

[GitHub] spark pull request #20485: [SPARK-23315][SQL] failed to get output from cano...

2018-02-01 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20485 [SPARK-23315][SQL] failed to get output from canonicalized data source v2 related plans ## What changes were proposed in this pull request? `DataSourceV2Relation` keeps a `fullOutput` a

[GitHub] spark pull request #20446: [SPARK-23254][ML] Add user guide entry for DataFr...

2018-02-01 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20446#discussion_r165578020 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/SummarizerExample.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to the Apache Soft

<    2   3   4   5   6   7