[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20537 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/679/

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20537 This is pretty important bug fix that we should try to get in Spark 2.3...Thanks @felixcheung for reporting this! --- - To

[GitHub] spark issue #20537: [SPARK-23314][PYTHON] Add ambiguous=False when localizin...

2018-02-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20537 cc @felixcheung @BryanCutler @ueshin @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166780472 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166782197 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-07 Thread icexelloss
GitHub user icexelloss opened a pull request: https://github.com/apache/spark/pull/20537 [SPARK-23314][PYTHON] Add ambiguous=False when localizing tz-naive timestamps to deal with dst ## What changes were proposed in this pull request? When tz_localize a tz-naive timetamp,

[GitHub] spark pull request #20518: [SPARK-22119][FOLLOWUP][ML] Use spherical KMeans ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20518#discussion_r166780413 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala --- @@ -745,4 +763,27 @@ private[spark] class CosineDistanceMeasure extends

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166779585 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 seems like a unrelated flaky test ^ --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166778741 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166778140 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20484 ok --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166776296 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20519 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87173/ Test PASSed. ---

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20519 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87173/testReport)** for PR 20519 at commit

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166774944 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166774153 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166771387 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,18 @@ private[spark] object RandomForest extends

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166770380 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -931,7 +925,8 @@ private[spark] object RandomForest extends Logging {

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166771033 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,18 @@ private[spark] object RandomForest extends

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166773606 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,18 @@ private[spark] object RandomForest extends

[GitHub] spark pull request #20527: [SPARK-23348][SQL] append data using saveAsTable ...

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20527#discussion_r166770787 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -132,6 +134,32 @@ class InMemoryCatalogedDDLSuite

[GitHub] spark pull request #20529: [SPARK-23350][SS]Bug fix for exception handling w...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20529#discussion_r166768784 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -92,12 +92,14 @@ case class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767609 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767674 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767449 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166767860 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166766796 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,53 @@ class

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87172/ Test PASSed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87172/testReport)** for PR 20474 at commit

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166767068 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -190,9 +190,13 @@ object FileFormatWriter

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87175/ Test PASSed. ---

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20536 **[Test build #87175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87175/testReport)** for PR 20536 at commit

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r166763629 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,44 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20484 Oh, I thought in this way, @tgravescs . - For ORC files, the parameter names (ORC/Hive) are the same as we see https://github.com/apache/spark/pull/20517 . - For Hive tables, we don't

[GitHub] spark issue #20495: [SPARK-23327] [SQL] Update the description and tests of ...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20495 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20524 LGTM Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20536 Thank you for pining me. +1 for the decision. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87174/ Test FAILed. ---

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87174/testReport)** for PR 20445 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87177/ Test PASSed. ---

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #87177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87177/testReport)** for PR 20484 at commit

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20382 #20445 will be merged in a few hours. please go ahead and update your PR with the refactoring that was suggested (mainly, no v1 version). ---

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/20484 why did you remove the bit about the orc configs? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #87177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87177/testReport)** for PR 20484 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/678/

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20484 I see. I removed `spark.sql.hive.convertMetastoreOrc` and Hive ORC table stuff from this PR accordingly. We can add that later if we fix the regresson of `convertMetastoreOrc/Parquet`. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87170/ Test PASSed. ---

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87170/testReport)** for PR 20387 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20484 Just FYI, https://github.com/apache/spark/pull/20536 is reverting the conf `convertMetastoreOrc` back `false`. However, we still can turn it on by default in 2.3 after we fix the regression.

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87169/ Test PASSed. ---

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87169/testReport)** for PR 20490 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87176/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/677/

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20445 LGTM pending passing run of that HiveDDLSuite test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166736890 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -149,18 +149,12 @@ case class MemoryStream[A : Encoder](id:

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20536 **[Test build #87175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87175/testReport)** for PR 20536 at commit

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20536 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/676/

[GitHub] spark issue #20536: Revert [SPARK-22279][SQL] Turn on spark.sql.hive.convert...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20536 After a discussion with @marmbrus @rxin @yhuai , we decided to revert this for avoiding any regression and blocking the release of Spark 2.3. cc @sameeragarwal @dongjoon-hyun @cloud-fan

[GitHub] spark pull request #20536: Revert "[SPARK-22279][SQL] Turn on spark.sql.hive...

2018-02-07 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20536 Revert "[SPARK-22279][SQL] Turn on spark.sql.hive.convertMetastoreOrc by default ## What changes were proposed in this pull request? This is to revert the changes made in

[GitHub] spark issue #20530: [SPARK-23349][SQL]ShuffleExchangeExec: Duplicate and red...

2018-02-07 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20530 @wujianping10043419 this change is too trivial to merge, I am sorry. The main reason for not merging is because this will mess up the git blame for the given code, which is sometimes very useful

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166725859 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala --- @@ -149,18 +149,12 @@ case class MemoryStream[A :

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/675/

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87174/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSo...

2018-02-07 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20445#discussion_r166720161 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/ForeachSinkSuite.scala --- @@ -46,49 +46,34 @@ class ForeachSinkSuite extends

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-07 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20445 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r166716818 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,44 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-07 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20373 @HyukjinKwon would it be good to update this PR to match the upcoming 4.3 release you are working on? If the code is the same, then just updating the title/description so it is clear ---

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87173/testReport)** for PR 20519 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/20499 I'd fix this in 2.3, and 2.2.1 as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166709990 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -171,7 +171,8 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87165/ Test PASSed. ---

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87172/testReport)** for PR 20474 at commit

[GitHub] spark issue #20534: [SPARK-23319][TESTS][BRANCH-2.3] Explicitly specify Pand...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20534 **[Test build #87165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87165/testReport)** for PR 20534 at commit

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20474 jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20531 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20531 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87164/ Test PASSed. ---

[GitHub] spark issue #20531: [SPARK-23352][PYTHON] Explicitly specify supported types...

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20531 **[Test build #87164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87164/testReport)** for PR 20531 at commit

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87167/ Test FAILed. ---

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20474 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20474 **[Test build #87167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87167/testReport)** for PR 20474 at commit

[GitHub] spark issue #19145: [spark-21933][yarn] Spark Streaming request more executo...

2018-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19145 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-02-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r166701501 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DataSourceOptions.java --- @@ -27,6 +27,39 @@ /** * An immutable string-to-string

[GitHub] spark pull request #20524: [SPARK-23345][SQL] Remove open stream record even...

2018-02-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20524 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20535: [SPARK-23341][SQL] define some standard options for data...

2018-02-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20535 This should move the standard options to `DataSourceV2Relation` to avoid needing to instantiate `DataSourceOptions` wherever the relation is created. ---

[GitHub] spark pull request #20524: [SPARK-23345][SQL] Remove open stream record even...

2018-02-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20524#discussion_r166699026 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSparkSession.scala --- @@ -111,7 +111,7 @@ trait SharedSparkSession

<    1   2   3   4   5   6   >