[GitHub] spark issue #20525: SPARK-23271 Parquet output contains only _SUCCESS file a...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/656/

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87148/testReport)** for PR 20525 at commit

[GitHub] spark pull request #20525: SPARK-23271 Parquet output contains only _SUCCESS...

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166540368 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileFormatWriterSuite.scala --- @@ -32,6 +33,24 @@ class

[GitHub] spark issue #20525: SPARK-23271 Parquet output contains only _SUCCESS file a...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20525: SPARK-23271 Parquet output contains only _SUCCESS...

2018-02-06 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r166540285 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -301,7 +301,6 @@ class DataFrameReaderWriterSuite

[GitHub] spark issue #20525: SPARK-23271 Parquet output contains only _SUCCESS file a...

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20525 Update the title to `[SPARK-23271] [SQL] Parquet output contains only _SUCCESS file after writing an empty dataframe` --- -

[GitHub] spark issue #20525: SPARK-23271 Parquet output contains only _SUCCESS file a...

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20525 cc @cloud-fan @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20525: SPARK-23271 Parquet output contains only _SUCCESS...

2018-02-06 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/20525 SPARK-23271 Parquet output contains only _SUCCESS file after writing an empty dataframe ## What changes were proposed in this pull request? Below are the two cases. ``` SQL case 1

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87141/ Test PASSed. ---

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87141 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87141/testReport)** for PR 20509 at commit

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20524 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly specify Pandas an...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r166536018 --- Diff: python/pyspark/sql/session.py --- @@ -646,6 +646,9 @@ def createDataFrame(self, data, schema=None, samplingRatio=None, verifySchema=Tr

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87139/ Test PASSed. ---

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87139/testReport)** for PR 20487 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20484 @dongjoon-hyun This is still a regression to the existing Hive ORC users. cc @cloud-fan @sameeragarwal Maybe we should fix it before the release? ---

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20484 > Are they still effective in Hive? I just want to confirm whether all the Hive readers work fine. Could you add a test case like what we did in `CliSuite`? ---

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87138/ Test PASSed. ---

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87138/testReport)** for PR 19041 at commit

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87140/ Test FAILed. ---

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87140 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87140/testReport)** for PR 20521 at commit

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20524 **[Test build #87147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87147/testReport)** for PR 20524 at commit

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20524 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/655/

[GitHub] spark issue #20524: [SPARK-23345][SQL] Remove open stream record even closin...

2018-02-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20524 cc @cloud-fan @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20516: [SPARK-23343][CORE][TEST] Increase the exception test fo...

2018-02-06 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20516 @srowen please review it again.thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20524: [SPARK-23345][SQL] Remove open stream record even...

2018-02-06 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20524 [SPARK-23345][SQL] Remove open stream record even closing it fails ## What changes were proposed in this pull request? When `DebugFilesystem` closes opened stream, if any exception occurs,

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166526321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,139 @@ package

[GitHub] spark pull request #20378: [SPARK-11222][Build][Python] Python document styl...

2018-02-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20378#discussion_r166525289 --- Diff: dev/lint-python --- @@ -83,6 +84,53 @@ else rm "$PEP8_REPORT_PATH" fi + Python Document Style Checks + +#

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20503 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20503 **[Test build #87143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87143/testReport)** for PR 20503 at commit

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87143/ Test PASSed. ---

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87146/testReport)** for PR 20477 at commit

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/654/

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20477 The result was out-dated, I've updated it the PR description, please check again, thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87145/testReport)** for PR 20477 at commit

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/653/

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87137/ Test FAILed. ---

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19041 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87137/testReport)** for PR 19041 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20445 **[Test build #87144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87144/testReport)** for PR 20445 at commit

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20445: [SPARK-23092][SQL] Migrate MemoryStream to DataSourceV2 ...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20445 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/652/

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20503 **[Test build #87143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87143/testReport)** for PR 20503 at commit

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-06 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20519 >corner case described in the JIRA I don't have a good feeling for how often people use python site customizations. I did encounter a real life case that caused the failure described

[GitHub] spark issue #20503: [SPARK-23299][SQL][PYSPARK] Fix __repr__ behaviour for R...

2018-02-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20503 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166520104 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87142/testReport)** for PR 19041 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87140 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87140/testReport)** for PR 20521 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87141/testReport)** for PR 20509 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/651/

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/650/

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166517749 --- Diff: core/src/test/scala/org/apache/spark/CacheRecoveryManagerSuite.scala --- @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166517817 --- Diff: core/src/test/scala/org/apache/spark/CacheRecoveryManagerSuite.scala --- @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166517771 --- Diff: core/src/test/scala/org/apache/spark/CacheRecoveryManagerSuite.scala --- @@ -0,0 +1,200 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166517452 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -72,13 +72,14 @@ case class

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-06 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/20477 @cloud-fan I have a question about the Optimized Logical Plan. In the "What changed were proposed" section, it is said that after this PR, the Optimized Logical Plan will be as following

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166517178 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -493,9 +510,23 @@ case class DataSource(

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r166514084 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataWriterFactory.java --- @@ -20,6 +20,7 @@ import java.io.Serializable;

[GitHub] spark issue #20523: [SPARK-23122][PYSPARK][FOLLOWUP] Replace registerTempTab...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20523 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20523: [SPARK-23122][PYSPARK][FOLLOWUP] Replace registerTempTab...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20523 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87135/ Test PASSed. ---

[GitHub] spark issue #20523: [SPARK-23122][PYSPARK][FOLLOWUP] Replace registerTempTab...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20523 **[Test build #87135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87135/testReport)** for PR 20523 at commit

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87139/testReport)** for PR 20487 at commit

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/649/

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20487 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20304: [SPARK-23139]Read eventLog file with mixed encodings

2018-02-06 Thread hellodengfei
Github user hellodengfei commented on the issue: https://github.com/apache/spark/pull/20304 Sorry, it's too late for UT. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166510668 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -72,13 +72,14 @@ case class

[GitHub] spark pull request #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20521#discussion_r166508678 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -438,7 +440,7 @@ case class DataSource( *

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87134/ Test FAILed. ---

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly specify Pandas and PyArr...

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87134/testReport)** for PR 20487 at commit

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166508110 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -123,6 +123,16 @@ package object config {

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166507738 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166507761 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20516: [SPARK-23343][CORE][TEST] Increase the exception ...

2018-02-06 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/20516#discussion_r166507524 --- Diff: core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferServiceSuite.scala --- @@ -77,16 +79,68 @@ class

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87138/testReport)** for PR 19041 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87136/ Test PASSed. ---

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #87136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87136/testReport)** for PR 20484 at commit

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166506093 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166505998 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166505845 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166505712 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19041: [SPARK-21097][CORE] Add option to recover cached ...

2018-02-06 Thread brad-kaiser
Github user brad-kaiser commented on a diff in the pull request: https://github.com/apache/spark/pull/19041#discussion_r166505668 --- Diff: core/src/main/scala/org/apache/spark/CacheRecoveryManager.scala --- @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19041: [SPARK-21097][CORE] Add option to recover cached data

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19041 **[Test build #87137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87137/testReport)** for PR 19041 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20484 **[Test build #87136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87136/testReport)** for PR 20484 at commit

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/648/

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20484 For the HiveConf configuration file, I tested manually. It works correctly. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20509 nit: we can remove some imports due to moving into the same package, from `ContinuousReadSupport.java`, `MicroBatchReadSupport.java`, and `StreamWriteSupport.java`. ---

[GitHub] spark pull request #20244: [SPARK-23053][CORE] taskBinarySerialization and t...

2018-02-06 Thread ivoson
Github user ivoson commented on a diff in the pull request: https://github.com/apache/spark/pull/20244#discussion_r166502288 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2399,6 +2424,121 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle ...

2018-02-06 Thread lucio-yz
Github user lucio-yz commented on a diff in the pull request: https://github.com/apache/spark/pull/20472#discussion_r166501185 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -1001,11 +996,19 @@ private[spark] object RandomForest extends

  1   2   3   4   5   6   >