[GitHub] spark pull request #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15474 [DO_NOT_MERGE] Test netty ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain h

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15474 **[Test build #66925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66925/consoleFull)** for PR 15474 at commit [`f3c8b7d`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15474 **[Test build #66925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66925/consoleFull)** for PR 15474 at commit [`f3c8b7d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15474 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66925/ Test FAILed. ---

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15474 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83287126 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/LogPage.scala --- @@ -115,6 +117,17 @@ private[ui] class LogPage(parent: WorkerWebUI) extends W

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83286425 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/LogPage.scala --- @@ -115,6 +117,17 @@ private[ui] class LogPage(parent: WorkerWebUI) extends W

[GitHub] spark pull request #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15285#discussion_r83342325 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1471,16 +1494,15 @@ private[spark] object Utils extends Logging { * and `endIndex

[GitHub] spark pull request #15284: [SPARK-17368] [SQL] Add support for value class s...

2016-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15284 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83342839 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala --- @@ -34,7 +34,7 @@ import org.apache.spark.util.Utils // The data where

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15453 **[Test build #66926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66926/consoleFull)** for PR 15453 at commit [`520e207`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread bdrillard
Github user bdrillard commented on a diff in the pull request: https://github.com/apache/spark/pull/15453#discussion_r83343796 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/ObjectType.scala --- @@ -19,7 +19,7 @@ package org.apache.spark.sql.types impor

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-13 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15475 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark pull request #15475: [WIP] [SQL] Enable metastore partition pruning by...

2016-10-13 Thread ericl
GitHub user ericl opened a pull request: https://github.com/apache/spark/pull/15475 [WIP] [SQL] Enable metastore partition pruning by default. ## What changes were proposed in this pull request? This should apply to non-converted metastore relations. WIP to see if this caus

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15453 **[Test build #66926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66926/consoleFull)** for PR 15453 at commit [`520e207`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15453 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66926/ Test FAILed. ---

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15453 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15475 **[Test build #66927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66927/consoleFull)** for PR 15475 at commit [`2581380`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15471: [SPARK-17919] Make timeout to RBackend configurable in S...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15471 **[Test build #66928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66928/consoleFull)** for PR 15471 at commit [`d27233c`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15453 **[Test build #66929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66929/consoleFull)** for PR 15453 at commit [`232c01e`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15453 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15453 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66929/ Test FAILed. ---

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15453 **[Test build #66929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66929/consoleFull)** for PR 15453 at commit [`232c01e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15453: [SPARK-17770] [CATALYST] making ObjectType public

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15453 **[Test build #66930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66930/consoleFull)** for PR 15453 at commit [`e064667`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14158: [SPARK-13547] [SQL] [WEBUI] Add SQL query in web UI's SQ...

2016-10-13 Thread nblintao
Github user nblintao commented on the issue: https://github.com/apache/spark/pull/14158 @markhamstra Queries longer than 1000 chars are truncated to 1000 chars after the parsing. "+more" will only expand 140 chars to at most 1000 chars. --- If your project is set up for it, you can r

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14690 **[Test build #66931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66931/consoleFull)** for PR 14690 at commit [`e1635e4`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #15476: [SPARK-17926][SQL][STREAMING] Added json for stat...

2016-10-13 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/15476 [SPARK-17926][SQL][STREAMING] Added json for statuses ## What changes were proposed in this pull request? StreamingQueryStatus exposed through StreamingQueryListener often needs to be recorde

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15190 **[Test build #66919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66919/consoleFull)** for PR 15190 at commit [`f32fe21`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15190 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15190 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66919/ Test FAILed. ---

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15476 **[Test build #66932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66932/consoleFull)** for PR 15476 at commit [`de259a0`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #15382: [SPARK-17810] [SQL] Default spark.sql.warehouse.d...

2016-10-13 Thread avulanov
Github user avulanov commented on a diff in the pull request: https://github.com/apache/spark/pull/15382#discussion_r83346125 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -757,7 +758,10 @@ private[sql] class SQLConf extends Serializable with

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-13 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15476 @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-13 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/15421 I just tried the patch on R version 3.3.1 (2016-06-21) -- "Bug in Your Hair" on Linux and it passed tests. @HyukjinKwon how can kick another AppVeyor test? --- If your project is set up fo

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14690 **[Test build #66923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66923/consoleFull)** for PR 14690 at commit [`765f93c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66923/ Test FAILed. ---

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14690 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #7963: [SPARK-6227] [MLlib] [PySpark] Implement PySpark wrappers...

2016-10-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/7963 @MechCoder Thanks! I'll look around and see if anyone else is interested in taking this over and bringing it to the finish line otherwise I'll pick it up myself after OSCON :) --- If your project i

[GitHub] spark issue #15421: [SPARK-17811] SparkR cannot parallelize data.frame with ...

2016-10-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15421 Build started: [SparkR] `ALL` [![PR-15421](https://ci.appveyor.com/api/projects/status/github/spark-test/spark?branch=82AF3188-11E3-40D8-A71C-7806413EDAE3&svg=true)](https://ci.appveyor.com/proje

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83347251 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val

[GitHub] spark pull request #15477: [SPARK-17927][SQL] Remove dead code in WriterCont...

2016-10-13 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/15477 [SPARK-17927][SQL] Remove dead code in WriterContainer. ## What changes were proposed in this pull request? speculationEnabled and DATASOURCE_OUTPUTPATH seem like just dead code. ## How wa

[GitHub] spark issue #15477: [SPARK-17927][SQL] Remove dead code in WriterContainer.

2016-10-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15477 cc @hvanhovell @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15477: [SPARK-17927][SQL] Remove dead code in WriterContainer.

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15477 **[Test build #66933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66933/consoleFull)** for PR 15477 at commit [`4597a32`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83347636 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val

[GitHub] spark issue #15473: [SPARK-17925][SQL] Break fileSourceInterfaces.scala into...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15473 **[Test build #66934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66934/consoleFull)** for PR 15473 at commit [`be176c3`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83347741 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val METR

[GitHub] spark issue #12761: [SPARK-14464] [MLLIB] Better support for logistic regres...

2016-10-13 Thread daniel-siegmann-aol
Github user daniel-siegmann-aol commented on the issue: https://github.com/apache/spark/pull/12761 I'll work on merging the changes. However, I no longer work at AOL so I don't have the data to do thoroughly test it. --- If your project is set up for it, you can reply to this email a

[GitHub] spark issue #15473: [SPARK-17925][SQL] Break fileSourceInterfaces.scala into...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15473 **[Test build #66924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66924/consoleFull)** for PR 15473 at commit [`2a824ba`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15473: [SPARK-17925][SQL] Break fileSourceInterfaces.scala into...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15473 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66924/ Test FAILed. ---

[GitHub] spark issue #15473: [SPARK-17925][SQL] Break fileSourceInterfaces.scala into...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15473 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15469: [SPARK-17900][SQL] Graduate a list of Spark SQL APIs to ...

2016-10-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15469 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83348022 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83348200 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val METR

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83348167 --- Diff: core/src/main/scala/org/apache/spark/metrics/source/StaticSources.scala --- @@ -60,3 +60,32 @@ object CodegenMetrics extends Source { val METR

[GitHub] spark issue #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-level sta...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15360 **[Test build #66935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66935/consoleFull)** for PR 15360 at commit [`d782c14`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15468 **[Test build #66920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66920/consoleFull)** for PR 15468 at commit [`f6dc8ac`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66920/ Test PASSed. ---

[GitHub] spark issue #15468: [SPARK-17915][SQL] Prepare a new ColumnVector implementa...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15468 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13988: [SPARK-16101][SQL] Refactoring CSV data source to be con...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13988 **[Test build #66936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66936/consoleFull)** for PR 13988 at commit [`97eec87`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83349072 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala --- @@ -34,7 +34,7 @@ import org.apache.spark.util.Utils // The data whe

[GitHub] spark issue #15297: [WIP][SPARK-9862]Handling data skew

2016-10-13 Thread ron8hu
Github user ron8hu commented on the issue: https://github.com/apache/spark/pull/15297 cc @rxin @hvanhovell Can you review and comment this PR? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-level sta...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15360 **[Test build #66937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66937/consoleFull)** for PR 15360 at commit [`30ac539`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14690 **[Test build #66938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66938/consoleFull)** for PR 14690 at commit [`71049d1`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #15398: [SPARK-17647][SQL][WIP] Fix backslash escaping in...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15398#discussion_r83349251 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/RegexpExpressionsSuite.scala --- @@ -54,20 +57,32 @@ class RegexpExpressi

[GitHub] spark pull request #15398: [SPARK-17647][SQL][WIP] Fix backslash escaping in...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15398#discussion_r83349295 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/RegexpExpressionsSuite.scala --- @@ -90,6 +105,28 @@ class RegexpExpressi

[GitHub] spark pull request #15398: [SPARK-17647][SQL][WIP] Fix backslash escaping in...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15398#discussion_r83349306 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/RegexpExpressionsSuite.scala --- @@ -90,6 +105,28 @@ class RegexpExpressi

[GitHub] spark issue #13988: [SPARK-16101][SQL] Refactoring CSV data source to be con...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13988 **[Test build #66939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66939/consoleFull)** for PR 13988 at commit [`697f276`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #15297: [WIP][SPARK-9862]Handling data skew

2016-10-13 Thread ron8hu
Github user ron8hu commented on the issue: https://github.com/apache/spark/pull/15297 The design note of this PR has been posted at jira page. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15460: [SPARK-17903][SQL] MetastoreRelation should talk to exte...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15460 LGTM except one comment about test case --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to ru...

2016-10-13 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/15338#discussion_r83349053 --- Diff: sbin/spark-daemon.sh --- @@ -146,13 +176,11 @@ run_command() { case "$mode" in (class) - nohup nice -n "$SPARK_NICEN

[GitHub] spark pull request #15460: [SPARK-17903][SQL] MetastoreRelation should talk ...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15460#discussion_r83349707 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -636,6 +637,13 @@ private[spark] class HiveExternalCatalog(co

[GitHub] spark issue #15460: [SPARK-17903][SQL] MetastoreRelation should talk to exte...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15460 **[Test build #66922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66922/consoleFull)** for PR 15460 at commit [`628fd05`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to ru...

2016-10-13 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/15338#discussion_r83349154 --- Diff: sbin/spark-daemon.sh --- @@ -122,6 +123,35 @@ if [ "$SPARK_NICENESS" = "" ]; then export SPARK_NICENESS=0 fi +execute_comma

[GitHub] spark issue #15460: [SPARK-17903][SQL] MetastoreRelation should talk to exte...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15460 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66922/ Test PASSed. ---

[GitHub] spark issue #15460: [SPARK-17903][SQL] MetastoreRelation should talk to exte...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15460 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15430: [SPARK-15957][Follow-up][ML][PySpark] Add Python API for...

2016-10-13 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15430 Since it's a minor change, I will merge it into master. Any comments are still appreciated and I will address in follow-up work. Thanks. --- If your project is set up for it, you can reply to th

[GitHub] spark pull request #15338: [SPARK-11653][Deploy] Allow spark-daemon.sh to ru...

2016-10-13 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/15338#discussion_r83349121 --- Diff: sbin/spark-daemon.sh --- @@ -146,13 +176,11 @@ run_command() { case "$mode" in (class) - nohup nice -n "$SPARK_NICEN

[GitHub] spark pull request #15430: [SPARK-15957][Follow-up][ML][PySpark] Add Python ...

2016-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15430 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15354: [SPARK-17764][SQL] Add `to_json` supporting to co...

2016-10-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15354#discussion_r83350521 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala --- @@ -343,4 +343,23 @@ class JsonExpression

[GitHub] spark pull request #15478: [SPARK-17899][SQL][DO-NOT-MERGE] Check the Impact...

2016-10-13 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/15478 [SPARK-17899][SQL][DO-NOT-MERGE] Check the Impact When Turning On spark.sql.debug ### What changes were proposed in this pull request? **DO NOT MERGE** This PR is just to do a quick

[GitHub] spark issue #15478: [SPARK-17899][SQL][DO-NOT-MERGE] Check the Impact When T...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15478 **[Test build #66940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66940/consoleFull)** for PR 15478 at commit [`26df77a`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #66941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66941/consoleFull)** for PR 15044 at commit [`eeb2747`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15360: [SPARK-17073] [SQL] [FOLLOWUP] generate column-level sta...

2016-10-13 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15360 resolve conflicts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or i

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 @tdas Add `fileUncompressedLengthCacheSize` to executor conf. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark issue #14158: [SPARK-13547] [SQL] [WEBUI] Add SQL query in web UI's SQ...

2016-10-13 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/14158 @nblintao Got it; thanks. There may be distinct queries that will be entirely the same within the first 1000 characters, but that's just the nature of working with these very large queries -- t

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15285 **[Test build #66942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66942/consoleFull)** for PR 15285 at commit [`a6fbebe`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15476 **[Test build #66932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66932/consoleFull)** for PR 15476 at commit [`de259a0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15476 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15476: [SPARK-17926][SQL][STREAMING] Added json for statuses

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66932/ Test FAILed. ---

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14690 **[Test build #66931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66931/consoleFull)** for PR 14690 at commit [`e1635e4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66931/ Test FAILed. ---

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14690 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15475 **[Test build #66927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66927/consoleFull)** for PR 15475 at commit [`2581380`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15475 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15475: [WIP] [SQL] Enable metastore partition pruning by defaul...

2016-10-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15475 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66927/ Test PASSed. ---

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15190 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #15474: [DO_NOT_MERGE] Test netty

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15474 **[Test build #66943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66943/consoleFull)** for PR 15474 at commit [`ca7295e`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #15190: [SPARK-17620][SQL] Determine Serde by hive.default.filef...

2016-10-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15190 **[Test build #66944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66944/consoleFull)** for PR 15190 at commit [`f32fe21`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83352838 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -626,6 +627,40 @@ private[spark] class HiveExternalCatalog(con

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-10-13 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r83352979 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -626,6 +627,40 @@ private[spark] class HiveExternalCatalog(conf:

<    1   2   3   4   5   6   >