[GitHub] spark issue #14921: [SPARK-17361][SQL] file-based external table without pat...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14921 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64888/ Test PASSed. ---

[GitHub] spark issue #14921: [SPARK-17361][SQL] file-based external table without pat...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14921 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14921: [SPARK-17361][SQL] file-based external table without pat...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14921 **[Test build #64888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64888/consoleFull)** for PR 14921 at commit

[GitHub] spark issue #14937: [WIP] [SPARK-8519][SPARK-11560] [ML] [MLlib] Optimize KM...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14937 **[Test build #64890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64890/consoleFull)** for PR 14937 at commit

[GitHub] spark issue #14919: [SPARK-17354][SQL] Partitioning by dates/timestamps shou...

2016-09-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14919 @sameeragarwal Thanks for your comment. Yeap, I will. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14931 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64887/ Test PASSed. ---

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14931 **[Test build #64887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64887/consoleFull)** for PR 14931 at commit

[GitHub] spark pull request #14942: [SparkR][Minor] Fix docs for sparkR.session and c...

2016-09-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14942 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14942 Thanks - Merging this into master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14850: [SPARK-17279][SQL] better error message for exceptions d...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14850 **[Test build #64889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64889/consoleFull)** for PR 14850 at commit

[GitHub] spark issue #14550: [SPARK-16959] [SQL] Rebuild Table Comment when Retrievin...

2016-09-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14550 @cloud-fan To backport #14531, we need to backport this. Do you want me to do it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14302: [SPARK-16663][SQL] desc table should be consistent betwe...

2016-09-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14302 @cloud-fan To backport https://github.com/apache/spark/pull/14531, we need to backport this. Do you want me to do it? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14537: [SPARK-16948][SQL] Use metastore schema instead of infer...

2016-09-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14537 can you address this https://github.com/apache/spark/pull/14537/files#r76355262? thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveCl...

2016-09-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14888#discussion_r77430743 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -613,21 +613,22 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14712 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14712 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64886/ Test PASSed. ---

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14712 **[Test build #64886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64886/consoleFull)** for PR 14712 at commit

[GitHub] spark issue #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract `outpu...

2016-09-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14864 mark it as `@transient`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14921: [SPARK-17361][SQL] file-based external table without pat...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14921 **[Test build #64888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64888/consoleFull)** for PR 14921 at commit

[GitHub] spark pull request #14828: [SPARK-17258][SQL] Parse scientific decimal liter...

2016-09-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14828#discussion_r77430498 --- Diff: sql/core/src/test/resources/sql-tests/inputs/literals.sql --- @@ -50,14 +50,14 @@ select 1D, 1.2D, 1e10, 1.5e5, .10D, 0.10D, .1e5, .9e+2,

[GitHub] spark pull request #14921: [SPARK-17361][SQL] file-based external table with...

2016-09-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14921#discussion_r77430470 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/CatalogSuite.scala --- @@ -305,6 +306,26 @@ class CatalogSuite

[GitHub] spark issue #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveClient in...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64884/ Test PASSed. ---

[GitHub] spark issue #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveClient in...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveClient in...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14888 **[Test build #64884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64884/consoleFull)** for PR 14888 at commit

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14931 **[Test build #64887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64887/consoleFull)** for PR 14931 at commit

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14931 Updated to only flag worker loss in standalone mode. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14942 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14942 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64885/ Test PASSed. ---

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14942 **[Test build #64885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64885/consoleFull)** for PR 14942 at commit

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14712 **[Test build #64886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64886/consoleFull)** for PR 14712 at commit

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-09-02 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/14712 @gatorsmile Thank you for the good test cases! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-09-02 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14690 Yeah, I wouldn't rule out making that change since it can get us significantly better planning, but I also think we should find a better way of doing that than making pruned copies of

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77428539 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/fileSourceInterfaces.scala --- @@ -346,11 +340,30 @@ trait FileCatalog {

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14942 **[Test build #64885 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64885/consoleFull)** for PR 14942 at commit

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14867 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64882/ Test PASSed. ---

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14867 **[Test build #64882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64882/consoleFull)** for PR 14867 at commit

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77428351 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64880/ Test PASSed. ---

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14867 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14867 **[Test build #64880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64880/consoleFull)** for PR 14867 at commit

[GitHub] spark pull request #14912: [SPARK-17357][SQL] Simplified predicates should b...

2016-09-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14912#discussion_r77427992 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala --- @@ -171,6 +172,27 @@ class FilterPushdownSuite

[GitHub] spark issue #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveClient in...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14888 **[Test build #64884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64884/consoleFull)** for PR 14888 at commit

[GitHub] spark pull request #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveCl...

2016-09-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14888#discussion_r77427610 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -240,54 +240,45 @@ case class

[GitHub] spark issue #14690: [SPARK-16980][SQL] Load only catalog table partition met...

2016-09-02 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/14690 Thanks @ericl and @davies for your latest feedback. I'd like to take the opportunity to rebase this PR off of #14750 after it's merged to master before pushing another commit, however I understand

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14944 seems like the failure is related to https://github.com/apache/spark/pull/14797? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14919: [SPARK-17354][SQL] Partitioning by dates/timestamps shou...

2016-09-02 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14919 @HyukjinKwon can we add a more targeted unit test in one of the vectorized reader test suites that can explicitly test for partitioned columns? --- If your project is set up for it, you can

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14944 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64883/ Test FAILed. ---

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14944 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14944 **[Test build #64883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64883/consoleFull)** for PR 14944 at commit

[GitHub] spark issue #14537: [SPARK-16948][SQL] Use metastore schema instead of infer...

2016-09-02 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the issue: https://github.com/apache/spark/pull/14537 Sorry about the delay. Updated the PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77426981 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2531,6 +2531,8 @@ class Dataset[T] private[sql]( */ def

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77426759 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/fileSourceInterfaces.scala --- @@ -346,11 +340,30 @@ trait FileCatalog {

[GitHub] spark issue #14945: [SPARK-17386] Set default trigger interval to 1/10 secon...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14945 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-02 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14943 @tgravescs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #14945: [SPARK-17386] Set default trigger interval to 1/1...

2016-09-02 Thread frreiss
GitHub user frreiss opened a pull request: https://github.com/apache/spark/pull/14945 [SPARK-17386] Set default trigger interval to 1/10 second ## What changes were proposed in this pull request? This pull request implements the most expedient change to fix SPARK-17386:

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77426633 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77426636 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77426535 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -79,8 +79,16 @@ object FileSourceStrategy

[GitHub] spark pull request #14926: [SPARK-17365][Core] Remove/Kill multiple executor...

2016-09-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14926#discussion_r77426133 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -392,10 +397,36 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77425878 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77425776 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -79,8 +79,16 @@ object FileSourceStrategy

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-02 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r77424940 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,10 +245,19 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-02 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r77424851 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,10 +245,19 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77424735 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -184,7 +184,7 @@ case class FileSourceScanExec(

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/14944 cc @davies --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14944: [SPARK-16334][BACKPORT] Reusing same dictionary column f...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14944 **[Test build #64883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64883/consoleFull)** for PR 14944 at commit

[GitHub] spark pull request #14944: [SPARK-16334][BACKPORT] Reusing same dictionary c...

2016-09-02 Thread sameeragarwal
GitHub user sameeragarwal opened a pull request: https://github.com/apache/spark/pull/14944 [SPARK-16334][BACKPORT] Reusing same dictionary column for decoding consecutive row groups shouldn't throw an error ## What changes were proposed in this pull request? Backports

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77423958 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -184,7 +184,7 @@ case class FileSourceScanExec(

[GitHub] spark issue #14867: [SPARK-17296][SQL] Simplify parser join processing.

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14867 **[Test build #64882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64882/consoleFull)** for PR 14867 at commit

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64876/ Test PASSed. ---

[GitHub] spark issue #14881: [SPARK-17315][SparkR] Kolmogorov-Smirnov test SparkR wra...

2016-09-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14881 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14888: [SPARK-17324] [SQL] Remove Direct Usage of HiveCl...

2016-09-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14888#discussion_r77423309 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -240,54 +240,49 @@ case class

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14116 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14943 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14943 **[Test build #64881 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64881/consoleFull)** for PR 14943 at commit

[GitHub] spark issue #14116: [SPARK-16452][SQL] Support basic INFORMATION_SCHEMA

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14116 **[Test build #64876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64876/consoleFull)** for PR 14116 at commit

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14426 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64875/ Test PASSed. ---

[GitHub] spark issue #14942: [SparkR][Minor] Fix docs for sparkR.session and count

2016-09-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14942 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #14942: [SparkR][Minor] Fix docs for sparkR.session and c...

2016-09-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14942#discussion_r77422506 --- Diff: R/pkg/R/sparkR.R --- @@ -314,8 +314,10 @@ sparkRHive.init <- function(jsc = NULL) { #' Get the existing SparkSession or initialize a

[GitHub] spark issue #14426: [SPARK-16475][SQL] Broadcast Hint for SQL Queries

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14426 **[Test build #64875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64875/consoleFull)** for PR 14426 at commit

[GitHub] spark pull request #14690: [SPARK-16980][SQL] Load only catalog table partit...

2016-09-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14690#discussion_r77422471 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala --- @@ -0,0 +1,102 @@ +/* + * Licensed to the

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64873/ Test PASSed. ---

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14623 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14623: [SPARK-17044][SQL] Make test files for window functions ...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14623 **[Test build #64873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64873/consoleFull)** for PR 14623 at commit

[GitHub] spark pull request #14867: [SPARK-17296][SQL] Simplify parser join processin...

2016-09-02 Thread srinathshankar
Github user srinathshankar commented on a diff in the pull request: https://github.com/apache/spark/pull/14867#discussion_r77421576 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala --- @@ -360,10 +360,28 @@ class PlanParserSuite

[GitHub] spark issue #14864: [SPARK-15453] [SQL] FileSourceScanExec to extract `outpu...

2016-09-02 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/14864 @cloud-fan : Sounds good to me. I tried doing that but got a `Task not serializable: java.io.NotSerializableException: org.apache.hadoop.fs.LocatedFileStatus`. This is because the new

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64874/ Test PASSed. ---

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14527 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14527: [SPARK-16938][SQL] `drop/dropDuplicate` should handle th...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14527 **[Test build #64874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64874/consoleFull)** for PR 14527 at commit

[GitHub] spark pull request #14517: [SPARK-16931][PYTHON] PySpark APIS for bucketBy a...

2016-09-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/14517#discussion_r77420949 --- Diff: python/pyspark/sql/readwriter.py --- @@ -692,8 +734,7 @@ def orc(self, path, mode=None, partitionBy=None, compression=None):

[GitHub] spark issue #14943: [SPARK-15891][yarn] Clean up some logging in the YARN AM...

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14943 **[Test build #64881 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64881/consoleFull)** for PR 14943 at commit

[GitHub] spark pull request #14943: [SPARK-15891][yarn] Clean up some logging in the ...

2016-09-02 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/14943 [SPARK-15891][yarn] Clean up some logging in the YARN AM. To make the log file more readable, rework some of the logging done by the AM: - log executor command / env just once, since

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64877/ Test PASSed. ---

[GitHub] spark issue #14931: [SPARK-17370] Shuffle service files not invalidated when...

2016-09-02 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14931 In standalone mode I was thinking of this case: Master calls removeWorker() -> sends ExecutorUpdated -> StandaloneAppClient -> StandaloneSchedulerBackend, which generates the SlaveLost message.

[GitHub] spark issue #11119: [SPARK-10780][ML] Add an initial model to kmeans

2016-09-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #64877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64877/consoleFull)** for PR 9 at commit

[GitHub] spark pull request #14867: [SPARK-17296][SQL] Simplify parser join processin...

2016-09-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14867#discussion_r77420330 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala --- @@ -360,10 +360,25 @@ class PlanParserSuite extends

  1   2   3   4   5   >