[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13061 > Aren't we already doing that No, you're throwing a generic exception that doesn't explain what's wrong. None of the existing errors say "properties file not found", which is what a null wo

[GitHub] spark pull request #13499: [SPARK-15754] [yarn] Not letting the credentials ...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13499#discussion_r65764703 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala --- @@ -64,7 +68,7 @@ private[spark] class YarnClientSchedu

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/13061 Aren't we already doing that. The generic catch would display the error message and also the exception that caused it. Having an extra check isn't going to add more value here. --- If your project

[GitHub] spark issue #13499: [SPARK-15754] [yarn] Not letting the credentials contain...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13499 **[Test build #59955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59955/consoleFull)** for PR 13499 at commit [`d53ad55`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #13115: [SPARK-12492] Using spark-sql commond to run query, writ...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13115 **[Test build #3052 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3052/consoleFull)** for PR 13115 at commit [`e756a0a`](https://github.com/apache/spark/commit

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13476 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13476 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59952/ Test PASSed. ---

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13476 **[Test build #59952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59952/consoleFull)** for PR 13476 at commit [`d256621`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13499: [SPARK-15754] [yarn] Not letting the credentials ...

2016-06-03 Thread subrotosanyal
Github user subrotosanyal commented on a diff in the pull request: https://github.com/apache/spark/pull/13499#discussion_r65762991 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala --- @@ -64,7 +68,7 @@ private[spark] class YarnClien

[GitHub] spark issue #13490: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13490 **[Test build #59954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59954/consoleFull)** for PR 13490 at commit [`1e3fd2e`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #13490: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-03 Thread andrewor14
Github user andrewor14 commented on the issue: https://github.com/apache/spark/pull/13490 Sounds good. I've updated the patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13498 Seems to be related to the sequence in which tests are executed... Perhaps `src` is changed during tests. --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13061 > Feel that it would be better to error out rather than running an unknown version I meant a null check so you could throw a proper exception instead of a generic `NullPointerException`. -

[GitHub] spark issue #13481: [SPARK-15738][PYSPARK][ML] Adding Pyspark ml RFormula __...

2016-06-03 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/13481 Yeah actually looking at it the Scala side is a bit ugly. We can make it prettier there and then just call into it from PySpark, e.g. ``` scala> val formula = new RFormula().setFormula("c

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3058 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3058/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3057 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3057/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #59953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59953/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3056 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3056/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3054 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3054/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3055 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3055/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3053 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3053/consoleFull)** for PR 13498 at commit [`06d5b92`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13418: [SPARK-15677][SQL] Query with scalar sub-query in the SE...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13418 thanks, merging to master and 2.0! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/13061 Fixed the minor nits. Feel that it would be better to error out rather than running an unknown version of the framework, so haven't included the null check for ```resourceStream```. --- If your pro

[GitHub] spark pull request #13418: [SPARK-15677][SQL] Query with scalar sub-query in...

2016-06-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13418 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13476 **[Test build #59952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59952/consoleFull)** for PR 13476 at commit [`d256621`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #12313: [SPARK-14543] [SQL] Improve InsertIntoTable colum...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12313#discussion_r65759591 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -284,8 +284,128 @@ class InsertIntoHiveTableSuite extend

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13446 can you also update your title to say that this PR only adds regression test? thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request #12313: [SPARK-14543] [SQL] Improve InsertIntoTable colum...

2016-06-03 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/12313#discussion_r65758947 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -284,8 +284,128 @@ class InsertIntoHiveTableSuite extends Q

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13494 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59940/ Test PASSed. ---

[GitHub] spark pull request #12313: [SPARK-14543] [SQL] Improve InsertIntoTable colum...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12313#discussion_r65758393 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -284,8 +284,128 @@ class InsertIntoHiveTableSuite extend

[GitHub] spark issue #13494: [SPARK-15752] [SQL] support optimization for metadata on...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13494 **[Test build #59940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59940/consoleFull)** for PR 13494 at commit [`153293e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3050/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3051/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13476 **[Test build #59948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59948/consoleFull)** for PR 13476 at commit [`b399878`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59948/ Test FAILed. ---

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13476 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13492: [SPARK-15749][SQL]make the error message more meaningful

2016-06-03 Thread huaxingao
Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/13492 retest please. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #13484: [SPARK-15742][SQL] Reduce temp collections allocations i...

2016-06-03 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13484 LGTM too, probably this will be even more of a improvement once we get the other bottlenecks fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark issue #13318: [SPARK-15391] [SQL] manage the temporary memory of timso...

2016-06-03 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/13318 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #13318: [SPARK-15391] [SQL] manage the temporary memory o...

2016-06-03 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/13318#discussion_r65757237 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -221,7 +221,8 @@ public BytesToBytesMap( SparkEnv.get()

[GitHub] spark issue #13490: [SPARK-15722][SQL] Disallow specifying schema in CTAS st...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13490 I think this should also throw exception ``` CREATE TABLE students (age INT, name STRING) AS SELECT 1, 'hello' ``` A `SELECT` also defines both the name and data type of its ou

[GitHub] spark pull request #12313: [SPARK-14543] [SQL] Improve InsertIntoTable colum...

2016-06-03 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/12313#discussion_r65757163 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -284,8 +284,128 @@ class InsertIntoHiveTableSuite extends Q

[GitHub] spark issue #13491: [SPARK-15748][SQL] Replace inefficient foldLeft() call w...

2016-06-03 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13491 LGTM provided tests still pass --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59939/ Test PASSed. ---

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13446 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13500: [SPARK-15756] [SQL] Support create table stored as orcfi...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13500 **[Test build #59951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59951/consoleFull)** for PR 13500 at commit [`518bdd5`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13446 **[Test build #59939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59939/consoleFull)** for PR 13446 at commit [`e892324`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13418: [SPARK-15677][SQL] Query with scalar sub-query in the SE...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13418 **[Test build #3045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3045/consoleFull)** for PR 13418 at commit [`77166bc`](https://github.com/apache/spark/commit

[GitHub] spark issue #13070: [SPARK-15286] [SQL] Make the output readable for EXPLAIN...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13070 **[Test build #59950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59950/consoleFull)** for PR 13070 at commit [`0cc3268`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #13070: [SPARK-15286] [SQL] Make the output readable for EXPLAIN...

2016-06-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13070 @cloud-fan Thank you for your review! Just quickly added the space. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark issue #13318: [SPARK-15391] [SQL] manage the temporary memory of timso...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13318 **[Test build #59949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59949/consoleFull)** for PR 13318 at commit [`a929a06`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #13318: [SPARK-15391] [SQL] manage the temporary memory o...

2016-06-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13318#discussion_r65754375 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -221,7 +221,8 @@ public BytesToBytesMap( SparkEnv.get() != nul

[GitHub] spark issue #13269: [SPARK-15494][SQL] encoder code cleanup

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13269 yea, but may not happen before 2.0. It needs some more refactor to the object operator execution model. For example, the serializer in `AppendColumns` has no corresponding attribute, its input is

[GitHub] spark pull request #13318: [SPARK-15391] [SQL] manage the temporary memory o...

2016-06-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13318#discussion_r65753944 --- Diff: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java --- @@ -221,7 +221,8 @@ public BytesToBytesMap( SparkEnv.get() != nul

[GitHub] spark pull request #13318: [SPARK-15391] [SQL] manage the temporary memory o...

2016-06-03 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13318#discussion_r65753668 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleInMemorySorter.java --- @@ -70,9 +70,14 @@ public int compare(PackedRecordPointer left, Pac

[GitHub] spark issue #13501: [SPARK-15759] [SQL] Fallback to non-codegen when fail to...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13501 **[Test build #59947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59947/consoleFull)** for PR 13501 at commit [`550e386`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #13476: [SPARK-15684][SparkR]Not mask startsWith and endsWith in...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13476 **[Test build #59948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59948/consoleFull)** for PR 13476 at commit [`b399878`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #13501: [SPARK-15759] [SQL] Fallback to non-codegen when ...

2016-06-03 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/13501 [SPARK-15759] [SQL] Fallback to non-codegen when fail to compile generated code ## What changes were proposed in this pull request? In case of any bugs in whole-stage codegen, the generated

[GitHub] spark pull request #13484: [SPARK-15742][SQL] Reduce temp collections alloca...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/13484#discussion_r65752894 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -169,11 +170,24 @@ abstract class TreeNode[BaseType <: Tr

[GitHub] spark issue #13484: [SPARK-15742][SQL] Reduce temp collections allocations i...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13484 **[Test build #59946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59946/consoleFull)** for PR 13484 at commit [`b3e63bb`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #13500: [SPARK-15756] [SQL] Support create table stored as orcfi...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13500 **[Test build #59945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59945/consoleFull)** for PR 13500 at commit [`4550f41`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #13500: [SPARK-15756] [SQL] Support create table stored a...

2016-06-03 Thread lianhuiwang
GitHub user lianhuiwang opened a pull request: https://github.com/apache/spark/pull/13500 [SPARK-15756] [SQL] Support create table stored as orcfile/parquetfile/avrofile ## What changes were proposed in this pull request? in https://github.com/apache/hive/blob/master/ql/src/jav

[GitHub] spark pull request #13499: [SPARK-15754] [yarn] Not letting the credentials ...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13499#discussion_r65752573 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala --- @@ -64,7 +68,7 @@ private[spark] class YarnClientSchedu

[GitHub] spark issue #13115: [SPARK-12492] Using spark-sql commond to run query, writ...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13115 **[Test build #3052 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3052/consoleFull)** for PR 13115 at commit [`e756a0a`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #13499: [SPARK-15754] [yarn] Not letting the credentials ...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13499#discussion_r65752132 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala --- @@ -21,9 +21,13 @@ import scala.collection.mutable.Arra

[GitHub] spark issue #13469: [SPARK-15140][SQL] make the semantics of null input obje...

2016-06-03 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13469 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13070: [SPARK-15286] [SQL] Make the output readable for EXPLAIN...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13070 LGTM, I'm wondering if it's slightly better if we always put a space after `:`(not a big deal) --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark issue #13269: [SPARK-15494][SQL] encoder code cleanup

2016-06-03 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13269 So there will be a follow-up for replacing `BoundReference` in serializer expressions with `GetColumnByOrdinal`, right? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #13469: [SPARK-15140][SQL] make the semantics of null inp...

2016-06-03 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/13469#discussion_r65751003 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -519,7 +519,7 @@ case class CreateExternalRow

[GitHub] spark issue #13474: [SPARK-15547][SQL] nested case class in encoder can have...

2016-06-03 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/13474 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #13061: [SPARK-14279] [Build] Pick the spark version from...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13061#discussion_r65750838 --- Diff: core/src/main/scala/org/apache/spark/package.scala --- @@ -41,7 +41,53 @@ package org.apache * level interfaces. These are subject to changes o

[GitHub] spark pull request #13061: [SPARK-14279] [Build] Pick the spark version from...

2016-06-03 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/13061#discussion_r65750639 --- Diff: core/src/main/scala/org/apache/spark/package.scala --- @@ -41,7 +41,53 @@ package org.apache * level interfaces. These are subject to changes o

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3046/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13483: [SPARK-15688][SQL] RelationalGroupedDataset.toDF should ...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13483 I don't think it's a big problem. After users understand the semantic, they can easily work around it, by not putting grouping columns in agg list, or turning off the config. Well, it looks be

[GitHub] spark issue #8559: [SPARK-10372] [CORE] basic test framework for entire spar...

2016-06-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/8559 great thnx, it passes locally now. I will test my pr too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3048/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3047/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3044/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3049/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit

[GitHub] spark issue #13499: [SPARK-15754] [yarn] Not letting the credentials contain...

2016-06-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13499 BTW the change looks fine, I just want to take a look at the rest of the code (the parts you're not touching) and also run some tests internally with kerberos enabled. --- If your project is set up

[GitHub] spark pull request #13326: [SPARK-15560] [Mesos] Queued/Supervise drivers wa...

2016-06-03 Thread devaraj-kavali
Github user devaraj-kavali commented on a diff in the pull request: https://github.com/apache/spark/pull/13326#discussion_r65748851 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -188,10 +188,10 @@ private[spark] class Mes

[GitHub] spark pull request #12313: [SPARK-14543] [SQL] Improve InsertIntoTable colum...

2016-06-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12313#discussion_r65748370 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala --- @@ -284,8 +284,128 @@ class InsertIntoHiveTableSuite extend

[GitHub] spark pull request #13061: [SPARK-14279] [Build] Pick the spark version from...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13061#discussion_r65748365 --- Diff: core/src/main/scala/org/apache/spark/package.scala --- @@ -41,7 +41,53 @@ package org.apache * level interfaces. These are subject to changes o

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3051 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3051/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13446 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13446 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59936/ Test PASSed. ---

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13498 **[Test build #3050 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3050/consoleFull)** for PR 13498 at commit [`2949d70`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13446: [SPARK-15704] [SQL] fix a crash in TungstenAggregate whi...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13446 **[Test build #59936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59936/consoleFull)** for PR 13446 at commit [`3837825`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13061: [SPARK-14279] [Build] Pick the spark version from pom

2016-06-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/13061 Two nits that I missed before and this is good to go. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request #13061: [SPARK-14279] [Build] Pick the spark version from...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13061#discussion_r65747950 --- Diff: core/src/main/scala/org/apache/spark/package.scala --- @@ -41,7 +41,53 @@ package org.apache * level interfaces. These are subject to changes o

[GitHub] spark pull request #13061: [SPARK-14279] [Build] Pick the spark version from...

2016-06-03 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/13061#discussion_r65747983 --- Diff: core/src/main/scala/org/apache/spark/package.scala --- @@ -41,7 +41,53 @@ package org.apache * level interfaces. These are subject to changes o

[GitHub] spark issue #13474: [SPARK-15547][SQL] nested case class in encoder can have...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13474 **[Test build #59943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59943/consoleFull)** for PR 13474 at commit [`2e289c8`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #13401: [SPARK-15657][SQL] RowEncoder should validate the data t...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13401 **[Test build #59944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59944/consoleFull)** for PR 13401 at commit [`83fe3e0`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #13499: [SPARK-15754] [yarn] Not letting the credentials contain...

2016-06-03 Thread subrotosanyal
Github user subrotosanyal commented on the issue: https://github.com/apache/spark/pull/13499 @vanzin Updated the title. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #13491: [SPARK-15748][SQL] Replace inefficient foldLeft() call w...

2016-06-03 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/13491 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13498 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13491: [SPARK-15748][SQL] Replace inefficient foldLeft() call w...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13491 **[Test build #59942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59942/consoleFull)** for PR 13491 at commit [`50a6270`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #13498: [SPARK-15011][SQL] Re-enable 'analyze MetastoreRelations...

2016-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13498 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59937/ Test FAILed. ---

[GitHub] spark issue #13469: [SPARK-15140][SQL] make the semantics of null input obje...

2016-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13469 **[Test build #59941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59941/consoleFull)** for PR 13469 at commit [`6b2641f`](https://github.com/apache/spark/commit/6

<    1   2   3   4   5   6   >