[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1825#discussion_r15980176 --- Diff: bin/spark-shell --- @@ -32,12 +32,28 @@ set -o posix FWDIR="$(cd `dirname $0`/..; pwd)" if [[ "$@" = *--help ]] || [[ "$@" = *-h ]

[GitHub] spark pull request: [SPARK-1812] remove default args to overloaded...

2014-08-07 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1704#issuecomment-51569041 @avati, I think you might have misunderstood the compilation error. Default arguments are supported. Its okay if you can fix these in your PR soon. --- If your projec

[GitHub] spark pull request: [SPARK-1812] remove default args to overloaded...

2014-08-07 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1704#discussion_r15979913 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -85,15 +85,28 @@ private[spark] object JettyUtils extends Logging { pat

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1841 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/1841#discussion_r15979883 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1233,6 +1233,11 @@ abstract class RDD[T: ClassTag]( dependencies.head.rdd.asInstance

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1841#issuecomment-51568573 Merging in master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1841#issuecomment-51568405 QA results for PR 1841:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-1812] remove default args to overloaded...

2014-08-07 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/1704#discussion_r15979804 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala --- @@ -85,22 +85,46 @@ class StreamingContext private[streaming] (

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1825#issuecomment-51568034 QA results for PR 1825:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds the following public classes (experimental):$FWDIR/bin/spark-submit --c

[GitHub] spark pull request: [HOTFIX][STREAMING] Allow the JVM/Netty to dec...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1820#issuecomment-51568020 QA tests have started for PR 1820. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18196/consoleFull --- If

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-51567875 Failed in `Running PySpark tests` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [HOTFIX][STREAMING] Allow the JVM/Netty to dec...

2014-08-07 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/1820#issuecomment-51567907 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51567887 Sorry, I mean add the field `file`, see `'file:/tmp/sparkHiveWarehouse1280221975983654134/tmp_showcrt1'` --- If your project is set up for it, you can reply to th

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51567841 @chenghao-intel I did not understand what‘s your mean about "add the file" --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-51567758 QA results for PR 1846:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds the following public classes (experimental):case class CreateTableAsSel

[GitHub] spark pull request: [SPARK-2169] Don't copy appName / basePath eve...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1252#issuecomment-51567603 QA results for PR 1252:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51567507 I saw that, in order not to modify the `TestHive.scala`, properly we can also add the `file`. PS: From the golden file ``` 'org.apache.hadoop.hive.

[GitHub] spark pull request: [SPARK-2849 / 2914] Handle certain Spark confi...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1845#issuecomment-51567374 Tests sound like a great idea. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51567277 @chenghao-intel , I had added a line “last_modified_by” in nonDeterministicLineIndicators --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/1760#discussion_r15979418 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TestHive.scala --- @@ -95,7 +101,9 @@ class TestHiveContext(sc: SparkContext) extends HiveContext(

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51567155 @tianyi sorry, I couldn't open the last failure log of the unit test, if it failed in the dynamic value comparison, you can add the property in the [nonDeterminist

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/1760#discussion_r15979361 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TestHive.scala --- @@ -95,7 +101,9 @@ class TestHiveContext(sc: SparkContext) extends HiveContext(

[GitHub] spark pull request: SPARK-2878 Use Spark's classloader when instan...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1850#issuecomment-51566941 QA tests have started for PR 1850. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18195/consoleFull --- If

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on a diff in the pull request: https://github.com/apache/spark/pull/1760#discussion_r15979301 --- Diff: sql/hive/pom.xml --- @@ -139,6 +139,11 @@ org.scalatest scalatest-maven-plugin + --- End diff --

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/1760#discussion_r15979304 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TestHive.scala --- @@ -95,7 +101,9 @@ class TestHiveContext(sc: SparkContext) extends Hive

[GitHub] spark pull request: [SPARK-2923][MLLIB] Implement some basic BLAS ...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1849#issuecomment-51566676 QA tests have started for PR 1849. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18194/consoleFull --- If

[GitHub] spark pull request: SPARK-2878 Use Spark's classloader when

2014-08-07 Thread ash211
GitHub user ash211 opened a pull request: https://github.com/apache/spark/pull/1850 SPARK-2878 Use Spark's classloader when This is more of a test change than one that'll actually fix the problem. The solution might require putting the application jars and even the repl URL

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/1760#discussion_r15979199 --- Diff: sql/hive/pom.xml --- @@ -139,6 +139,11 @@ org.scalatest scalatest-maven-plugin + --- End

[GitHub] spark pull request: [SPARK-2877] [SQL] MetastoreRelation should us...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1806#issuecomment-51566416 QA tests have started for PR 1806. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18193/consoleFull --- If

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-51566443 QA results for PR 1513:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2923][MLLIB] Implement some basic BLAS ...

2014-08-07 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/1849 [SPARK-2923][MLLIB] Implement some basic BLAS routines Having some basic BLAS operations implemented in MLlib can help simplify the current implementation and improve some performance. Teste

[GitHub] spark pull request: [SPARK-2919] [SQL] Basic support for analyze c...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1848#issuecomment-51566195 QA results for PR 1848:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2877] [SQL] MetastoreRelation should us...

2014-08-07 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/1806#issuecomment-51566138 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1841#issuecomment-51565767 QA tests have started for PR 1841. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18192/consoleFull --- If

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565580 > BTW not sure if you commented out the actual unit test, but nothing was run in Jenkins ... Yep, I commented that out so the cycle runs quickly. Once I'm done

[GitHub] spark pull request: [SPARK-2911]: provide rdd.parent[T](j) to obta...

2014-08-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1841#issuecomment-51565558 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP][SPARK-2406][SQL] Initial support for usi...

2014-08-07 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/1819#discussion_r15978931 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -32,6 +38,113 @@ private[hive] trait HiveStrategies { va

[GitHub] spark pull request: [SPARK-2817] [SQL] add "show create table" sup...

2014-08-07 Thread tianyi
Github user tianyi commented on the pull request: https://github.com/apache/spark/pull/1760#issuecomment-51565490 Hi Michael , could you review these codes again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565477 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18191/consoleFull) for PR 1816 at commit [`50dbca6`](50dbca6). --- If your p

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1825#issuecomment-51565466 QA tests have started for PR 1825. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18190/consoleFull --- If

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565482 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18191/consoleFull) for PR 1816 at commit [`50dbca6`](50dbca6): * This pat

[GitHub] spark pull request: [SPARK-2849 / 2914] Handle certain Spark confi...

2014-08-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1845#issuecomment-51565448 Given the complexity of the bash scripts, can we make sure we have some tests for this? --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-2169] Don't copy appName / basePath eve...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1252#issuecomment-51564836 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [WIP][SPARK-2406][SQL] Initial support for usi...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/1819#discussion_r15978899 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -32,6 +38,113 @@ private[hive] trait HiveStrategies {

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565282 BTW not sure if you commented out the actual unit test, but nothing was run in Jenkins ... --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-2169] Don't copy appName / basePath eve...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1252#issuecomment-51565043 QA tests have started for PR 1252. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18189/consoleFull --- If

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565042 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18188/consoleFull) for PR 1816 at commit [`3b9ca49`](3b9ca49): * This pat

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51565040 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18188/consoleFull) for PR 1816 at commit [`3b9ca49`](3b9ca49). --- If your p

[GitHub] spark pull request: [SPARK-2862] Use shorthand range notation to a...

2014-08-07 Thread nrchandan
Github user nrchandan commented on the pull request: https://github.com/apache/spark/pull/1787#issuecomment-51564667 Sorry couldn't look into this yesterday. I plan to fix this today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-2169] Don't copy appName / basePath eve...

2014-08-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1252#issuecomment-51564568 Hey @andrewor14 - can you take a look at this one? seems pretty straightforward, though I'm not super familiar with this code. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51564445 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18187/consoleFull) for PR 1816 at commit [`84dfc5a`](84dfc5a): * This pat

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-5156 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18187/consoleFull) for PR 1816 at commit [`84dfc5a`](84dfc5a). --- If your p

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-51564439 QA tests have started for PR 1846. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18186/consoleFull --- If

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-51564211 QA tests have started for PR 1513. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18185/consoleFull --- If

[GitHub] spark pull request: [SPARK-2917] [SQL] Avoid table creation in log...

2014-08-07 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1846#issuecomment-51564189 Hello, Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1813#issuecomment-51564110 Hey so after thinking about it a bit more, maybe it's worth experimenting with this simpler approach of in-lining the shaded dependency in our jars using build plug-ins.

[GitHub] spark pull request: [SPARK-2608] fix executor backend launch commo...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1513#issuecomment-51563959 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-2848] Shade Guava in uber-jars.

2014-08-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1813#discussion_r15978313 --- Diff: project/Relocator.scala --- @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51563593 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18184/consoleFull) for PR 1816 at commit [`6ffb1f5`](6ffb1f5). --- If your p

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51563594 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18184/consoleFull) for PR 1816 at commit [`6ffb1f5`](6ffb1f5): * This pat

[GitHub] spark pull request: [SPARK-2913] Place our log4j.properties at the...

2014-08-07 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/1844#issuecomment-51563228 One thing thats good to keep in mind that often applications written on top of Spark may also have their own log4j.properties and core-site.xml etc. So putting Spark's c

[GitHub] spark pull request: [SPARK-2406][SQL] Initial support for using Pa...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1819#issuecomment-51562953 QA results for PR 1819:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds the following public classes (experimental):case class OutputFaker(outp

[GitHub] spark pull request: [SPARK-2919] [SQL] Basic support for analyze c...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1848#issuecomment-51562943 QA tests have started for PR 1848. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18183/consoleFull --- If

[GitHub] spark pull request: [SPARK-2913] Place our log4j.properties at the...

2014-08-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1844#issuecomment-51562742 I think it's reasonable to handle this specially in spark itself - @aarondav what do you think? --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-2919] [SQL] Basic support for analyze c...

2014-08-07 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/1848 [SPARK-2919] [SQL] Basic support for analyze command in HiveQl The command we will support is ``` ANALYZE TABLE tablename COMPUTE STATISTICS noscan ``` Other cases shown in https://cwi

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977882 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2678][Core] Backport PR #1801 to branch...

2014-08-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1831#issuecomment-51562641 @liancheng can you close this? I've merged it but things don't close properly in this case. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark pull request: [SPARK-2897][SPARK-2920]TorrentBroadcast does ...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1836#issuecomment-51562551 QA results for PR 1836:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977787 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977782 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977737 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977683 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977646 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977620 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1816#discussion_r15977569 --- Diff: dev/run-tests-jenkins --- @@ -24,62 +24,110 @@ FWDIR="$(cd `dirname $0`/..; pwd)" cd "$FWDIR" +function get_jq () { + # Get

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51561557 Oh man - this is epic! @nchammas for the various environment variables, we get them from the github pull request builder. The relevant source code is here (btw - it woul

[GitHub] spark pull request: [SPARK-2918] [SQL] [WIP] Support the extended ...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1847#issuecomment-51561446 QA results for PR 1847:- This patch FAILED unit tests.- This patch merges cleanly- This patch adds the following public classes (experimental):case class ExplainCommand(p

[GitHub] spark pull request: [SPARK-2897][SPARK-2920]TorrentBroadcast does ...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1836#issuecomment-51560343 QA tests have started for PR 1836. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18180/consoleFull --- If

[GitHub] spark pull request: [SPARK-2406][SQL] Initial support for using Pa...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1819#issuecomment-51559638 QA tests have started for PR 1819. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18179/consoleFull --- If

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread sarutak
Github user sarutak commented on a diff in the pull request: https://github.com/apache/spark/pull/1825#discussion_r15976793 --- Diff: bin/spark-shell --- @@ -46,14 +48,38 @@ function main(){ # (see https://github.com/sbt/sbt/issues/562). stty -icanon min 1

[GitHub] spark pull request: [Web UI]Make decision order of Worker's WebUI ...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1838#issuecomment-51558687 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [Jenkins] Fix Jenkins not posting to Github so...

2014-08-07 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/1809#issuecomment-51558626 I think I've got it. [Check it out.](https://github.com/apache/spark/pull/1816#issuecomment-51558443) I also have [the typo](https://github.com/apache/spark/pul

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51558440 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18178/consoleFull) for PR 1816 at commit [`642fbf5`](642fbf5). --- If your p

[GitHub] spark pull request: [SPARK-2912] [Spark QA] [WIP] Include commit h...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1816#issuecomment-51558443 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18178/consoleFull) for PR 1816 at commit [`642fbf5`](642fbf5): * This pat

[GitHub] spark pull request: [SPARK-2849 / 2914] Handle certain Spark confi...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1845#issuecomment-51558467 QA results for PR 1845:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [SPARK-2918] [SQL] [WIP] Support the extended ...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1847#issuecomment-51558449 QA tests have started for PR 1847. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18177/consoleFull --- If

[GitHub] spark pull request: [SPARK-2678][Core][SQL] A workaround for SPARK...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1801#issuecomment-51558315 Yeah, I agree that this break in backward compatibility is pretty bad. At the same time we don't want to introduce some new config (e.g. the proposals in #1715) until

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1825#issuecomment-51558310 @liancheng `ipython` has a ton of command line options that someone might hypothetically want to use (just take a look at `ipython --help`). Maybe try things out with

[GitHub] spark pull request: [SPARK-2918] [SQL] [WIP] Support the extended ...

2014-08-07 Thread chenghao-intel
GitHub user chenghao-intel opened a pull request: https://github.com/apache/spark/pull/1847 [SPARK-2918] [SQL] [WIP] Support the extended & native command for EXPLAIN Currently, EXPLAIN doesn't support the SQL native command, or printing the logical plan. This PR will solve this.

[GitHub] spark pull request: [Jenkins] Fix Jenkins not posting to Github so...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1809#issuecomment-51558078 Let me know once you fix it so I can close this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1825#issuecomment-51557678 @JoshRosen I'm checking `pyspark`, but I'm not very familiar with this. Would you mind to help confirming are there any other Python/IPython/PySpark specific command li

[GitHub] spark pull request: [Jenkins] Fix Jenkins not posting to Github so...

2014-08-07 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/1809#issuecomment-51557404 Yeah, Jenkins and I are good buddies now. Heh. I'll take a crack at fixing this in my PR in parallel. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-2678][Core][SQL] A workaround for SPARK...

2014-08-07 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/1801#issuecomment-51557390 @vanzin I understand what you're worrying about :) Actually in #1715 we've roughly reached a cleaner solution (see the commends) than this one, but some details haven't

[GitHub] spark pull request: [Jenkins] Fix Jenkins not posting to Github so...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1809#issuecomment-51557160 Oh interesting, I didn't realize there was a related PR. Try adding a default parameter to your class: ``` class DeleteMe(name: String = "I have quotes in me")

[GitHub] spark pull request: [SPARK-2849] bin/spark-submit should respect s...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1770#issuecomment-51557059 QA results for PR 1770:- This patch PASSES unit tests.- This patch merges cleanly- This patch adds no public classesFor more information see test ouptut:https://amplab.c

[GitHub] spark pull request: [Jenkins] Fix Jenkins not posting to Github so...

2014-08-07 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/1809#issuecomment-51557025 Andrew, could you post an example of a class definition that causes this? I'm working on [a related PR](https://github.com/apache/spark/pull/1816), so I'm curious.

[GitHub] spark pull request: Clean up and simplify Spark configuration

2014-08-07 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/299#discussion_r15976048 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -64,9 +64,10 @@ private[spark] class Executor( // to what Yarn on this sy

[GitHub] spark pull request: [SPARK-2886] Use more specific actor system na...

2014-08-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1810#issuecomment-51557011 QA tests have started for PR 1810. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18175/consoleFull --- If

[GitHub] spark pull request: [SPARK-2886] Use more specific actor system na...

2014-08-07 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1810#issuecomment-51556854 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-2894] spark-shell doesn't accept flags

2014-08-07 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/1825#discussion_r15975980 --- Diff: bin/spark-shell --- @@ -46,14 +48,38 @@ function main(){ # (see https://github.com/sbt/sbt/issues/562). stty -icanon min

  1   2   3   4   >