[GitHub] spark pull request: [SPARK-1507][YARN]specify num of cores for AM

2014-12-21 Thread XuTingjun
Github user XuTingjun commented on the pull request: https://github.com/apache/spark/pull/3686#issuecomment-67810373 sorry, I think this patch works in yarn-client and yarn-cluster mode. The param "--driver-cores" is standalone cluster only. am I missing something? --- If your proje

[GitHub] spark pull request: [SPARK-4493][SQL] Tests for IsNull / IsNotNull...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3748#issuecomment-67808668 @marmbrus This is ready to go. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67807970 [Test build #24698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24698/consoleFull) for PR 3555 at commit [`efc4210`](https://gith

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67807971 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-2883][SQL] Orc support through datasour...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3753#issuecomment-67807238 We are planning to add first class support for partitioned tables in the external data source API in 1.3. Some interface like `PartitionedRelation` will be provided to

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3740#issuecomment-67807114 Thanks - I will merge the new PR once tests pass. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67806819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67806817 [Test build #24697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24697/consoleFull) for PR 3555 at commit [`bd2c444`](https://gith

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67806439 LGTM, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-4636] [SQL] WIP: Cluster By & Distribut...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3496#issuecomment-67806310 Also it would be good to add a comment to explain the reason why we choose a suboptimal plan here is because we tend to respect Hive's semantics. Otherwise future devel

[GitHub] spark pull request: [SPARK-4871][SQL] Show sql statement in spark ...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3718#issuecomment-67806102 @marmbrus Maybe I missed something here, but why do we prefer `setJobDescription` over `setJobGroup`, especially when the former is deprecated? @scwf For job c

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3740#issuecomment-67805801 See #3758 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-2075][Core] backport for branch-1.2

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3758#issuecomment-67805807 [Test build #24699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24699/consoleFull) for PR 3758 at commit [`b57d440`](https://githu

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67805806 [Test build #24698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24698/consoleFull) for PR 3555 at commit [`efc4210`](https://githu

[GitHub] spark pull request: [SPARK-2075][Core] backport for branch-1.2

2014-12-21 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/3758 [SPARK-2075][Core] backport for branch-1.2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/zsxwing/spark SPARK-2075-branch-1.2 Alternatively yo

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2542#issuecomment-67805590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2542#issuecomment-67805585 [Test build #24696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24696/consoleFull) for PR 2542 at commit [`b708fc7`](https://gith

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3740#issuecomment-67805360 `RDD.rddToPairRDDFunctions` needs to be changed to `rddToPairRDDFunctions` for branch-1.2 --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/3740#issuecomment-67805327 Need to backport this one to branch-1.2, as the implicit APIs fixes do not exist in branch-1.2 --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3740 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2075][Core] Make the compiler generate ...

2014-12-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/3740#issuecomment-67805239 Thanks - merging in master & branch-1.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67804795 This LGTM except for a minor styling issue. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If y

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3555#discussion_r22155207 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala --- @@ -56,6 +56,14 @@ class HiveQuerySuite extends HiveComparis

[GitHub] spark pull request: [SPARK-4915] Fix classname to be specified for...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3757#issuecomment-67804280 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-4915] Fix classname to be specified for...

2014-12-21 Thread oza
Github user oza commented on the pull request: https://github.com/apache/spark/pull/3757#issuecomment-67804178 @andrewor14 could you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-4915] Fix classname to be specified for...

2014-12-21 Thread oza
GitHub user oza opened a pull request: https://github.com/apache/spark/pull/3757 [SPARK-4915] Fix classname to be specified for external shuffle service. You can merge this pull request into a Git repository by running: $ git pull https://github.com/oza/spark SPARK-4915 Alter

[GitHub] spark pull request: [SPARK-3431] [WIP] Parallelize Scala/Java test...

2014-12-21 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3564#issuecomment-67804058 By the way, I opened [a question on Stack Overflow](http://stackoverflow.com/q/27588350/877069) about some kind of "show execution plan" feature in sbt. It would make un

[GitHub] spark pull request: [SPARK-4692] [SQL] Support ! boolean logic ope...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3555#issuecomment-67803723 [Test build #24697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24697/consoleFull) for PR 3555 at commit [`bd2c444`](https://githu

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2542#issuecomment-67802692 [Test build #24696 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24696/consoleFull) for PR 2542 at commit [`b708fc7`](https://githu

[GitHub] spark pull request: [SPARK-3688][SQL]LogicalPlan can't resolve col...

2014-12-21 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2542#issuecomment-67802647 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-67800596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-67800592 [Test build #24694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24694/consoleFull) for PR 3222 at commit [`f327b66`](https://gith

[GitHub] spark pull request: [SPARK-3083] fix unnecessarily removing sendin...

2014-12-21 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/1985 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-2491] Don't handle uncaught exceptions ...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1482#issuecomment-67799172 [Test build #24695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24695/consoleFull) for PR 1482 at commit [`cfe98b8`](https://gith

[GitHub] spark pull request: [SPARK-2491] Don't handle uncaught exceptions ...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1482#issuecomment-67799173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-2491] Don't handle uncaught exceptions ...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1482#issuecomment-67799100 [Test build #24695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24695/consoleFull) for PR 1482 at commit [`cfe98b8`](https://githu

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67798870 [Test build #24693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24693/consoleFull) for PR 3717 at commit [`ccd30d7`](https://gith

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67798872 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67798662 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67798659 [Test build #24692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24692/consoleFull) for PR 3717 at commit [`330f70c`](https://gith

[GitHub] spark pull request: [SPARK-4161]Spark shell class path is not corr...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3051#issuecomment-67798093 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4161]Spark shell class path is not corr...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3051#issuecomment-67798089 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24691/consoleFull) for PR 3051 at commit [`5d1edf1`](https://github.com/a

[GitHub] spark pull request: [SPARK-4871][SQL] Show sql statement in spark ...

2014-12-21 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/3718#issuecomment-67797406 Ok, i will add a pop-up containing the full --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [WIP][SPARK-4251][SPARK-2352][MLLIB]Add RBM, A...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3222#issuecomment-67797217 [Test build #24694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24694/consoleFull) for PR 3222 at commit [`f327b66`](https://githu

[GitHub] spark pull request: [SPARK-4813][Streaming] Fix the issue that Con...

2014-12-21 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/3661#discussion_r22152847 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ContextWaiter.scala --- @@ -17,30 +17,63 @@ package org.apache.spark.streaming

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67795738 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67795733 [Test build #24690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24690/consoleFull) for PR 3386 at commit [`38c78cc`](https://gith

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67795433 Looks like the test failed not because of this PR. Please test again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67795286 [Test build #24693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24693/consoleFull) for PR 3717 at commit [`ccd30d7`](https://githu

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3717#issuecomment-67795082 [Test build #24692 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24692/consoleFull) for PR 3717 at commit [`330f70c`](https://githu

[GitHub] spark pull request: [SPARK-4870] Add spark version to driver log

2014-12-21 Thread liyezhang556520
Github user liyezhang556520 commented on a diff in the pull request: https://github.com/apache/spark/pull/3717#discussion_r22152604 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -240,6 +240,8 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with L

[GitHub] spark pull request: [SPARK-4161]Spark shell class path is not corr...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3051#issuecomment-67794885 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24691/consoleFull) for PR 3051 at commit [`5d1edf1`](https://github.com/ap

[GitHub] spark pull request: [SPARK-4636] [SQL] WIP: Cluster By & Distribut...

2014-12-21 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/3496#issuecomment-67793074 @marmbrus , you're right, I will update the code after #3386 be merged. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: [Spark-4512] [SQL] Unresolved Attribute Except...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3386#issuecomment-67792894 [Test build #24690 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24690/consoleFull) for PR 3386 at commit [`38c78cc`](https://githu

[GitHub] spark pull request: [SPARK-3760] [STREAMING] Add Twitter4j FilterQ...

2014-12-21 Thread ezhulenev
Github user ezhulenev closed the pull request at: https://github.com/apache/spark/pull/2618 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-21 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r22151650 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -182,34 +206,43 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-21 Thread Lewuathe
Github user Lewuathe commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r22151644 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -77,6 +80,17 @@ class GradientDescent private[mllib] (privat

[GitHub] spark pull request: [SPARK-4912][SQL] Persistent tables for the Sp...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3752#issuecomment-67790096 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4912][SQL] Persistent tables for the Sp...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3752#issuecomment-67790092 [Test build #24689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24689/consoleFull) for PR 3752 at commit [`1002d20`](https://gith

[GitHub] spark pull request: [SPARK-3431] [WIP] Parallelize Scala/Java test...

2014-12-21 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3564#issuecomment-67788245 Hmm, taking a second look at [how the Python tests are invoked](https://github.com/apache/spark/blob/c6a3c0d5052e5bf6f981e5f91e05cba38b707237/python/run-tests#L38), I wo

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67788102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67788098 [Test build #24688 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24688/consoleFull) for PR 3751 at commit [`03fddaa`](https://gith

[GitHub] spark pull request: [SPARK-4912][SQL] Persistent tables for the Sp...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3752#issuecomment-67787708 [Test build #24689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24689/consoleFull) for PR 3752 at commit [`1002d20`](https://githu

[GitHub] spark pull request: [SPARK-3431] [WIP] Parallelize test execution

2014-12-21 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3564#issuecomment-67786877 Yeah, I looked briefly at ways of parallelizing the Python tests, because those take around 10-12 minutes in total, which will become a significant fraction of the total

[GitHub] spark pull request: [SPARK-4813][Streaming] Fix the issue that Con...

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3661#discussion_r22150536 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ContextWaiter.scala --- @@ -17,30 +17,63 @@ package org.apache.spark.streaming

[GitHub] spark pull request: [SPARK-3431] [WIP] Parallelize test execution

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3564#issuecomment-67785676 Hmm, looks like we're only down to a few test failures and they tend to be the same failures across runs (plus some known flaky tests). Once we finish the streaming te

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67785433 [Test build #24688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24688/consoleFull) for PR 3751 at commit [`03fddaa`](https://githu

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67785301 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67785296 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-67785117 @vanzin It didn't break the pull request builder, but it did cause the Hadoop 1.x master builds to fail. There's a fix in #3754 which I've merged, so this should be fi

[GitHub] spark pull request: SPARK-4910 [CORE] build failed (use of FileSta...

2014-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3754 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-4910 [CORE] build failed (use of FileSta...

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3754#issuecomment-67785054 According to [HADOOP-6585](https://issues.apache.org/jira/browse/HADOOP-6585), the old behavior / usage was that `!isDir` implies `isFile`. Symlinks broke this, but w

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67783732 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [Minor] Fix scala doc

2014-12-21 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/3751#issuecomment-67783704 @srowen Are you able to trigger Jenkins builds for PRs from non-whitelisted authors? --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: SPARK-4910 [CORE] build failed (use of FileSta...

2014-12-21 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3754#issuecomment-67783119 @JoshRosen Yes I agree, `isDir` is not quite the same as `!isFile` but it's the closest thing available in 1.x. I'll fix the message above, oops. --- If your project is s

[GitHub] spark pull request: SPARK-4910 [CORE] build failed (use of FileSta...

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3754#issuecomment-67782883 This seems fine to me, especially since this only affects test code. I guess that newer versions of Hadoop have `isSymlink`, so it may not always be the case that `!is

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67781953 /cc @vanzin @andrewor14 (#1222) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67781504 [Test build #24686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24686/consoleFull) for PR 3756 at commit [`e2bd21d`](https://gith

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67781505 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-67779856 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-67779852 [Test build #24685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24685/consoleFull) for PR 3249 at commit [`11a238d`](https://gith

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67779795 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67779791 [Test build #24687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24687/consoleFull) for PR 3755 at commit [`b5730a1`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-67779527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-67779525 [Test build #24684 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24684/consoleFull) for PR 3249 at commit [`8cae35b`](https://gith

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67778692 [Test build #24687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24687/consoleFull) for PR 3755 at commit [`b5730a1`](https://githu

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67778691 [Test build #24686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24686/consoleFull) for PR 3756 at commit [`e2bd21d`](https://githu

[GitHub] spark pull request: [SPARK-4913] Fix incorrect event log path

2014-12-21 Thread ash211
Github user ash211 commented on the pull request: https://github.com/apache/spark/pull/3755#issuecomment-67778586 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/3756#discussion_r22149121 --- Diff: dev/run-tests --- @@ -141,20 +141,22 @@ CURRENT_BLOCK=$BLOCK_BUILD { # NOTE: echo "q" is needed because sbt on encountering a bui

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67778325 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4914][Build] Cleans lib_managed before ...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3756#issuecomment-67778323 [Test build #24683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24683/consoleFull) for PR 3756 at commit [`c9f2f3e`](https://gith

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-67778115 Thank you for reviewing it. I have worked on review comments.Please review it. I guess the ```SubqueryExpression``` may not be resolved along with main query

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22149004 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -314,6 +318,113 @@ class Analyzer(catalog: Catalog, reg

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22148917 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SubqueryExpression.scala --- @@ -0,0 +1,40 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22148913 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SubqueryExpression.scala --- @@ -0,0 +1,40 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22148903 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SubqueryExpression.scala --- @@ -0,0 +1,40 @@ +/* + * Licensed to

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3249#issuecomment-6608 [Test build #24685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24685/consoleFull) for PR 3249 at commit [`11a238d`](https://githu

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22148896 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -314,6 +318,113 @@ class Analyzer(catalog: Catalog, reg

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r2214 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -314,6 +318,113 @@ class Analyzer(catalog: Catalog, reg

[GitHub] spark pull request: [SPARK-4226][SQL] SparkSQL - Add support for s...

2014-12-21 Thread ravipesala
Github user ravipesala commented on a diff in the pull request: https://github.com/apache/spark/pull/3249#discussion_r22148859 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -314,6 +318,113 @@ class Analyzer(catalog: Catalog, reg

  1   2   >