[GitHub] spark pull request: [SPARK-5595][SPARK-5603][SQL] Add a rule to do...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5595][SPARK-5603][SQL] Add a rule to do...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4373#issuecomment-73308129 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-5648][SQL] support "alter ... unset tbl...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4424#issuecomment-73308025 [Test build #26938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26938/consoleFull) for PR 4424 at commit [`6dd8bee`](https://githu

[GitHub] spark pull request: [SPARK-5657][Examples][PySpark] Add PySpark Av...

2015-02-06 Thread staslos
GitHub user staslos opened a pull request: https://github.com/apache/spark/pull/4434 [SPARK-5657][Examples][PySpark] Add PySpark Avro Output Format example There is an Avro Input Format example that shows how to read Avro data in PySpark, but nothing shows how to write from PySpark

[GitHub] spark pull request: [SPARK-5324][SQL] Results of describe can't be...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4249#issuecomment-73307633 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-5324][SQL] Results of describe can't be...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4249 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5648][SQL] support "alter ... unset tbl...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4424#issuecomment-73307280 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73307326 Btw, did you mean 1.2.1? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-5651][SQL] Support 'create db.table' in...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4427#issuecomment-73307227 I don't think this is valid. You use backticks to escape cases where you have invalid characters like `.` in your identifiers. --- If your project is set up for it, yo

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73307181 Thanks for the review @JoshRosen. I'll tag the JIRA issue for backport. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4397 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73306881 Thanks! Merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73306120 [Test build #26928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26928/consoleFull) for PR 4397 at commit [`f819b6c`](https://gith

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73306129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73305908 [Test build #26927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26927/consoleFull) for PR 3986 at commit [`13e257d`](https://gith

[GitHub] spark pull request: [SPARK-4905][STREAMING] FlumeStreamSuite fix.

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4371#issuecomment-73305923 [Test build #26937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26937/consoleFull) for PR 4371 at commit [`af3ba14`](https://githu

[GitHub] spark pull request: [SPARK-4983]insert waiting time before tagging...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3986#issuecomment-73305913 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73305737 [Test build #26931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26931/consoleFull) for PR 4188 at commit [`8e91cc3`](https://gith

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-73305769 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-73305760 [Test build #26932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26932/consoleFull) for PR 3233 at commit [`3f768e3`](https://gith

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73305744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-4905][STREAMING] FlumeStreamSuite fix.

2015-02-06 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/4371#issuecomment-73305408 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73305358 Looks good, but not too familiar with this class --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305341 In this case, you are only running SparkSubmit as the proxy user. Should we not have the executor code also run as the proxy user, so any writes from the app to H

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305254 [Test build #26930 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26930/consoleFull) for PR 4405 at commit [`b6c947d`](https://gith

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73305261 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4432#discussion_r24268769 --- Diff: mllib/src/test/java/org/apache/spark/mllib/regression/JavaStreamingLinearRegressionSuite.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to th

[GitHub] spark pull request: SPARK-5633 pyspark saveAsTextFile support for ...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4403#issuecomment-73305089 LGTM pending Jenkins; thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4431#issuecomment-73304864 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5650][SQL] Support optional 'FROM' clau...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4426 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5650][SQL] Support optional 'FROM' clau...

2015-02-06 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/4426#issuecomment-73304622 Thanks! Merged to master and 1.3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4162#issuecomment-73304383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73304352 Actually, I'm going to hold off on the `branch-1.2` (1.2.2) commit for now, since there's a bit of divergence in that branch and I don't want to break anything. --- I

[GitHub] spark pull request: [SPARK-5366][EC2] Check the mode of private ke...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4162#issuecomment-73304375 [Test build #26924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26924/consoleFull) for PR 4162 at commit [`01ed464`](https://gith

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4414 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5628] Add version option to spark-ec2

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4414#issuecomment-73303739 LGTM. I'm going to pull this into `master` (1.4.0) and `branch-1.3` (1.3.0). I'll also commit it to `branch-1.2` (1.2.2), but for that I'll update the version number

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-73303658 [Test build #26936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26936/consoleFull) for PR 4067 at commit [`bd919be`](https://githu

[GitHub] spark pull request: [SPARK-4808] Configurable spillable memory thr...

2015-02-06 Thread mingyukim
Github user mingyukim commented on the pull request: https://github.com/apache/spark/pull/4420#issuecomment-73303667 Can you elaborate on the "memory size as an additional heuristic" idea? This is consistently causing OOMs in one of our workflows, which is exactly what spilling to dis

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24268035 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -179,121 +182,190 @@ object KafkaUtils { errs =

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24268021 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/OffsetRange.scala --- @@ -19,16 +19,35 @@ package org.apache.spark.streaming.kafka

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73303308 In this particular case we might actually need separate PRs for 1.2 and the Master because the event logs are produced differently there. I wonder if this also applies

[GitHub] spark pull request: [SPARK-4874] [CORE] Collect record count metri...

2015-02-06 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4067#issuecomment-73303194 Jenkins, test this please. This LGTM pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73303249 [Test build #26935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26935/consoleFull) for PR 4311 at commit [`5d9eedf`](https://gith

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73303253 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4311#discussion_r24267784 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -88,6 +88,10 @@ private[spark] class ApplicationMaster(ar

[GitHub] spark pull request: SPARK-5633 pyspark saveAsTextFile support for ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4403#issuecomment-73303027 [Test build #26934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26934/consoleFull) for PR 4403 at commit [`94c014e`](https://githu

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73302969 [Test build #26935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26935/consoleFull) for PR 4311 at commit [`5d9eedf`](https://githu

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24267609 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -179,121 +182,190 @@ object KafkaUtils { errs => thr

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73302682 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73302672 [Test build #26929 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26929/consoleFull) for PR 4433 at commit [`a604816`](https://gith

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24267569 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala --- @@ -179,121 +182,190 @@ object KafkaUtils { errs => thr

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4431#issuecomment-73302546 [Test build #26923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26923/consoleFull) for PR 4431 at commit [`c5da21e`](https://gith

[GitHub] spark pull request: [SPARK-5640] Synchronize ScalaReflection where...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4431#issuecomment-73302558 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-4705:Creating different log directories ...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4311#issuecomment-73302414 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-02-06 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-73302432 I'll do my best to look at it today---I hope! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: SPARK-5633 pyspark saveAsTextFile support for ...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4403#issuecomment-73302426 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24267412 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class MatrixFactorizationModel

[GitHub] spark pull request: [SPARK-5644] [Core]Delete tmp dir when sc is s...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4412#discussion_r24267365 --- Diff: core/src/main/scala/org/apache/spark/SparkEnv.scala --- @@ -93,6 +93,14 @@ class SparkEnv ( // actorSystem.awaitTermination()

[GitHub] spark pull request: [SPARK-4502] [SQL] Fix reads unnecessary neste...

2015-02-06 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/4398#discussion_r24267316 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -53,7 +53,7 @@ object BindReferences extends Log

[GitHub] spark pull request: [Spark-3490] Disable SparkUI for tests (backpo...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3959#issuecomment-73302243 [Test build #26933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26933/consoleFull) for PR 3959 at commit [`5425314`](https://githu

[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267256 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes

[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267242 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes

[GitHub] spark pull request: function hasShutdownDeleteTachyonDir should us...

2015-02-06 Thread haoyuan
Github user haoyuan commented on the pull request: https://github.com/apache/spark/pull/4418#issuecomment-73301858 Thanks @viper-kun Agree w/ @JoshRosen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4428#issuecomment-73301647 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5396] Syntax error in spark scripts on ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4428#issuecomment-73301632 [Test build #26925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26925/consoleFull) for PR 4428 at commit [`ec18465`](https://gith

[GitHub] spark pull request: [SPARK-1825] Fixes cross-platform submit probl...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/899#issuecomment-73301694 Hey @zeodtr I believe this is fixed in #3924 would you mind closing this PR? --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: function hasShutdownDeleteTachyonDir should us...

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4418#issuecomment-73301472 I think we should create a JIRA for this, if only to help us keep track of where this change is committed. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-5613: Catch the ApplicationNotFoundExcep...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4392#issuecomment-73301321 Hey @kasjain can you open this against the master branch next time? It will be easier for us to back port stuff from there --- If your project is set up for it, you c

[GitHub] spark pull request: [Spark-3490] Disable SparkUI for tests (backpo...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3959#issuecomment-73301377 yes, the tests are still not passing I believe, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-4361][Doc] Add more docs for Hadoop Con...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3225 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4502] [SQL] Fix reads unnecessary neste...

2015-02-06 Thread cenyuhai
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/spark/pull/4398#discussion_r24266591 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/dataTypes.scala --- @@ -36,6 +36,8 @@ import org.apache.spark.util.Utils ob

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73300317 To fix the YARN issue maybe we should do something specific there, like escaping the double quotes before passing them to YARN? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-02-06 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-73300352 ping @jkbradley ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-5598][MLLIB] model save/load for ALS

2015-02-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4422#discussion_r24266304 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala --- @@ -136,3 +147,69 @@ class MatrixFactorizationModel(

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73299916 Hey @srowen it seems like this will break existing behavior though. What if I want to run an application with the following arguments ``` a "b c" d ```

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4188#issuecomment-73299811 [Test build #26931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26931/consoleFull) for PR 4188 at commit [`8e91cc3`](https://githu

[GitHub] spark pull request: [SPARK-5493] [core] Add option to impersonate ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4405#issuecomment-73299786 [Test build #26930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26930/consoleFull) for PR 4405 at commit [`b6c947d`](https://githu

[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-73299761 [Test build #26932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26932/consoleFull) for PR 3233 at commit [`3f768e3`](https://githu

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4188#discussion_r24266009 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1283,6 +1291,7 @@ private[spark] object Utils extends Logging { if (inWord || i

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73298114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5601][MLLIB] make streaming linear algo...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4432#issuecomment-73298105 [Test build #26922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26922/consoleFull) for PR 4432 at commit [`1f662b3`](https://gith

[GitHub] spark pull request: SPARK-4267 [CORE] Failing to launch jobs on Sp...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4188#discussion_r24265176 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1283,6 +1291,7 @@ private[spark] object Utils extends Logging { if (inWord

[GitHub] spark pull request: [SPARK-4964][Streaming][Kafka] More updates to...

2015-02-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/4384#discussion_r24265028 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala --- @@ -36,14 +36,12 @@ import kafka.utils.VerifiableProperties *

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73297460 [Test build #26929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26929/consoleFull) for PR 4433 at commit [`a604816`](https://githu

[GitHub] spark pull request: [SPARK-4361][Doc] Add more docs for Hadoop Con...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3225#discussion_r24264861 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -630,7 +634,10 @@ class SparkContext(config: SparkConf) extends SparkStatusAPI wit

[GitHub] spark pull request: [SPARK-4361][Doc] Add more docs for Hadoop Con...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3225#issuecomment-73296838 I think it's safe to say that we won't implement the alternative behavior that @JoshRosen suggested by the release. For this reason I think we should at least document

[GitHub] spark pull request: [SPARK-5652][Mllib] Use broadcasted weights in...

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4429 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73296645 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-5652][Mllib] Use broadcasted weights in...

2015-02-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4429#issuecomment-73296580 Merged into master and branch-1.3. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-4361][Doc] Add more docs for Hadoop Con...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3225#discussion_r24264456 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -242,7 +242,11 @@ class SparkContext(config: SparkConf) extends SparkStatusAPI wit

[GitHub] spark pull request: [SPARK-5611] [EC2] Allow spark-ec2 repo and br...

2015-02-06 Thread nchammas
Github user nchammas commented on the pull request: https://github.com/apache/spark/pull/4385#issuecomment-73295741 @florianverhein This is looking good. Have you tested this against a fork named `spark-ec2` as well as a fork named something else? --- If your project is set up for it

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4433#issuecomment-73295691 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73295760 [Test build #26928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26928/consoleFull) for PR 4397 at commit [`f819b6c`](https://githu

[GitHub] spark pull request: [SPARK-5611] [EC2] Allow spark-ec2 repo and br...

2015-02-06 Thread nchammas
Github user nchammas commented on a diff in the pull request: https://github.com/apache/spark/pull/4385#discussion_r24264051 --- Diff: ec2/spark_ec2.py --- @@ -1007,6 +1023,11 @@ def real_main(): print >> stderr, "ebs-vol-num cannot be greater than 8" sys.e

[GitHub] spark pull request: [SPARK-5555] Enable UISeleniumSuite tests

2015-02-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4334 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5619][SQL] Support 'show roles' in Hive...

2015-02-06 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4397#issuecomment-73295369 ok to test @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-5656] Fail gracefully for large values ...

2015-02-06 Thread mbittmann
GitHub user mbittmann opened a pull request: https://github.com/apache/spark/pull/4433 [SPARK-5656] Fail gracefully for large values of k and/or n that will ex... ...ceed max int. Large values of k and/or n in EigenValueDecomposition.symmetricEigs will result in array initi

[GitHub] spark pull request: [SPARK-5555] Enable UISeleniumSuite tests

2015-02-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4334#issuecomment-73295273 I'm merging this into `master` (1.4.0) and `branch-1.3` (1.3.0). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

  1   2   3   4   >