[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1172#discussion_r14053159 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorBackend.scala --- @@ -26,4 +26,7 @@ import org.apache.spark.TaskState.TaskState */

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46773549 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46773550 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46773551 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-22 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-46773577 1.1 is not released yet. This PR is in master but not in 1.0 (it may be released in 1.0.1 or if not then 1.1). So you'll have to clone master and run sbt/sbt

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46773627 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46773676 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46773677 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1996. Remove use of special Maven repo f...

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1170 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/980#discussion_r14053246 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaInputDStream.scala --- @@ -112,10 +114,14 @@ class KafkaReceiver[ val

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/980#discussion_r14053248 --- Diff: external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaInputDStream.scala --- @@ -112,10 +114,14 @@ class KafkaReceiver[ val

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/980#issuecomment-46774049 LGTM pending tests. Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46774097 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/980#issuecomment-46774099 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46774098 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16006/ --- If your

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/980#issuecomment-46774102 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/1175 SPARK-2231: dev/run-tests should include YARN and use a recent Hadoop version ...rsion You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1175#issuecomment-46774162 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46774166 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16007/ --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1175#issuecomment-46774164 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1082#discussion_r14053327 --- Diff: core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala --- @@ -559,6 +559,19 @@ class JavaSparkContext(val sc: SparkContext) extends

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1082#issuecomment-46774517 LGTM with a minor comment that can be addressed on merge. @rxin any further comments? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/980#issuecomment-46774624 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16008/ --- If your project

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1175#issuecomment-46774725 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1175#issuecomment-46774727 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16009/ --- If your

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1172#discussion_r14053359 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorBackend.scala --- @@ -26,4 +26,7 @@ import org.apache.spark.TaskState.TaskState */

[GitHub] spark pull request: SPARK-2231: dev/run-tests should include YARN ...

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1175 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46775109 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2034. KafkaInputDStream doesn't close re...

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/980 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46775157 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46775153 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46775720 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16010/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46775719 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-2229: FileAppender throw an llegalArgume...

2014-06-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/1174#issuecomment-46778033 +1 I literally ran into this too 6 hours ago and had the same fix. It's from the change for SPARK-1940. I think it's a good idea that test be run on Java 6 as a result?

[GitHub] spark pull request: [MLLIB] [SPARK-2222] Add multiclass evaluation...

2014-06-22 Thread avulanov
Github user avulanov commented on the pull request: https://github.com/apache/spark/pull/1155#issuecomment-46778186 The micro averaged Precision and Recall are equal for multiclass classifier, because sum(fni)=sum(fpi), i.e. they are just the sum of all non-diagonal elements in

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46778249 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46778248 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46779166 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16011/ --- If your

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46779165 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT...

2014-06-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/906#issuecomment-46781514 Ah OK, it did fail for me locally with `sbt clean assembly test`. Sorry, this did in fact have a problem. I think akka does need the old Netty; the second commit was a

[GitHub] spark pull request: SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/906#issuecomment-46781581 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/906#issuecomment-46782563 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16012/ --- If your project

[GitHub] spark pull request: SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/906#issuecomment-46782562 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [Spark 1199][WIP] Changed wrappers to not use ...

2014-06-22 Thread ScrapCodes
GitHub user ScrapCodes opened a pull request: https://github.com/apache/spark/pull/1176 [Spark 1199][WIP] Changed wrappers to not use vals and thus avoid Path dependent types problem. TODO: Write description. basically it fails for one particular scenario and I am enjoying tough

[GitHub] spark pull request: [Spark 1199][WIP] Changed wrappers to not use ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1176#issuecomment-46785360 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1199][WIP] Changed wrappers to not use ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1176#issuecomment-46785365 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [MLLIB] [SPARK-2222] Add multiclass evaluation...

2014-06-22 Thread xiejuncs
Github user xiejuncs commented on the pull request: https://github.com/apache/spark/pull/1155#issuecomment-46786256 It makes sense. You are right. sum(fni)=sum(fpi). The recall and precision are the same. Thanks very much. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-1199][WIP] Changed wrappers to not use ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1176#issuecomment-46786334 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1199][WIP] Changed wrappers to not use ...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1176#issuecomment-46786336 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16013/ --- If your project is set up for it, you can

[GitHub] spark pull request: Update BasicOperationsSuite.scala

2014-06-22 Thread baishuo
Github user baishuo commented on the pull request: https://github.com/apache/spark/pull/1084#issuecomment-46786699 let me do a check --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-06-22 Thread rjurney
Github user rjurney commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-46789138 Thanks, master doesn't build for me. Is there a particular commit you recommend using? [error] [error] last tree to typer:

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1173#issuecomment-46789151 Thanks. I've merged this in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1316. Remove use of Commons IO

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1173 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2141] Adding getPersistentRddIds and un...

2014-06-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1082#issuecomment-46789198 Yup looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46789575 I think these are failing because our tests assume that in local mode we enforce the frame size limit (which we actually don't need to). I'll make the appropriate

[GitHub] spark pull request: add a materialize method to materialize Vertex...

2014-06-22 Thread bxshi
GitHub user bxshi opened a pull request: https://github.com/apache/spark/pull/1177 add a materialize method to materialize VertexRDD by calling RDD's count Seems one can not materialize VertexRDD by simply calling count method, which is overridden by VertexRDD. But if you call

[GitHub] spark pull request: add a materialize method to materialize Vertex...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1177#issuecomment-46792651 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: add a materialize method to materialize Vertex...

2014-06-22 Thread bxshi
Github user bxshi commented on the pull request: https://github.com/apache/spark/pull/1177#issuecomment-46792759 Here's a simple code that could reproduce the problem ``` val conf = new SparkConf().setAppName(HDTM) .setMaster(local[4]) val sc =

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46793729 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/1172#discussion_r14056893 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorBackend.scala --- @@ -26,4 +26,7 @@ import org.apache.spark.TaskState.TaskState */

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46793847 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46793850 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46794756 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46794757 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16014/ --- If your

[GitHub] spark pull request: [SPARK-2124] Move aggregation into shuffle imp...

2014-06-22 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/1064#issuecomment-46798422 Hi Matei, thanks for your review, I will update the code soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: SPARK-2229: FileAppender throw an llegalArgume...

2014-06-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1174#issuecomment-46799166 Thanks. I'm merging this in master. @pwendell - we probably want to run tests on JDK6 ... (if possible both in the build matrix) --- If your project is set up for

[GitHub] spark pull request: SPARK-2229: FileAppender throw an llegalArgume...

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1174 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-06-22 Thread YanjieGao
Github user YanjieGao commented on the pull request: https://github.com/apache/spark/pull/1151#issuecomment-46800258 Hi marmbrus I update these files as your comment tips ,but i think i may make some mistakes in the code .Could you help me and give me some tips ?I will continue

[GitHub] spark pull request: [SQL][SPARK-2212]HashJoin(Shuffled)

2014-06-22 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/1147#issuecomment-46800787 Thank you all for the comments, I will changed some of the code accordingly. This PR actually contains 2 relevant parts: - Code Re-factor for Join -

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46801344 @aarondav mind taking a final pass and merging this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46801472 Absolutely. LGTM, merging into branch-1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Compression should be a setting for individual...

2014-06-22 Thread ScrapCodes
Github user ScrapCodes closed the pull request at: https://github.com/apache/spark/pull/1091 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Compression should be a setting for individual...

2014-06-22 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/1091#issuecomment-46801657 Thanks @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1937: fix issue with task locality

2014-06-22 Thread lirui-intel
Github user lirui-intel commented on the pull request: https://github.com/apache/spark/pull/892#issuecomment-46801884 Sorry about the code style and thanks @mateiz for pointing out. I've updated the patch. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46801936 You may have to close this manually, @pwendell, I'm not sure github will close it if it's not in master. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: SPARK-1937: fix issue with task locality

2014-06-22 Thread lirui-intel
Github user lirui-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/892#discussion_r14059200 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -181,16 +181,14 @@ private[spark] class TaskSetManager( var

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell closed the pull request at: https://github.com/apache/spark/pull/1172 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1112, 2156] (1.0 edition) Use correct a...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1172#issuecomment-46802157 Thanks, closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SQL] SPARK-1800 Add broadcast hash join...

2014-06-22 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/1163#discussion_r14059227 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala --- @@ -44,10 +49,21 @@ import

[GitHub] spark pull request: spark-ec2: quote command line args

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1169#issuecomment-46803411 Thanks - I merged this into several maintenance branches and I also created this JIRA to track it: https://issues.apache.org/jira/browse/SPARK-2241 --- If

[GitHub] spark pull request: SPARK-2166 - Listing of instances to be termin...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/270#issuecomment-46804399 This was actually a pretty tough merge since we changed the spacing around a lot in `spark_ec2` recently. I went ahead and manually dealt with the merge. I also made two

[GitHub] spark pull request: SPARK-2166 - Listing of instances to be termin...

2014-06-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/270 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2150: Provide direct link to finished ap...

2014-06-22 Thread rahulsinghaliitd
Github user rahulsinghaliitd commented on the pull request: https://github.com/apache/spark/pull/1094#issuecomment-46804664 @sryza thanks for the thumbs up. Although I wonder if the approach in https://github.com/apache/spark/pull/1112 is better for passing the UI address

[GitHub] spark pull request: SPARK-2099. Report progress while task is runn...

2014-06-22 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1056#issuecomment-46805087 Sure - it would be great to add a general heartbeat mechanism that is shared between this and the blockmanager. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-2242] HOTFIX: Do not mask pyspark stder...

2014-06-22 Thread andrewor14
GitHub user andrewor14 opened a pull request: https://github.com/apache/spark/pull/1178 [SPARK-2242] HOTFIX: Do not mask pyspark stderr from output This reverts a change introduced in 3870248740d83b0292ccca88a494ce19783847f0 that masked stderr from surfacing to the `bin/pyspark`

[GitHub] spark pull request: [SPARK-2242] HOTFIX: Do not mask pyspark stder...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46805167 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2242] HOTFIX: Do not mask pyspark stder...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46805169 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-06-22 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14060166 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -369,6 +369,17 @@ class SQLQuerySuite extends QueryTest {

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-06-22 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14060224 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -119,6 +119,7 @@ class SqlParser extends StandardTokenParsers

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46806151 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46806156 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1946] Submit stage after (configured ra...

2014-06-22 Thread li-zhihui
Github user li-zhihui commented on a diff in the pull request: https://github.com/apache/spark/pull/900#discussion_r14060589 --- Diff: yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerBackend.scala --- @@ -0,0 +1,51 @@ +/* + * Licensed to

[GitHub] spark pull request: SPARK-1729. Make Flume pull data from source, ...

2014-06-22 Thread harishreedharan
Github user harishreedharan commented on the pull request: https://github.com/apache/spark/pull/807#issuecomment-46806458 @tdas - Have you gotten a chance to take a look at this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46806569 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46806570 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16015/ --- If your

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-06-22 Thread YanjieGao
Github user YanjieGao commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14060925 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -369,6 +369,17 @@ class SQLQuerySuite extends QueryTest {

[GitHub] spark pull request: [SPARK-2234][SQL]Spark SQL basicOperators add ...

2014-06-22 Thread YanjieGao
Github user YanjieGao commented on a diff in the pull request: https://github.com/apache/spark/pull/1151#discussion_r14060932 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/SqlParser.scala --- @@ -119,6 +119,7 @@ class SqlParser extends StandardTokenParsers

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46807454 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16016/ --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46807453 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2242] HOTFIX: pyspark shell hangs on si...

2014-06-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/1178#issuecomment-46807757 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   >