[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/120#discussion_r10553691 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -133,11 +148,11 @@ class ClientArguments(val args:

[GitHub] spark pull request: SPARK-1126. spark-app preliminary

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/86#discussion_r10553709 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1126. spark-app preliminary

2014-03-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/86#issuecomment-37502813 @mateiz maybe you could take a pass on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/120#discussion_r10553724 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -133,11 +148,11 @@ class ClientArguments(val args:

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/120#discussion_r10553741 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -67,24 +67,39 @@ class ClientArguments(val args:

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37503046 Sandy - looks good to me. Are you still changing things? I noticed there are a few comments that maybe should be updated: ```

[GitHub] spark pull request: SPARK-1236 - Upgrade Jetty to 9.1.3.v20140225.

2014-03-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/113#issuecomment-37503197 @andrewor14 any comments or reservations on this one? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread sryza
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/120#discussion_r10554025 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -133,11 +148,11 @@ class ClientArguments(val args:

[GitHub] spark pull request: SPARK-1019: pyspark RDD take() throws an NPE

2014-03-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/112 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37504134 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13150/ --- If your project

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37504125 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13152/ --- If your project

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37504124 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: MLI-1 Decision Trees

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/79#issuecomment-37504122 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37504133 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37504132 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13151/ --- If your project

[GitHub] spark pull request: Spark 1162 Implemented takeOrdered in pyspark.

2014-03-13 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/97#issuecomment-37505082 BTW as mentioned above please use PriorityQueue here instead of copying their heap. It's just a lot of work to copy the heap.. we can take the performance hit instead.

[GitHub] spark pull request: [SPARK-1198] Allow pipes tasks to run in diffe...

2014-03-13 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/128#discussion_r10554415 --- Diff: core/pom.xml --- @@ -184,13 +184,12 @@ artifactIdmetrics-graphite/artifactId /dependency dependency -

[GitHub] spark pull request: [Spark-1234] clean up text in running-on-yarn....

2014-03-13 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/130#issuecomment-37505269 Jenkins, add to whitelist and test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Spark 1162 Implemented takeOrdered in pyspark.

2014-03-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/97#issuecomment-37505304 PriorityQueue in a way is just a wrapper over heapq and allows for blocking for put and get(AFAIU). We would need maxheapq to retain the top N smallest elements. One

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37506200 @mengxr nice work! Thanks for this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Spark 1162 Implemented takeOrdered in pyspark.

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/97#issuecomment-37506805 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1234] clean up text in running-on-yarn....

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/130#issuecomment-37506800 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1126. spark-app preliminary

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/86#issuecomment-37506962 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1126. spark-app preliminary

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/86#issuecomment-37507039 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13155/ --- If your

[GitHub] spark pull request: SPARK-1126. spark-app preliminary

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/86#issuecomment-37507037 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/131#issuecomment-37507055 Thanks. I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Spark 1162 Implemented takeOrdered in pyspark.

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/97#issuecomment-37509934 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [Spark-1234] clean up text in running-on-yarn....

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/130#issuecomment-37509933 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: Spark 1162 Implemented takeOrdered in pyspark.

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/97#issuecomment-37509935 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13154/ --- If your project

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-37510206 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1237, 1238] Improve the computation of ...

2014-03-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/131 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-37514176 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13156/ --- If your project

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-37514175 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-37514378 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: MetadataCleaner - fine control cleanup documen...

2014-03-13 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/89#discussion_r10559136 --- Diff: docs/configuration.md --- @@ -487,6 +477,88 @@ Apart from these, the following properties are also available, and may be useful /tr /table

[GitHub] spark pull request: MetadataCleaner - fine control cleanup documen...

2014-03-13 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/89#discussion_r10559142 --- Diff: docs/configuration.md --- @@ -487,6 +477,88 @@ Apart from these, the following properties are also available, and may be useful /tr /table

[GitHub] spark pull request: SPARK-1096, a space after comment start style ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/124#issuecomment-37518903 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13157/ --- If your project

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37518963 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Add unit test to spark_ec2 script

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/134#issuecomment-37518956 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37518962 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37523239 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13158/ --- If your project

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37527761 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Prevent ContextClassLoader of Actor from becom...

2014-03-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-37529958 Thanks for the fix. Only for the record this happens only when MASTER=local or local[2]. Looks good. It might be good to add above test case in ReplSuite

[GitHub] spark pull request: Prevent ContextClassLoader of Actor from becom...

2014-03-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/15#issuecomment-37530227 Mind changing the PR title to add Jira ID? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Spark 615 map partitions with index callable f...

2014-03-13 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/16#issuecomment-37531010 It might be good to add this test in java8 API suite ? Not sure if its 100% necessary, but there exist one for all other APIs (I hope!!). Thoughts ? --- If your

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37533206 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37533178 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37540002 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37540073 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1144 Added license and RAT to check lice...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/125#issuecomment-37540004 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13160/ --- If your project

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37543156 Looks good to me. I made the small comment about perhaps leaving the --am-class out of the usage but I'm ok either way. --- If your project is set up for it, you can

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37547666 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37547667 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13161/ --- If your project

[GitHub] spark pull request: MetadataCleaner - fine control cleanup documen...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/89#issuecomment-37554932 @mridulm and @puravaggarwal Thanks for doing this. As a general direction, however, we are actually moving away from MetadataCleaner in the long run - PR#126 is the

[GitHub] spark pull request: SPARK-1236 - Upgrade Jetty to 9.1.3.v20140225.

2014-03-13 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/113#issuecomment-37555992 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576154 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10576181 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -149,14 +151,9 @@ private[spark] class MapOutputTracker(conf: SparkConf)

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576332 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37563131 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37563870 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/101#discussion_r10576136 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -222,4 +232,19 @@ private[spark] object HadoopRDD { def

[GitHub] spark pull request: SPARK-782. Shade ASM

2014-03-13 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/90#issuecomment-37563984 Do you mind closing this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-782. Shade ASM

2014-03-13 Thread sryza
Github user sryza closed the pull request at: https://github.com/apache/spark/pull/90 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37564441 Updated patch with feedback --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10577891 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -17,28 +17,24 @@ package org.apache.spark.scheduler

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10578300 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -17,28 +17,24 @@ package org.apache.spark.scheduler

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10578363 --- Diff: core/src/main/scala/org/apache/spark/ContextCleaner.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37570650 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1128: set hadoop task properties when co...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/101#issuecomment-37570651 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13162/ --- If your project

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37570639 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13163/ --- If your project

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37570638 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10579167 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf: SparkConf)

[GitHub] spark pull request: [SPARK-1186] : Enrich the Spark Shell to suppo...

2014-03-13 Thread berngp
Github user berngp commented on a diff in the pull request: https://github.com/apache/spark/pull/116#discussion_r10579188 --- Diff: bin/spark-shell --- @@ -30,69 +30,367 @@ esac # Enter posix mode for bash set -o posix -CORE_PATTERN=^[0-9]+$

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/135#issuecomment-37570921 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37570930 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37570929 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/135#discussion_r10579726 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -310,6 +310,9 @@ abstract class RDD[T: ClassTag]( * Return a sampled subset of this

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/135#discussion_r10580163 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -310,6 +310,9 @@ abstract class RDD[T: ClassTag]( * Return a sampled subset of

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/135#discussion_r10580151 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -310,6 +310,9 @@ abstract class RDD[T: ClassTag]( * Return a sampled subset of this

[GitHub] spark pull request: [MLLIB-18] [WIP] Adding sparse data support an...

2014-03-13 Thread dlwh
Github user dlwh commented on the pull request: https://github.com/apache/spark/pull/117#issuecomment-37572686 I'll release something this weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1132] Persisting Web UI through refacto...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10580179 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1241] Add sliding to RDD

2014-03-13 Thread mengxr
GitHub user mengxr opened a pull request: https://github.com/apache/spark/pull/136 [SPARK-1241] Add sliding to RDD Sliding is useful for operations like creating n-grams, calculating total variation, numerical integration, etc. This is similar to

[GitHub] spark pull request: [SPARK-1132] Persisting Web UI through refacto...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10580856 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/135#discussion_r10580942 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -457,6 +457,10 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10581534 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -17,28 +17,24 @@ package org.apache.spark.scheduler

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10581648 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf: SparkConf)

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/120#issuecomment-37577269 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1241] Add sliding to RDD

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/136#issuecomment-37577527 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1183. Don't use worker to mean executo...

2014-03-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/120 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/135#issuecomment-37577534 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10582800 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -181,15 +178,50 @@ private[spark] class MapOutputTracker(conf: SparkConf)

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/135#discussion_r10583451 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -457,6 +457,10 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-1132] Persisting Web UI through refacto...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10584092 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkReplayerBus.scala --- @@ -0,0 +1,97 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-1241] Add sliding to RDD

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/136#issuecomment-37583751 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Bundle tachyon

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/137#issuecomment-37584054 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Bundle tachyon

2014-03-13 Thread nicklan
GitHub user nicklan opened a pull request: https://github.com/apache/spark/pull/137 Bundle tachyon This should all work as expected with the current version of the tachyon tarball (0.4.1) You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/135#issuecomment-37584222 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10585234 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -17,28 +17,24 @@ package org.apache.spark.scheduler

[GitHub] spark pull request: [SPARK-1103] [WIP] Automatic garbage collectio...

2014-03-13 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/126#discussion_r10586047 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ShuffleMapTask.scala --- @@ -17,28 +17,24 @@ package org.apache.spark.scheduler

[GitHub] spark pull request: SPARK-1240: handle the case of empty RDD when ...

2014-03-13 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/135#issuecomment-37587059 LGTM. Waiting for Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   >