[GitHub] spark pull request: fix #SPARK-1149 Bad partitioners can cause Spa...

2014-02-28 Thread witgo
GitHub user witgo opened a pull request: https://github.com/apache/spark/pull/44 fix #SPARK-1149 Bad partitioners can cause Spark to hang You can merge this pull request into a Git repository by running: $ git pull https://github.com/witgo/spark SPARK-1149 Alternatively you

[GitHub] spark pull request: fix #SPARK-1149 Bad partitioners can cause Spa...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/44#issuecomment-36332248 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: remove sbt/ prefix in $JAR variable make the...

2014-02-28 Thread coderh
GitHub user coderh opened a pull request: https://github.com/apache/spark/pull/46 remove sbt/ prefix in $JAR variable make the script pass if test There is no sbt/ dir under sbt/sbt. if test on line 29 can not pass. remove sbt/ prefix in $JAR variable make the script pass

[GitHub] spark pull request: remove sbt/ prefix in $JAR variable make the...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/46#issuecomment-36338520 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: remove sbt/ prefix in $JAR variable make the...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/46#issuecomment-36338504 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1121 Only add avro if the build is for H...

2014-02-28 Thread lfrancke
Github user lfrancke commented on the pull request: https://github.com/apache/spark/pull/6#issuecomment-36343187 The Avro dependency is still in the yarn/pom.xml file and causes Maven to fail early. I'm happy to submit a pull request/issue but maybe you want to do another Hotfix

YARN Maven build questions

2014-02-28 Thread Lars Francke
Hey, I'm trying to dig into Spark's code but am running into a couple of problems. 1) The yarn-common directory is not included in the Maven build causing things to fail because the dependency is missing. If I see the history correct it used to be a Maven module but is not anymore. 2) When I

[GitHub] spark pull request: remove sbt/ prefix in $JAR variable make the...

2014-02-28 Thread coderh
Github user coderh closed the pull request at: https://github.com/apache/spark/pull/46 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: fix #SPARK-1149 Bad partitioners can cause Spa...

2014-02-28 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/44#discussion_r10164732 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -847,6 +847,8 @@ class SparkContext( partitions: Seq[Int],

[GitHub] spark pull request: fix #SPARK-1149 Bad partitioners can cause Spa...

2014-02-28 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/44#discussion_r10166854 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -847,6 +847,8 @@ class SparkContext( partitions: Seq[Int],

[GitHub] spark pull request: fix #SPARK-1149 Bad partitioners can cause Spa...

2014-02-28 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/44#discussion_r10166948 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -847,6 +847,8 @@ class SparkContext( partitions: Seq[Int],

[GitHub] spark pull request: SPARK-1032. If Yarn app fails before registeri...

2014-02-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/28#issuecomment-36362791 I committed this. Thanks Sandy! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Update dev merge script to use spark.git inste...

2014-02-28 Thread tgravescs
GitHub user tgravescs opened a pull request: https://github.com/apache/spark/pull/47 Update dev merge script to use spark.git instead of incubator-spark You can merge this pull request into a Git repository by running: $ git pull https://github.com/tgravescs/spark

[GitHub] spark pull request: Update dev merge script to use spark.git inste...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/47#issuecomment-36363038 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Update dev merge script to use spark.git inste...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/47#issuecomment-36363010 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: fix repo location in create script

2014-02-28 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/48 fix repo location in create script fix the repo location in create_release script You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-1150] fix repo location in create scrip...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/48#issuecomment-36364625 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Update dev merge script to use spark.git inste...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/47#issuecomment-36366007 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12930/ --- If your project

[GitHub] spark pull request: Update dev merge script to use spark.git inste...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/47#issuecomment-36366006 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1032. If Yarn app fails before registeri...

2014-02-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/28 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36367636 +1. Looks Good. Thanks Sandy! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36368483 @sryza it looks like this is no longer mergeable due to other check ins. Can you please update it to the latest. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: SPARK-1032. If Yarn app fails before registeri...

2014-02-28 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/28#issuecomment-36368855 Thanks Tom! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1121 Only add avro if the build is for H...

2014-02-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/6#issuecomment-36370069 I disagree with this change going in and breaking the yarn hadoop0.23 build. If we are going to support maven and hadoop 0.23 you should be able to build it without

Re: YARN Maven build questions

2014-02-28 Thread Lars Francke
Hey, so currently it doesn't work because of https://github.com/apache/spark/pull/6#issuecomment-36343187 IntelliJ reports a lot of warnings with default settings and I haven't found a way to tell IntellJ to use different Hadoop versions yet. mvn clean compile -Pyarn fails as well (compilation

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/29#issuecomment-36375413 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12931/ --- If your project

[GitHub] spark pull request: SPARK-1121 Only add avro if the build is for H...

2014-02-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/6#issuecomment-36378496 @tgravescs hey tom if we can't do this with a profile we can revert it. Didn't realize 0.23.X was widely used. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: MLI-2: Start adding k-fold cross validation to...

2014-02-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/18#discussion_r10174439 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala --- @@ -62,6 +67,20 @@ object MLUtils { } /** + * Return a k

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10174657 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -84,12 +84,27 @@ private class DiskStore(blockManager: BlockManager,

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10174727 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala --- @@ -146,6 +146,12 @@ object BlockFetcherIterator { }

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10175006 --- Diff: core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala --- @@ -339,8 +217,8 @@ class JobLogger(val user: String, val logDirName:

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36380483 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36380718 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36380733 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36383621 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36383623 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12932/ --- If your

[GitHub] spark pull request: SPARK-1051. On YARN, executors don't doAs subm...

2014-02-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/29 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Spark JIRA

2014-02-28 Thread Evan Chan
Hey guys, There is no plan to move the Spark JIRA from the current https://spark-project.atlassian.net/ right? -- -- Evan Chan Staff Engineer e...@ooyala.com |

Re: Spark JIRA

2014-02-28 Thread Nan Zhu
I think they are working on it? https://issues.apache.org/jira/browse/SPARK Best, -- Nan Zhu On Friday, February 28, 2014 at 2:29 PM, Evan Chan wrote: Hey guys, There is no plan to move the Spark JIRA from the current https://spark-project.atlassian.net/ right? -- -- Evan

Re: Spark JIRA

2014-02-28 Thread Josh Rosen
Apache INFRA has been trying to import our JIRA issues since June 2013: https://issues.apache.org/jira/browse/INFRA-6419. It seems that JIRA introduces minor import incompatibilities with every update to JIRA OnDemand, so it's been a challenge to get the Apache and Atlasssian JIRA versions to

Re: Spark JIRA

2014-02-28 Thread Evan Chan
Ok, will continue to work with the existing one for now. On Fri, Feb 28, 2014 at 11:36 AM, Josh Rosen rosenvi...@gmail.com wrote: Apache INFRA has been trying to import our JIRA issues since June 2013: https://issues.apache.org/jira/browse/INFRA-6419. It seems that JIRA introduces minor

New JIRA ticket: cleaning up app-* folders

2014-02-28 Thread Evan Chan
Hey guys, FYI, I just filed a new ticket, https://spark-project.atlassian.net/browse/SPARK-1154, concerning that Spark leaves tons of app-* folders on disk, which can quickly fill up the disk. Feel free to comment. I believe it is better for Spark to clean itself up; ie users should not need a

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/38#issuecomment-36387844 I'd like to hold off on merging this until we see how https://github.com/ipython/ipython/pull/5226 is resolved. It looks like IPython executes `PYTHONSTARTUP` for

[GitHub] spark pull request: SPARK-1134 pyspark only uses ipython if starte...

2014-02-28 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/38#discussion_r10178681 --- Diff: bin/pyspark --- @@ -58,7 +58,8 @@ if [ -n $IPYTHON_OPTS ]; then IPYTHON=1 fi -if [[ $IPYTHON = 1 ]] ; then +# Only use

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/40#discussion_r10178847 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -149,7 +149,13 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/40#discussion_r10178907 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentSuite.scala --- @@ -104,4 +104,45 @@ class GradientDescentSuite extends

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread pwendell
GitHub user pwendell opened a pull request: https://github.com/apache/spark/pull/49 SPARK-1121: Use 'avro' profile in Maven. This lets us explicitly include Avro based on a profile for 0.23.X builds. It makes me sad how convoluted it is to express this logic in Maven.

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36389807 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36389809 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10180027 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -84,12 +84,27 @@ private class DiskStore(blockManager: BlockManager, diskManager:

Re: [DISCUSS] Necessity of Maven *and* SBT Build in Spark

2014-02-28 Thread Patrick Wendell
Hey, Thanks everyone for chiming in on this. I wanted to summarize these issues a bit particularly wrt the constituents involved - does this seem accurate? = Spark Users = In general those linking against Spark should be totally unaffected by the build choice. Spark will continue to publish

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36392301 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12933/ --- If your project

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36392299 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

New blog post on Spark + Parquet + Scrooge

2014-02-28 Thread Evan Chan
http://engineering.ooyala.com/blog/using-parquet-and-scrooge-spark Enjoy! By the way, I was not able to subscribe to the user-digest list for some reason. The help email claims Similar addresses exist for the digest list: user-allow-digest-subscr...@spark.apache.org

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/40#discussion_r10181447 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -149,7 +149,13 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36398304 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36398321 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10183037 --- Diff: core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala --- @@ -339,8 +217,8 @@ class JobLogger(val user: String, val logDirName: String)

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10183222 --- Diff: core/src/main/scala/org/apache/spark/scheduler/JobLogger.scala --- @@ -339,8 +217,8 @@ class JobLogger(val user: String, val logDirName:

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36398886 I'll give it a try. Any reason we don't just tie this to the yarn-alpha profile? Or does it not apply to the hadoop 2.0.2 type builds? --- If your project is set up

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10183713 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -194,12 +194,19 @@ class DAGScheduler( } }

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10184513 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -17,21 +17,23 @@ package org.apache.spark.executor

Re: [DISCUSS] Necessity of Maven *and* SBT Build in Spark

2014-02-28 Thread Mark Hamstra
Couple of comments: 1) Whether the Spark POM is produced by SBT or Maven shouldn't matter for those who just need to link against published artifacts, but right now SBT and Maven do not produce equivalent POMs for Spark -- I think 2) Incremental builds using Maven are trivially more difficult

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10186680 --- Diff: core/src/main/scala/org/apache/spark/ui/UISparkListener.scala --- @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10186902 --- Diff: core/src/main/scala/org/apache/spark/util/FileLogger.scala --- @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread kellrott
GitHub user kellrott opened a pull request: https://github.com/apache/spark/pull/50 Patch for SPARK-942 This is a port of a pull request original targeted at incubator-spark: https://github.com/apache/incubator-spark/pull/180 Essentially if a user returns a generative

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36408974 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36408973 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/42#discussion_r10187737 --- Diff: core/src/main/scala/org/apache/spark/util/FileLogger.scala --- @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: Patch for SPARK-942

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/50#issuecomment-36410331 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/40#discussion_r10188555 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -149,7 +149,14 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413266 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413264 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/40#discussion_r10188617 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -149,7 +149,14 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413338 @rxin It seems that I rebased and squashed the commits which made Jenkins unhappy. Could you ask Jenkins to build again? Thanks. --- If your project is set up for it, you

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413420 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1151: Update dev merge script to use spa...

2014-02-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/47#issuecomment-36413458 Thanks tom - merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36413556 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413637 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12936/ --- If your

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36413838 It is running now. Let's wait for Jenkins to come back. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36414230 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12937/ --- If your

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/43#issuecomment-36414229 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414246 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414262 Your suggestion of @tgravescs tying this yarn-alpha I think is strictly better. Just updated the patch. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: Initialized the regVal for first iteration in ...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/40#issuecomment-36414270 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10188851 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -84,12 +84,27 @@ private class DiskStore(blockManager: BlockManager,

[GitHub] spark pull request: SPARK-1151: Update dev merge script to use spa...

2014-02-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/47 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-1145: Memory mapping with many small blo...

2014-02-28 Thread pwendell
Github user pwendell commented on a diff in the pull request: https://github.com/apache/spark/pull/43#discussion_r10188872 --- Diff: core/src/main/scala/org/apache/spark/storage/DiskStore.scala --- @@ -84,12 +84,27 @@ private class DiskStore(blockManager: BlockManager,

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414646 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12939/ --- If your

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414645 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414655 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-1100] prevent Spark from overwriting di...

2014-02-28 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/11#issuecomment-36414832 @pwendell , I just updated the code In the latest update, I make the checking only applicable to FileOutputFormat, the difference with your suggestion is that I

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36414847 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36416705 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36416731 Build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36416732 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12941/ --- If your

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36417618 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36417620 Build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [WIP] [SPARK-1132] Persisting Web UI through r...

2014-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/42#issuecomment-36418483 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12942/ --- If your project

[GitHub] spark pull request: SPARK-1121: Use 'avro' profile in Maven.

2014-02-28 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/49#issuecomment-36419075 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

  1   2   >