[GitHub] spark pull request: Fixed streaming examples docs to use run-examp...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/722#discussion_r12514182 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala --- @@ -35,8 +35,8 @@ import org.apache.spark.SparkConf *

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/715#discussion_r12515148 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -259,19 +238,30 @@ private[spark] class Executor( }

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/715#issuecomment-42799372 Looks good to me too ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/715#discussion_r12514626 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -259,19 +238,30 @@ private[spark] class Executor( }

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread ScrapCodes
Github user ScrapCodes commented on the pull request: https://github.com/apache/spark/pull/715#issuecomment-42797519 > It also turns out that it is unlikely that the IndestructibleActorSystem actually works, given testing (here). I works but in case of OOMs, the behavior can be v

[GitHub] spark pull request: [SPARK-1752][MLLIB] Standardize text format fo...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/685#issuecomment-42623397 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Include the sbin/spark-config.sh in spark-exec...

2014-05-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/651#issuecomment-42615028 LGTM - @bouk did you test this new fix and make sure it works? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/715#issuecomment-42797560 Looks pretty good to me, just made one small comment. I think it's good to eliminate these now. I haven't seen many cases where they're super useful. --- If your project i

[GitHub] spark pull request: [SPARK-1745] Move interrupted flag from TaskCo...

2014-05-11 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/675#issuecomment-42592936 LGTM, merging into master and 1.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: Update RoutingTable.scala

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/647#issuecomment-42465781 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-1778] [SQL] Add 'limit' transformation ...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/711#issuecomment-42642514 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1754] [SQL] Add missing arithmetic DSL ...

2014-05-11 Thread ueshin
Github user ueshin commented on the pull request: https://github.com/apache/spark/pull/689#issuecomment-42516176 Oops, I'll add it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: SPARK-571: forbid return statements in cleaned...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/717#issuecomment-42715653 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [RFC] SPARK-1772 Stop catching Throwable, let ...

2014-05-11 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/715#discussion_r12514419 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala --- @@ -71,7 +71,7 @@ private[spark] class PythonWorkerFactory(pythonExe

[GitHub] spark pull request: [SPARK-1745] Move interrupted flag from TaskCo...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/675#discussion_r12406092 --- Diff: core/src/main/scala/org/apache/spark/TaskContext.scala --- @@ -58,6 +60,6 @@ class TaskContext( def executeOnCompleteCallbacks() {

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/636#issuecomment-42504003 @mridulm @lianhuiwang thanks for the comments, I addressed all of them and now it should be correct --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: support leftsemijoin for sparkSQL

2014-05-11 Thread adrian-wang
Github user adrian-wang commented on the pull request: https://github.com/apache/spark/pull/395#issuecomment-42795884 I'll switch to a newer branch with #418 to split leftsemi from other joins. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: Fixed streaming examples docs to use run-examp...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/722#discussion_r12514206 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/CustomReceiver.scala --- @@ -30,32 +30,27 @@ import org.apache.spark.streaming.receiv

[GitHub] spark pull request: Fixed streaming examples docs to use run-examp...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/722#discussion_r12514192 --- Diff: examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala --- @@ -31,8 +31,7 @@ import org.apache.spark.storage.StorageL

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42793347 My only concern is that I would prefer things work slowly than fail. With reference tracking disabled it is not possible to serialize user defined types from the spark-s

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/727#issuecomment-42787858 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: Improve build configuration � �

2014-05-11 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/590#issuecomment-42792564 @pwendell Big changes have been removed. The PR can be merged into master and branch-1.0. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: SPARK-1577: Enabling reference tracking by def...

2014-05-11 Thread jegonzal
Github user jegonzal closed the pull request at: https://github.com/apache/spark/pull/499 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42791693 Alternatively found a way to work around that in the repl so it can safely turned on. On Sunday, May 11, 2014, Matei Zaharia wrote: > Alright, then I'll

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42791799 I think we can warn if it's on or something. I wouldn't add code to disable it since we might be able to fix it to work there too. --- If your project is set up for it, yo

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42791672 btw as far as I can tell Kryo reference should always be disabled in the spark repl. Should we just do that in the future? On Sunday, May 11, 2014, Matei Zaharia

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/724 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: Fix error in 2d Graph Partitioner

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/709 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42791250 This looks good to me. Re-enabling Kryo reference tracking will have a performance penalty, but we can easily fix that after the release. --- If your project is set up

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42791443 Alright, then I'll merge this as is. You guys should add some docs in both the GraphX programming guide and GraphXKryoSerializer to mention that it's recommended to turn of

[GitHub] spark pull request: SPARK-1577: Enabling reference tracking by def...

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/499#issuecomment-42791416 Actually it looks like this will be subsumed by https://github.com/apache/spark/pull/724. You should close this pull request, as GitHub won't automatically close it. ---

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42791203 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14895/ --- If your project

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42791202 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-1631] Correctly set the Yarn app name w...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/539#issuecomment-42513184 @vanzin Your proposal about having SparkSubmit calling `System.setProperty(spark.app.name)` can be made clean if we just always convert `--name` to `spark.app.name`, wh

[GitHub] spark pull request: Unify GraphImpl RDDs + other graph load optimi...

2014-05-11 Thread ankurdave
Github user ankurdave commented on a diff in the pull request: https://github.com/apache/spark/pull/497#discussion_r12456744 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/impl/RoutingTablePartition.scala --- @@ -0,0 +1,158 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request: [Docs] Warn about PySpark on YARN on Red Hat

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/682#issuecomment-42478442 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14782/ --- If your project is set up for it, you can r

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12512000 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -466,30 +466,14 @@ private[spark] class Master( * launched an executor

[GitHub] spark pull request: Fix for SPARK-1758: failing test org.apache.sp...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/692#issuecomment-42519649 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: Unify GraphImpl RDDs + other graph load optimi...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/497#issuecomment-42483842 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1470] Spark logger moving to use scala-...

2014-05-11 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/332#discussion_r12512463 --- Diff: project/SparkBuild.scala --- @@ -317,6 +317,7 @@ object SparkBuild extends Build { val excludeFastutil = ExclusionRule(organization = "it.unimi.d

[GitHub] spark pull request: [SPARK-1470] Spark logger moving to use scala-...

2014-05-11 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/332#discussion_r12512415 --- Diff: core/src/main/scala/org/apache/spark/Logging.scala --- @@ -116,7 +121,8 @@ trait Logging { val log4jInitialized = LogManager.getRootLogger.get

[GitHub] spark pull request: Nicer logging for SecurityManager startup

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/678 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: Fixing typo in als.py

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/696#issuecomment-42580069 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Fix error in 2d Graph Partitioner

2014-05-11 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/709#issuecomment-42703154 @rxin and @ankurdave take a look at this minor change when you get a chance. I would like to get it into the next release if possible. --- If your project is set up fo

[GitHub] spark pull request: SPARK-1668: Add implicit preference as an opti...

2014-05-11 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/597#issuecomment-42404618 @mengxr Here are few results ``` implicitPref rank numInterations lambda -> rmse true 10 20 1.0 -> 0.5985187619423589 true

[GitHub] spark pull request: [Docs] Update YARN docs

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/701#issuecomment-42617518 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14826/ --- If your project

[GitHub] spark pull request: [SPARK-1157][MLlib] Bug fix: lossHistory shoul...

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/582 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: SPARK-1686: keep schedule() calling in the mai...

2014-05-11 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/639#issuecomment-42715116 @markhamstra Absolutely agree. @CodingCat The test failure is unrelated, I submitted #716 to fix it. Had one last minor comment, other than that LGTM. --- If yo

[GitHub] spark pull request: Fix for SPARK-1758: failing test org.apache.sp...

2014-05-11 Thread techaddict
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/691#issuecomment-42520460 Why make pull requests for brach-1.0 and master both? I think #692 should be the only one. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [Spark-1461] Deferred Expression Evaluation (s...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/446#issuecomment-42713579 Build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [Docs] Warn about PySpark on YARN on Red Hat

2014-05-11 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/682#issuecomment-42469878 I took a quick pass at the latest docs, and it looks like for 1.0+ we only mention maven when we talk about building. I wonder if we should still document the requireme

[GitHub] spark pull request: SPARK-1565, update examples to be used with sp...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/552#issuecomment-42517525 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/636#issuecomment-42536702 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-1395] Fix "local:" URI support in Yarn ...

2014-05-11 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/560#issuecomment-42463814 Just rebased on top of master. No changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: SPARK-1416: PySpark support for SequenceFile a...

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/455#issuecomment-42790245 Hey Nick, sorry I still haven't looked much at this, been delayed with other 1.0 stuff. I'll get to it when I can though (or get someone else to try it). --- If your proj

[GitHub] spark pull request: SPARK-1577: Enabling reference tracking by def...

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/499#issuecomment-42790193 @jegonzal @rxin is this still needed for GraphX to work in the shell in 1.0 or do you guys have a workaround? --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42789960 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: Fix error in 2d Graph Partitioner

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/709#issuecomment-42789961 Going to merge this, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42789966 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/730 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42789924 Jenkins, this is ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: Feat kryo max buffersize

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/735#issuecomment-42789917 Hey, so is this a new feature that was recently added to Kryo? Seems super useful, but in this case, I'd actually make the max buffer size higher by default. Or we can use

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42789506 LGTM, merged into master and branch-1.0. Thanks!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12512027 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -532,6 +516,99 @@ private[spark] class Master( } } +

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/727#issuecomment-42788831 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14894/ --- If your project

[GitHub] spark pull request: SPARK-1487 [SQL] Support record filtering via ...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/511#issuecomment-42737719 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/727#issuecomment-42788830 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42788446 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14892/ --- If your project

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42788445 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/727 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: [SPARK-1779] add warning when memoryFraction i...

2014-05-11 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/714#discussion_r12511779 --- Diff: core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala --- @@ -76,6 +76,12 @@ class ExternalAppendOnlyMap[K, V, C](

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42787495 @aarondav check again? Responded to your feedback. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/727#issuecomment-42787822 Looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511553 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -532,6 +516,99 @@ private[spark] class Master( } } +

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/727#issuecomment-42787862 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Synthetic GraphX Benchmark

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/720#issuecomment-42787533 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14893/ --- If your project is set up for it, you can r

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511614 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -532,6 +516,99 @@ private[spark] class Master( } }

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511602 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -466,30 +466,14 @@ private[spark] class Master( * launched an executor

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/727#discussion_r12511646 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -202,6 +202,39 @@ class RDDSuite extends FunSuite with SharedSparkContext { as

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511647 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -466,30 +466,14 @@ private[spark] class Master( * launched an execut

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/727#discussion_r12511628 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -328,11 +328,20 @@ abstract class RDD[T: ClassTag]( def coalesce(numPartitions: Int, s

[GitHub] spark pull request: Synthetic GraphX Benchmark

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/720#issuecomment-42787532 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1770: Load balance elements when reparti...

2014-05-11 Thread mateiz
Github user mateiz commented on a diff in the pull request: https://github.com/apache/spark/pull/727#discussion_r12511624 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -328,11 +328,20 @@ abstract class RDD[T: ClassTag]( def coalesce(numPartitions: Int, s

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42787516 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Synthetic GraphX Benchmark

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/720#issuecomment-42787518 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Synthetic GraphX Benchmark

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/720#issuecomment-42787512 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42787511 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511580 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -532,6 +516,99 @@ private[spark] class Master( } } +

[GitHub] spark pull request: SPARK-1786: Edge Partition Serialization

2014-05-11 Thread jegonzal
Github user jegonzal commented on the pull request: https://github.com/apache/spark/pull/724#issuecomment-42787343 I would like to get it into 1.0 if possible. Otherwise, we could run into issues if the user persists graphs to disk or straggler mitigation is used. @ankurdave do you s

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread CodingCat
Github user CodingCat commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511545 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -466,30 +466,14 @@ private[spark] class Master( * launched an execut

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511541 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -532,6 +516,99 @@ private[spark] class Master( } } +

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/731#discussion_r12511528 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -466,30 +466,14 @@ private[spark] class Master( * launched an executor

[GitHub] spark pull request: [WIP] Simplify the build with sbt 0.13.2 featu...

2014-05-11 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/706#issuecomment-42786776 Here is a link to our mailing list discussion on the topic: http://apache-spark-developers-list.1001551.n3.nabble.com/DISCUSS-Necessity-of-Maven-and-SBT-Build-in-Spark

[GitHub] spark pull request: SPARK-1706: Allow multiple executors per worke...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/731#issuecomment-42774786 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/730#discussion_r12511378 --- Diff: bin/spark-submit --- @@ -35,8 +35,10 @@ while (($#)); do shift done -if [ ! -z $DRIVER_MEMORY ] && [ ! -z $DEPLOY_MODE ] && [ $

[GitHub] spark pull request: SPARK-1791 - SVM implementation does not use t...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/725#issuecomment-42757678 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1652: Set driver memory correctly in spa...

2014-05-11 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/730#issuecomment-42783193 @aarondav - would never want to make you nervous. I made the suggested change. Mind taking a look? --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-1690] Allow empty lines in PythonRDD

2014-05-11 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/644#issuecomment-42507072 @mateiz just realized I could test it from Python side. Added a doctest. This makes Python API behaves identical to Scala API. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-1755] Respect SparkSubmit --name on YAR...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/699#issuecomment-42596162 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SQL] Fix Performance Issue in data type casti...

2014-05-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/679#issuecomment-42409179 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14766/ --- If your project

  1   2   3   >