[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-08-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-52146688 QA tests have started for PR 1321. This patch merges cleanly. View progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18523/consoleFull --- If

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-08-13 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-52146322 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-08-13 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-52146296 Now I think about it more. LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-08-13 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-52146302 :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-15 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-49002113 I think it makes more sense if you can't run a command than certain commands happen to be runnable while there are no cluster resources. This sort of execution puts more

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-15 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48999161 When the cluster is busy and backlogged ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project d

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-15 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48999051 @rxin is there a case where you think local execution will yield a relevant performance improvement? I don't see why shopping a task for a few milliseconds is a bit deal

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-08 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48280965 Maybe we should also solve the problem that local execution should not transfer the whole in-memory block (as a matter of fact, perhaps local execution should just bypass th

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48250513 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48250515 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16384/ --- If your project is set up for it, you can

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48242208 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1321#issuecomment-48242188 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [RFC] Disable local execution of Spark jobs by...

2014-07-07 Thread aarondav
GitHub user aarondav opened a pull request: https://github.com/apache/spark/pull/1321 [RFC] Disable local execution of Spark jobs by default Currently, local execution of Spark jobs is only used by take(), and it can be problematic as it can load a significant amount of data onto th