[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4747 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25379656 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -63,6 +63,7 @@ private[spark] object Utils extends Logging { val random = new Rand

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25365970 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -345,11 +345,11 @@ private[spark] class Worker( }

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-25 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25366051 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -728,6 +746,11 @@ private[spark] object Utils extends Logging { localDirs

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25331165 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala --- @@ -345,11 +345,11 @@ private[spark] class Worker( }

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread Sephiroth-Lin
Github user Sephiroth-Lin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25322767 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -728,6 +746,11 @@ private[spark] object Utils extends Logging { localDirs

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75864366 [Test build #27909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27909/consoleFull) for PR 4747 at commit [`e0114e1`](https://gith

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75864378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75863854 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75863845 [Test build #27908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27908/consoleFull) for PR 4747 at commit [`18ee0a7`](https://gith

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75851747 [Test build #27909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27909/consoleFull) for PR 4747 at commit [`e0114e1`](https://githu

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75850846 [Test build #27908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27908/consoleFull) for PR 4747 at commit [`18ee0a7`](https://githu

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75850649 retest this please. We really need to fix the `JavaAPISuite` at some point. --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75843342 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75843044 Hold off a little bit on this; let me see how to handle the SparkConf issue (I think Josh added that for the unit tests). --- If your project is set up for it, you can re

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75843329 [Test build #27903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27903/consoleFull) for PR 4747 at commit [`18ee0a7`](https://gith

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25290402 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -679,12 +680,25 @@ private[spark] object Utils extends Logging { * If no director

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25290358 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -679,12 +680,25 @@ private[spark] object Utils extends Logging { * If no director

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25290073 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -679,12 +680,25 @@ private[spark] object Utils extends Logging { * If no director

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25289755 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala --- @@ -135,7 +135,7 @@ private[spark] class ExecutorRunner( logIn

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25289178 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala --- @@ -135,7 +135,7 @@ private[spark] class ExecutorRunner( logIn

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4747#discussion_r25288970 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala --- @@ -135,7 +135,7 @@ private[spark] class ExecutorRunner( logIn

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4747#issuecomment-75833135 [Test build #27903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27903/consoleFull) for PR 4747 at commit [`18ee0a7`](https://githu

[GitHub] spark pull request: [SPARK-5801] [core] Avoid creating nested dire...

2015-02-24 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/4747 [SPARK-5801] [core] Avoid creating nested directories. Cache the value of the local root dirs to use for storing local data, so that the same directories are reused. Also, to avoid an ext