Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149191952
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149191967
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149194446
**[Test build #43924 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43924/consoleFull)**
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149165087
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149170941
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149170932
**[Test build #43922 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43922/consoleFull)**
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149170942
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user Zariel commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149158601
@andrewor14 that sounds like a good idea, log it at INFO or WARN?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user Zariel commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r42352139
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149165022
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149169896
**[Test build #43922 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43922/consoleFull)**
for PR 8358 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149230001
**[Test build #43924 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43924/consoleFull)**
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149230336
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-149230337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8358
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r42298202
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
//
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r42297685
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
//
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r42297881
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
//
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r42297847
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
//
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-148859165
@Zariel This looks pretty good. Should we log a message or something if we
detect shuffle service AND `MESOS_DIRECTORY`? Otherwise it might be confusing
to the user
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r41986284
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r41940139
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-147883511
@Zariel I think we need to document this logic somewhere in the docs, but
otherwise I think we should merge this and make changes afterwards if needed.
And we'll need a
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r41939777
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user Zariel commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-147414856
@tnachen is there anything my end which needs to be done to merge this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144107346
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144107047
retest please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144107739
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144109685
[Test build #43095 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43095/consoleFull)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144107765
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144157031
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144157030
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-144156852
[Test build #43095 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/43095/console)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140504282
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140504285
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140504197
[Test build #42492 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42492/console)
for PR 8358 at commit
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140448556
@andrewor14 besides the settings check I think it LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r39543253
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140458120
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140458181
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user Zariel commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140457896
@tnachen done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8358#discussion_r39530770
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -655,15 +655,19 @@ private[spark] object Utils extends Logging {
// created
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-140459122
[Test build #42492 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42492/consoleFull)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139792266
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139792234
[Test build #42376 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42376/console)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139792267
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user Zariel commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751123
What should be the correct order for the local dirs? Currently as I can see
the priority is `YARN_LOCAL_DIRS > LOCAL_DIRS > SPARK_EXECUTOR_DIRS >
SPARK_LOCAL_DIRS >
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751439
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751440
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139770572
[Test build #42376 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42376/consoleFull)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751154
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751149
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751370
[Test build #42372 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42372/consoleFull)
for PR 8358 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139751437
[Test build #42372 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42372/console)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139769023
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-139769017
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-138825979
@dragos what previous parameters are you referring to? SPARK_EXECUTOR_DIRS?
I think this logic actually makes sense, as we (Mesos) set the
MESOS_DIRECTORY whenever
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-138833460
@tnachen I copy-pasted the Spark configuration docs in my previous
comments. Here's what I meant by other parameters though:
- `spark.local.dir` Directory to use
Github user Zariel commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-138854320
I dont think there is an issue with running Mesos and wanting to run over
multiple disks, as it is the responsibility of whoever is managing Mesos to
setup this space,
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-138915449
I think we should respect `SPARK_LOCAL_DIRS`, but fallback to
`MESOS_DIRECTORY` as a default (instead of `java.io.tmpdir`, as it currently
stands).
---
If your project
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136817295
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136820689
[Test build #41879 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41879/consoleFull)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136819229
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136819200
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136863615
[Test build #41879 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41879/console)
for PR 8358 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136863741
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136863744
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-136785979
I doubt this works with dynamic allocation under Mesos. It also completely
shuns the previous configuration parameters. Here's what docs say:
> spark.local.dir
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8358#issuecomment-133446766
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user Zariel opened a pull request:
https://github.com/apache/spark/pull/8358
[SPARK-9708] [MESOS] Spark should create local temporary directories in
Mesos sandbox when launched with Mesos
This is my own original work and I license this to the project under the
project's
70 matches
Mail list logo
