[GitHub] spark issue #21150: [SPARK-24075][MESOS] Option to limit number of retries f...

2018-10-22 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/21150 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21150: [SPARK-24075][MESOS] Option to limit number of re...

2018-10-22 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21150#discussion_r227028091 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -728,6 +729,28 @@ private

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-09-12 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/22146 The PR works for me now as well for adding volumes to executors. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-30 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r214205873 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -59,5 +66,28 @@ private[spark] object

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-08-29 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/22146 I've been looking to mount additional volumes to the executor containers and just tried out the PR. It doesn't seem possible since if you add the container in the pod template

[GitHub] spark issue #22071: [SPARK-25088][CORE][MESOS][DOCS] Update Rest Server docs...

2018-08-13 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/22071 LGTM as well --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22071: [SPARK-25088][CORE][MESOS][DOCS] Update Rest Serv...

2018-08-13 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/22071#discussion_r209714362 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala --- @@ -51,6 +51,13 @@ private[mesos] class

[GitHub] spark pull request #22071: [SPARK-25088][CORE][MESOS][DOCS] Update Rest Serv...

2018-08-12 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/22071#discussion_r209464703 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcher.scala --- @@ -51,6 +51,13 @@ private[mesos] class

[GitHub] spark pull request #21027: [SPARK-23943][MESOS][DEPLOY] Improve observabilit...

2018-08-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21027#discussion_r208468846 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala --- @@ -63,6 +63,8 @@ private[spark] abstract class

[GitHub] spark pull request #21027: [SPARK-23943][MESOS][DEPLOY] Improve observabilit...

2018-08-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21027#discussion_r208468697 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/rest/mesos/MesosRestServer.scala --- @@ -50,6 +50,24 @@ private[spark] class

[GitHub] spark pull request #21027: [SPARK-23943][MESOS][DEPLOY] Improve observabilit...

2018-08-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21027#discussion_r208468594 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala --- @@ -331,3 +345,15 @@ private class ErrorServlet extends

[GitHub] spark pull request #21027: [SPARK-23943][MESOS][DEPLOY] Improve observabilit...

2018-08-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21027#discussion_r208468305 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala --- @@ -63,6 +63,8 @@ private[spark] abstract class

[GitHub] spark pull request #21027: [SPARK-23943][MESOS][DEPLOY] Improve observabilit...

2018-08-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21027#discussion_r208467704 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -160,7 +161,10 @@ trait

[GitHub] spark issue #21006: [SPARK-22256][MESOS] - Introduce spark.mesos.driver.memo...

2018-08-01 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/21006 Besides the test comment, everything else LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21006: [SPARK-22256][MESOS] - Introduce spark.mesos.driv...

2018-08-01 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21006#discussion_r207060617 --- Diff: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala --- @@ -199,6 +200,33 @@ class

[GitHub] spark issue #21006: [SPARK-22256][MESOS] - Introduce spark.mesos.driver.memo...

2018-08-01 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/21006 jenkins ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20451: [SPARK-23146][WIP] Support client mode for Kubern...

2018-07-04 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/20451#discussion_r200020286 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -18,43 +18,68

[GitHub] spark pull request #20451: [SPARK-23146][WIP] Support client mode for Kubern...

2018-07-01 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/20451#discussion_r199340795 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -140,13 +140,6

[GitHub] spark pull request #20451: [SPARK-23146][WIP] Support client mode for Kubern...

2018-07-01 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/20451#discussion_r199340757 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -88,6 +103,56 @@ private

[GitHub] spark pull request #20451: [SPARK-23146][WIP] Support client mode for Kubern...

2018-07-01 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/20451#discussion_r199340748 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -88,6 +103,56 @@ private

[GitHub] spark issue #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard limit o...

2018-04-12 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/21033 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard ...

2018-04-12 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/21033#discussion_r181199842 --- Diff: resource-managers/mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala --- @@ -165,18

[GitHub] spark issue #17714: [SPARK-20428][Core]REST interface about 'v1/submissions/...

2017-06-01 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17714 Unfortunately I'm not a committer, so need to loop in someone who is to help merge it though. @srowen do you know who's responsible for the general deploy package? --- If your project is set up

[GitHub] spark pull request #17714: [SPARK-20428][Core]REST interface about 'v1/submi...

2017-04-30 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/17714#discussion_r114089418 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionServer.scala --- @@ -214,15 +214,15 @@ private[rest] abstract class

[GitHub] spark pull request #17109: [SPARK-19740][MESOS]Add support in Spark to pass ...

2017-04-15 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/17109#discussion_r111670608 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -99,6 +99,26 @@ private

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-04-11 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17109 @srowen Appreciate the help you're doing, I think we're doing what we can to help review these patches and making sure Mesos support is still being maintained and improved over time. If you

[GitHub] spark pull request #17109: [SPARK-19740][MESOS]Add support in Spark to pass ...

2017-03-14 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/17109#discussion_r106053115 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -99,6 +99,26 @@ private

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-03-09 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17109 @srowen @mgummelt PTAL --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-03-09 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17109 Hey sorry for the late response, code looks good to me, however we need to add documentation about the new flag. Can you modify the Mesos configuration docs in the docs folder? --- If your

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-02-28 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17109 @yanji84 can you add a test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17109: [SPARK-19740][MESOS]Add support in Spark to pass arbitra...

2017-02-28 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/17109 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13072: [SPARK-15288] [Mesos] Mesos dispatcher should handle gra...

2017-01-22 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13072 LGTM, @srowen can you please take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-12-06 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 @devaraj-kavali let us know if you can still update this, otherwise I'll close this as it's no longer being updated. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-12-06 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 @srowen can you help review this? Besides my minor comment overall it looks fine with me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #12933: [Spark-15155][Mesos] Optionally ignore default ro...

2016-12-06 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/12933#discussion_r91195201 --- Diff: mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -50,6 +50,44 @@ trait MesosSchedulerUtils extends

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-12-06 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14936 @philipphoffmann Sorry for the long delay, one last ask. Can you add a simple unit test to verify it works? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15684: [SPARK-18171][MESOS] Show correct framework address in m...

2016-12-06 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/15684 LGTM, @srowen can you help on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16092: [SPARK-18662] Move resource managers to separate directo...

2016-12-01 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/16092 Changed suggested by @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16061: [SPARK-18278] [Scheduler] Support native submission of s...

2016-11-29 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/16061 @rxin Makes sense, @srowen also talked about starting the discussion of having a better support for external cluster managers as well. --- If your project is set up for it, you can reply

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90063528 --- Diff: kubernetes/src/main/scala/org/apache/spark/scheduler/cluster/kubernetes/KubernetesClusterScheduler.scala --- @@ -0,0 +1,236

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90063456 --- Diff: kubernetes/src/main/scala/org/apache/spark/scheduler/cluster/kubernetes/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1,222

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90063380 --- Diff: kubernetes/src/main/scala/org/apache/spark/scheduler/cluster/kubernetes/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1,222

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90062695 --- Diff: dev/make-distribution.sh --- @@ -154,7 +154,9 @@ export MAVEN_OPTS="${MAVEN_OPTS:--Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCac #

[GitHub] spark pull request #16061: [SPARK-18278] [Scheduler] Support native submissi...

2016-11-29 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/16061#discussion_r90062639 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -596,6 +599,26 @@ object SparkSubmit extends CommandLineUtils

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-10-13 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 I ran `mvn test` inside of the mesos folder. On Thu, Oct 13, 2016 at 3:21 AM, Chris Heller <notificati...@github.com> wrote: > You saw the error with `./dev/run-tests`

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-10-12 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 I just tried running it locally and I'm getting the same error. It seems like with your change that test is simply declining the offer. --- If your project is set up for it, you can reply

[GitHub] spark pull request #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch ...

2016-10-10 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13713#discussion_r82695978 --- Diff: docs/running-on-mesos.md --- @@ -506,8 +506,13 @@ See the [configuration page](configuration.html) for information on Spark config since

[GitHub] spark issue #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch cache i...

2016-10-10 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13713 Other than the 2 comments, the changes LGTM. @mgummelt @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch ...

2016-10-10 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13713#discussion_r82695810 --- Diff: mesos/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackendSuite.scala --- @@ -463,6 +463,21 @@ class

[GitHub] spark issue #14936: [SPARK-7877][MESOS] Allow configuration of framework tim...

2016-10-10 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14936 Hmm it is currently Integer.MAX because we assume the cluster scheduler to be long living, and without setting it be a long value Mesos will automatically terminate that framework when it's

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-10-09 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 I just tested it with a GPU instance and it works. @mgummelt @klueska any more comments? otherwise @srowen I think we should merge as we no longer have any comments --- If your project is set up

[GitHub] spark issue #9287: SPARK-11326: Split networking in standalone mode

2016-10-08 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/9287 This has been stale for a while, we should close this if there is no update here. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-10-08 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 @hellertime Are you able to rebase? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-10-08 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 @devaraj-kavali Are you still able to update this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch cache i...

2016-10-08 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13713 @drcrallen Are you still planning to update this? It's quite a useful feature, so hoping this can get in. Also since Fine grain mode is depcreated I don't think we need to update it too

[GitHub] spark pull request #14936: [SPARK-7877][MESOS] Allow configuration of framew...

2016-10-07 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14936#discussion_r82430755 --- Diff: mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -69,38 +68,51 @@ trait MesosSchedulerUtils extends

[GitHub] spark pull request #14936: [SPARK-7877][MESOS] Allow configuration of framew...

2016-10-07 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14936#discussion_r82430668 --- Diff: mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -69,38 +68,51 @@ trait MesosSchedulerUtils extends

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-09-29 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 Ya, the default GPU requirement I have is 0 (cores per executor/node is 1). I'm still gathering feedback what's the more sensible thing to do for GPUs. We can either set a configurable amount

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-09-29 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 1. Good catch, my old patch had docs but I rebased and it didn't apply for some reason. Let me add it. 2,3: we don't fail if you ask for more GPUs since it's not a hard requirement

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-09-28 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 @mgummelt @srowen Please review as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14644: [SPARK-14082][MESOS] Enable GPU support with Mesos

2016-09-22 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 @klueska Just updated the patch and I think it's using the right semantics now, where it has a global gpus max just like cores. Can you try it out? --- If your project is set up for it, you can

[GitHub] spark pull request #14644: [MESOS] Enable GPU support with Mesos

2016-09-12 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14644#discussion_r78348094 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -103,6 +103,7 @@ private[spark

[GitHub] spark pull request #14644: [MESOS] Enable GPU support with Mesos

2016-09-11 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14644#discussion_r78298417 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -103,6 +103,7 @@ private[spark

[GitHub] spark pull request #14644: [MESOS] Enable GPU support with Mesos

2016-09-08 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14644#discussion_r78002761 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -103,6 +103,7 @@ private[spark

[GitHub] spark issue #14644: [MESOS] Enable GPU support with Mesos

2016-08-15 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/14644 @srowen Mesos also supports node labels as well (which is how constraints is implemented in Spark framework). However GPUs are implemented as a resource (as we want to account for # of GPUs instead

[GitHub] spark pull request #14644: Enable GPU support with Mesos

2016-08-15 Thread tnachen
GitHub user tnachen opened a pull request: https://github.com/apache/spark/pull/14644 Enable GPU support with Mesos ## What changes were proposed in this pull request? Enable GPU resources to be used when running coarse grain mode with Mesos. ## How

[GitHub] spark issue #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch cache i...

2016-07-27 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13713 We also fetch URIs for running drivers in cluster mode (MesosClusterScheduler.scala). I'm thinking we should also allow this configuration to effect that too. --- If your project is set up

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-07-24 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-07-24 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13051 @srowen Or if you could help :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #14275: [SPARK-16637] Unified containerizer

2016-07-24 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/14275#discussion_r72003350 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -105,16 +105,27 @@ private[mesos] object

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-07-06 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r69690304 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -502,6 +502,9 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-07-06 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r69678895 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -502,6 +502,9 @@ class SparkContext(config: SparkConf) extends Logging

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-07-05 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r69676891 --- Diff: core/src/main/scala/org/apache/spark/ui/JettyUtils.scala --- @@ -186,6 +188,67 @@ private[spark] object JettyUtils extends Logging

[GitHub] spark pull request #13950: [SPARK-15487] [Web UI] Spark Master UI to reverse...

2016-07-05 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13950#discussion_r69676869 --- Diff: docs/configuration.md --- @@ -598,6 +598,20 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #13143: [SPARK-15359] [Mesos] Mesos dispatcher should han...

2016-06-23 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13143#discussion_r68194509 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -120,14 +120,25 @@ private[mesos] trait

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 @hellertime ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13072: [SPARK-15288] [Mesos] Mesos dispatcher should handle gra...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13072 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13143 Is it because MesosDriver actually threw an exception? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13143: [SPARK-15359] [Mesos] Mesos dispatcher should han...

2016-06-21 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13143#discussion_r67988954 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -120,14 +120,25 @@ private[mesos] trait

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 @devaraj-kavali Ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13326: [SPARK-15560] [Mesos] Queued/Supervise drivers waiting f...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13326 jenkins please retest --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13713: [SPARK-15994] [MESOS] Allow enabling Mesos fetch cache i...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13713 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13715: [SPARK-15992] [MESOS] Refactor MesosCoarseGrained...

2016-06-21 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13715#discussion_r67988575 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -382,59 +382,97 @@ private[spark

[GitHub] spark issue #13715: [SPARK-15992] [MESOS] Refactor MesosCoarseGrainedSchedul...

2016-06-21 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13715 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13323: [SPARK-15555] [Mesos] Driver with --supervise option can...

2016-06-09 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13323 Thanks @devaraj-kavali, this LGTM. @andrewor14 can you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13051 @andrewor14 PTAL, this PR LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #12933: [Spark-15155][Mesos] Optionally ignore default role reso...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/12933 @hellertime Can you rebase and submit again to rerun the tests? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #10949: [SPARK-12832][MESOS] mesos scheduler respect agent attri...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/10949 @atongen please rebase and try again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 I think instead of just moving to finished drivers, can we also show that message on the UI? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #13077: [SPARK-10748] [Mesos] Log error instead of crashing Spar...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13077 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13323: [SPARK-15555] [Mesos] Driver with --supervise option can...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13323 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13143 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13051: [SPARK-15271] [MESOS] Allow force pulling executor docke...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13051 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13323: [SPARK-15555] [Mesos] Driver with --supervise option can...

2016-06-02 Thread tnachen
Github user tnachen commented on the issue: https://github.com/apache/spark/pull/13323 Nice catch, can you add a unit test to test this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13326: [SPARK-15560] [Mesos] Queued/Supervise drivers wa...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/13326#discussion_r65580076 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -188,10 +188,10 @@ private[spark] class

[GitHub] spark pull request #11887: [SPARK-13041][Mesos]add driver sandbox uri to the...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/11887#discussion_r65579886 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/ui/MesosClusterPage.scala --- @@ -115,4 +142,58 @@ private[mesos] class MesosClusterPage(parent

[GitHub] spark pull request #11887: [SPARK-13041][Mesos]add driver sandbox uri to the...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/11887#discussion_r65579463 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/ui/MesosClusterPage.scala --- @@ -68,6 +75,25 @@ private[mesos] class MesosClusterPage(parent

[GitHub] spark pull request #11887: [SPARK-13041][Mesos]add driver sandbox uri to the...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/11887#discussion_r65579454 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/ui/MesosClusterPage.scala --- @@ -68,6 +75,25 @@ private[mesos] class MesosClusterPage(parent

[GitHub] spark pull request #11887: [SPARK-13041][Mesos]add driver sandbox uri to the...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/11887#discussion_r65579412 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/ui/MesosClusterPage.scala --- @@ -68,6 +75,25 @@ private[mesos] class MesosClusterPage(parent

[GitHub] spark pull request #11887: [SPARK-13041][Mesos]add driver sandbox uri to the...

2016-06-02 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/11887#discussion_r65579276 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/ui/MesosClusterPage.scala --- @@ -68,6 +75,25 @@ private[mesos] class MesosClusterPage(parent

  1   2   3   4   5   6   7   >