Github user tedyu commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59991684
--- Diff:
core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala
---
@@ -140,6 +140,13 @@ private[spark] class
Github user tedyu commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59988347
--- Diff:
core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala
---
@@ -140,6 +140,13 @@ private[spark] class
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210971481
@rxin I will open another JIRA and a PR to do this. Thanks for the review.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210966727
One thing - can you guys try to see if you can implement one of the
existing cluster manager with this, and then we can make sure this is a proper
API? Otherwise it is
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11723
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210966505
Merging in master. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59979061
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210876884
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210876883
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210876799
**[Test build #56012 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56012/consoleFull)**
for PR 11723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210861519
**[Test build #56012 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56012/consoleFull)**
for PR 11723 at commit
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210861478
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210861480
Jenkins add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210861458
@rxin how do I get this retested by Jenkins? There were few issues going on
with the Jenkins when I checked in my last changes and now it is not retesting
it?
---
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210783316
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210780609
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210780610
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210780561
**[Test build #55994 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55994/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210759998
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210759995
**[Test build #55998 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55998/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210759997
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210758869
**[Test build #55998 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55998/consoleFull)**
for PR 11723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210758141
**[Test build #55997 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55997/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210758156
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210758155
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210757877
**[Test build #55997 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55997/consoleFull)**
for PR 11723 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210756561
The latest changes LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210756486
there are some conflicts with master - can you rebase? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210754302
**[Test build #55994 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55994/consoleFull)**
for PR 11723 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-210295660
Sorry for the delay. This looks pretty good. Just have some comments about
the style to be more consistent with rest of the Spark codebase.
---
If your project is set up
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59829119
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59829110
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59829094
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59829070
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59829023
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2443,8 +2443,34 @@ object SparkContext extends Logging {
"in the form
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59828906
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2443,8 +2443,34 @@ object SparkContext extends Logging {
"in the form
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59828857
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59828785
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -150,6 +150,15 @@ private[spark] class Executor(
}
}
+
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r59828777
--- Diff:
core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala
---
@@ -135,6 +135,8 @@ private[spark] class
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-208235009
@rxin can you please review this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user tejasapatil commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-206071824
Can someone review this PR ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user tejasapatil commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-203252858
sure. @rxin can you please review the PR ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-202692781
@tejasapatil if you are done with the review, can we ask @rxin to have a
look at this?
---
If your project is set up for it, you can reply to this email and have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-201033544
**[Test build #2682 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2682/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-201018123
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-201018124
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-201017746
**[Test build #54066 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54066/consoleFull)**
for PR 11723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200972944
**[Test build #2682 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2682/consoleFull)**
for PR 11723 at commit
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200961545
@rxin @tejasapatil the previous build failure is not related to my
checkin and looks like some other issue as other build requests failed with
the same exception.
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57365175
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57365126
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57365152
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200960541
**[Test build #54066 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54066/consoleFull)**
for PR 11723 at commit
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57364628
--- Diff: dev/.rat-excludes ---
@@ -98,3 +98,4 @@ LZ4BlockInputStream.java
spark-deps-.*
.*csv
.*tsv
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200914410
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200914260
**[Test build #54037 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54037/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200914401
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57337365
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57337311
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57337345
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57336395
--- Diff: dev/.rat-excludes ---
@@ -98,3 +98,4 @@ LZ4BlockInputStream.java
spark-deps-.*
.*csv
.*tsv
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309581
--- Diff: dev/.rat-excludes ---
@@ -98,3 +98,4 @@ LZ4BlockInputStream.java
spark-deps-.*
.*csv
.*tsv
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309531
--- Diff:
core/src/test/resources/META-INF/services/org.apache.spark.scheduler.ExternalClusterManager
---
@@ -0,0 +1 @@
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309422
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309392
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309305
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309258
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -149,7 +149,14 @@ private[spark] class Executor(
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309289
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57309250
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -149,7 +149,14 @@ private[spark] class Executor(
Github user hbhanawat commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57308967
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2443,8 +2443,34 @@ object SparkContext extends Logging {
"in the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200809005
**[Test build #54037 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54037/consoleFull)**
for PR 11723 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200704870
OK - once you are done with your own review ping me. I will take a look at
it again.
---
If your project is set up for it, you can reply to this email and have your
Github user tejasapatil commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200704530
@rxin : Yes !! At Facebook we are using an internal scheduler to run Spark
executors. Maintaining an internal patch to have that "glue" and merging it
against
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200693181
Is this something Facebook needs too?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user tejasapatil commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200687233
@rxin : I would really like to have this PR in trunk. As things stand, for
anyone using their own scheduler, one has to maintain a patch over open source
release
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276779
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276711
--- Diff: dev/.rat-excludes ---
@@ -98,3 +98,4 @@ LZ4BlockInputStream.java
spark-deps-.*
.*csv
.*tsv
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276691
--- Diff:
core/src/test/resources/META-INF/services/org.apache.spark.scheduler.ExternalClusterManager
---
@@ -0,0 +1 @@
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276663
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276660
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276618
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/ExternalClusterManagerSuite.scala
---
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276344
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/ExternalClusterManager.scala ---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276268
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -149,7 +149,14 @@ private[spark] class Executor(
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276201
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -149,7 +149,14 @@ private[spark] class Executor(
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276211
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -149,7 +149,14 @@ private[spark] class Executor(
Github user tejasapatil commented on a diff in the pull request:
https://github.com/apache/spark/pull/11723#discussion_r57276135
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2443,8 +2443,34 @@ object SparkContext extends Logging {
"in the
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-200329683
@rxin Any update? Any changes needed from my side?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-198256400
**[Test build #53525 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53525/consoleFull)**
for PR 11723 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-198300194
**[Test build #53525 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53525/consoleFull)**
for PR 11723 at commit
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-198313907
@rxin I have completed the changes. Please review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197852650
**[Test build #53424 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53424/consoleFull)**
for PR 11723 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197898594
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197898255
**[Test build #53424 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53424/consoleFull)**
for PR 11723 at commit
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197804928
@rxin ok, I get it. I would make ExternalClusterManager as private[spark]
and mark it as developer API. I hope that should suffice.
---
If your project is set up
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197435845
@hbhanawat I understand that. The problem is not whether you can find a
single legitimate use case.
The introduction of every API always benefit something --
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197898597
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197454384
@hbhanawat to be clear, I think we might be able to add this as a
semi-private API and external resource managers can use, but with the
understanding that this is tied to
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-198300403
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-198300407
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user hbhanawat commented on the pull request:
https://github.com/apache/spark/pull/11723#issuecomment-197221049
@rxin Thanks for commenting.
Spark was designed such that it is agnostic to the underlying cluster
manager (as long as it can acquire executor processes,
1 - 100 of 108 matches
Mail list logo
