Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/16061
The diff showing our progress so far has moved to
https://github.com/apache-spark-on-k8s/spark/pull/200 for anyone following
along.
---
If your project is set up for it, you can reply to this
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/16061
+1 I think we should close this PR to avoid confusion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/16061
Work on this feature has moved to
https://github.com/apache-spark-on-k8s/spark. The exact diff we are working
with is this: https://github.com/apache-spark-on-k8s/spark/pull/1. Feel free to
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16061
We should acknowledge people that have done this in the merged commit, but
I don't think it'd make a lot of sense to do a merge rather than squash. The
initial Spark SQL commit was a single commit
Github user erikerlandson commented on the issue:
https://github.com/apache/spark/pull/16061
@iyanuobidele the current head of this branch should be equivalent, but it
can't hurt to rebase just in case.
@foxish yesterday occurred to me that it would've been sensible set up a
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/16061
I would think that it would be easier if we continue to keep PRs and issues
in foxish/spark since we have the previous PRs and issues over there. It should
be easy enough to pick those commits up
Github user iyanuobidele commented on the issue:
https://github.com/apache/spark/pull/16061
@erikerlandson are there any new commits to this branch ? Or it's at the
same level as @foxish's k8s-support. In case I need to rebase
---
If your project is set up for it, you can reply
Github user erikerlandson commented on the issue:
https://github.com/apache/spark/pull/16061
At the risk of redundant announcements, I am going to present this
spark-on-kube topic at the [Kubernetes
SIG-Apps](https://github.com/kubernetes/community/tree/master/sig-apps) next
Monday
Github user erikerlandson commented on the issue:
https://github.com/apache/spark/pull/16061
Since we have at least four people with interest in working on the code, I
have a workflow proposal, which is that we all submit candidate updates to
#16061 in the form of PRs against
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/16061
Cross posting link to [the
proposal](https://issues.apache.org/jira/secure/attachment/12840988/SPARK-18278%20-%20Spark%20on%20Kubernetes%20Design%20Proposal.pdf)
here.
---
If your project is set
Github user ash211 commented on the issue:
https://github.com/apache/spark/pull/16061
Another external scheduler backend I'm aware of is Two Sigma's scheduler
backend for the system they've created called
[Cook](https://github.com/twosigma/Cook). See
Github user erikerlandson commented on the issue:
https://github.com/apache/spark/pull/16061
@rxin, when you say "move all resource managers" does that mean "move
scheduler back-ends for mesos, yarn, etc, into some `resource-managers`
sub-project" ?
---
If your project is set up
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/16061
@erikerlandson For the RAT failure, you may either add Apache license
header to newly added files or add the file to `dev/.rat-excludes`.
---
If your project is set up for it, you can reply to
Github user tnachen commented on the issue:
https://github.com/apache/spark/pull/16061
@rxin Makes sense, @srowen also talked about starting the discussion of
having a better support for external cluster managers as well.
---
If your project is set up for it, you can reply to this
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16061
One thing - can we submit a separate pr to move all resource managers into
resource-managers/yarn
resource-managers/mesos
?
---
If your project is set up for it, you can reply
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16061
This is pretty cool.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16061
**[Test build #69335 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69335/consoleFull)**
for PR 16061 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16061
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16061
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69335/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16061
**[Test build #69335 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69335/consoleFull)**
for PR 16061 at commit
20 matches
Mail list logo
