It sounds like there's a discussion about the details coming, which is fine and good. That should maybe also have a VOTE. The debate here is then merely about what and when to call things a SPIP, but that's not important.
On Mon, Mar 4, 2019 at 10:23 AM Xiangrui Meng <men...@gmail.com> wrote: > I think the two requires more discussion are Mesos and K8s. Let me follow > what I suggested above and try to answer two questions for each: > > Mesos: > * Is it important? There are certainly Spark/Mesos users but the overall > usage is going downhill. See the attached Google Trend snapshot. > * How to implement it? I believe it is doable, similarly to other cluster > managers. However, we need to find someone from our community to do the work. > If we cannot find such a person, it would indicate that the feature is not > that important. I don't think that was the issue that was raised; I don't advocate for investing more in supporting this cluster manager, myself. The issue was that we _already_ have support for allocating GPUs in Mesos. Whatever limited support is there, presumably, doesn't get removed. It merely needs to be attached to whatever new mechanisms are implemented. I only pushed back on the idea that it should be ignored and (presumably) left as a separate unrelated implementation. > You see that such discussions can be done in parallel. It is not efficient if > we block the work on K8s because we cannot decide whether we should support > Mesos. Is the question blocking anything? An answer is: let's say we just make whatever support in Mesos exists still works coherently with the new mechanism, whatever those details may be. Is there any disagreement on that out there? I agree with you in that I think it shouldn't have been ruled out at this stage, per earlier comments. This doesn't seem hard to answer as a question of scope even now. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
