Re: Build customized resource manager

2019-11-10 Thread Klaus Ma
hm that'll be better to me if we can build customized resource manager out of core; otherwise, we have to go through the long discussion in the community :) But if we support that, why still mesos/yarn/k8s resource manager there in the tree? On Fri, Nov 8, 2019 at 10:18 PM Tom Graves wrote

Re: Build customized resource manager

2019-11-08 Thread Tom Graves
I don't know if it all works but some work was done to make cluster manager pluggable, see SPARK-13904. Tom On Wednesday, November 6, 2019, 07:22:59 PM CST, Klaus Ma wrote: Any suggestions? - Klaus On Mon, Nov 4, 2019 at 5:04 PM Klaus Ma wrote: Hi team, AFAIK, we built

Re: Build customized resource manager

2019-11-06 Thread Klaus Ma
Any suggestions? - Klaus On Mon, Nov 4, 2019 at 5:04 PM Klaus Ma wrote: > Hi team, > > AFAIK, we built k8s/yarn/mesos as resource manager; but I'd like to did > some enhancement to them, e.g. integrate with Volcano > in k8s. Is that possible to do > that

Build customized resource manager

2019-11-04 Thread Klaus Ma
Hi team, AFAIK, we built k8s/yarn/mesos as resource manager; but I'd like to did some enhancement to them, e.g. integrate with Volcano in k8s. Is that possible to do that without fork the whole spark project? For example, enable customized resource manager