I don't know if it all works but some work was done to make cluster manager pluggable, see SPARK-13904. Tom On Wednesday, November 6, 2019, 07:22:59 PM CST, Klaus Ma <klaus1982...@gmail.com> wrote: Any suggestions?
- Klaus On Mon, Nov 4, 2019 at 5:04 PM Klaus Ma <klaus1982...@gmail.com> wrote: Hi team, AFAIK, we built k8s/yarn/mesos as resource manager; but I'd like to did some enhancement to them, e.g. integrate with Volcano in k8s. Is that possible to do that without fork the whole spark project? For example, enable customized resource manager with configuration, e.g. replace `org.apache.spark.deploy.k8s.submit.KubernetesClientApplication` with `MyK8SClient`, so I can only maintain the resource manager instead of the whole project. -- Klaus