Good summary! One more advantage of running spark on mesos: community support. There are quite a big user base that runs spark on mesos, so if you encounter a problem with your deployment, it's very likely you can get the answer by a simple google search, or asking in the spark/mesos user list. By contrast it's not likely to achieve the same with spark on k8s.
On Sun, Aug 14, 2016 at 2:40 AM, Michael Gummelt <mgumm...@mesosphere.io> wrote: > Spark has a first-class scheduler for Mesos, whereas it doesn't for > Kubernetes. Running Spark on Kubernetes means running Spark in standalone > mode, wrapped in a Kubernetes service: https://github.com/kubernetes/ > kubernetes/tree/master/examples/spark > > So you're effectively comparing standalone vs. Mesos. For basic purposes, > standalone works fine. Mesos adds support for things like docker images, > security, resource reservations via roles, targeting specific nodes via > attributes, etc. > > The main benefit of Mesos, however, is that you can share the same > infrastructure with other, non-Spark services. We have users, for example, > running Spark on the same cluster as HDFS, Cassandra, Kafka, web apps, > Jenkins, etc. You can do this with Kubernetes to some extent, but running > in standalone means that the Spark "partition" isn't elastic. You must > statically partition to exclusively run Spark. > > On Sat, Aug 13, 2016 at 11:24 AM, guyoh <g12...@gmail.com> wrote: > >> My company is trying to decide whether to use kubernetes or mesos. Since >> we >> are planning to use Spark in the near future, I was wandering what is the >> best choice for us. >> Thanks, >> Guy >> >> >> >> -- >> View this message in context: http://apache-spark-user-list. >> 1001560.n3.nabble.com/mesos-or-kubernetes-tp27530.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> > > > -- > Michael Gummelt > Software Engineer > Mesosphere >
