That thread was mostly about benchmarking YARN vs standalone, and the results are what I'd expect -- spinning up a Spark cluster on demand through YARN has higher startup latency than using a standalone cluster, where the JVMs are already initialized and ready.
Given that there's a lot more commit activity around YARN as compared to Mesos, does that mean that YARN integration is just earlier in the maturity curve, or does it mean that YARN is the future and Mesos is in maintenance-only mode? That may be more a question for the Databricks team though: will YARN and Mesos be supported equally, or will one become the preferred method of doing cluster management under Spark? Andrew On Thu, Apr 17, 2014 at 1:27 PM, Arpit Tak <[email protected]>wrote: > Hi Wel, > > Take a look at this post... > > http://apache-spark-user-list.1001560.n3.nabble.com/Job-initialization-performance-of-Spark-standalone-mode-vs-YARN-td2016.html > > Regards, > Arpit Tak > > > On Thu, Apr 17, 2014 at 3:42 PM, Wei Wang <[email protected]> wrote: > >> Hi, there >> >> I would like to know is there any differences between Spark on Yarn and >> Spark on Mesos. Is there any comparision between them? What are the >> advantages and disadvantages for each of them. Is there any criterion for >> choosing between Yarn and Mesos? >> >> BTW, we need MPI in our framework, and I saw MPICH2 is included in Mesos. >> Should it be the reason for choosing Mesos? >> >> Thanks a lot! >> >> >> Weida >> > >
