Sounds like the right move. Simplifies things in important ways. Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com
On Thu, Nov 19, 2015 at 5:42 AM, Iulian Dragoș <iulian.dra...@typesafe.com> wrote: > Hi all, > > Mesos is the only cluster manager that has a fine-grained mode, but it's > more often than not problematic, and it's a maintenance burden. I'd like to > suggest removing it in the 2.0 release. > > A few reasons: > > - code/maintenance complexity. The two modes duplicate a lot of > functionality (and sometimes code) that leads to subtle differences or > bugs. See SPARK-10444 <https://issues.apache.org/jira/browse/SPARK-10444> and > also this thread > <https://mail-archives.apache.org/mod_mbox/spark-user/201510.mbox/%3ccalxmp-a+aygnwsiytm8ff20-mgwhykbhct94a2hwzth1jwh...@mail.gmail.com%3E> > and MESOS-3202 <https://issues.apache.org/jira/browse/MESOS-3202> > - it's not widely used (Reynold's previous thread > <http://apache-spark-developers-list.1001551.n3.nabble.com/Please-reply-if-you-use-Mesos-fine-grained-mode-td14930.html> > got very few responses from people relying on it) > - similar functionality can be achieved with dynamic allocation + > coarse-grained mode > > I suggest that Spark 1.6 already issues a warning if it detects > fine-grained use, with removal in the 2.0 release. > > Thoughts? > > iulian > >
