Re: Abstraction layer to support both YARN and Mesos
I thought some high availability and resource isolation features in Mesos are more matured. If no one is interested in this topic, MR should go with YARN. On Fri, Jul 26, 2013 at 7:14 PM, Harsh J ha...@cloudera.com wrote: Do we have a good reason to prefer Mesos over YARN for scheduling MR specifically? At what times would one prefer the other? On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work is useful for MapReduce users. On the other hand, this work can also be large, because MRv2's code base is tightly coupled with YARN currently. Thoughts? - Tsuyoshi -- Harsh J -- - Tsuyoshi
Re: Abstraction layer to support both YARN and Mesos
Actually, I am interested. Lots of different Apache top level projects seem to overlap and it can be confusing. Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework. On Jul 29, 2013, at 9:58 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: I thought some high availability and resource isolation features in Mesos are more matured. If no one is interested in this topic, MR should go with YARN. On Fri, Jul 26, 2013 at 7:14 PM, Harsh J ha...@cloudera.com wrote: Do we have a good reason to prefer Mesos over YARN for scheduling MR specifically? At what times would one prefer the other? On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work is useful for MapReduce users. On the other hand, this work can also be large, because MRv2's code base is tightly coupled with YARN currently. Thoughts? - Tsuyoshi -- Harsh J -- - Tsuyoshi
Re: Abstraction layer to support both YARN and Mesos
Harsh, yes, I know what you mean :-) Never mind. We should discuss this topic with MR users. On Tue, Jul 30, 2013 at 12:08 AM, Michael Segel msegel_had...@hotmail.com wrote: Actually, I am interested. Lots of different Apache top level projects seem to overlap and it can be confusing. Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework. On Jul 29, 2013, at 10:06 AM, Michael Segel mse...@segel.com wrote: Actually, I am interested. Lots of different Apache top level projects seem to overlap and it can be confusing. Its very easy for a good technology to get starved because no one asks how to combine these features in to the framework. On Jul 29, 2013, at 9:58 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: I thought some high availability and resource isolation features in Mesos are more matured. If no one is interested in this topic, MR should go with YARN. On Fri, Jul 26, 2013 at 7:14 PM, Harsh J ha...@cloudera.com wrote: Do we have a good reason to prefer Mesos over YARN for scheduling MR specifically? At what times would one prefer the other? On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work is useful for MapReduce users. On the other hand, this work can also be large, because MRv2's code base is tightly coupled with YARN currently. Thoughts? - Tsuyoshi -- Harsh J -- - Tsuyoshi -- - Tsuyoshi
Abstraction layer to support both YARN and Mesos
Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work is useful for MapReduce users. On the other hand, this work can also be large, because MRv2's code base is tightly coupled with YARN currently. Thoughts? - Tsuyoshi
Re: Abstraction layer to support both YARN and Mesos
Do we have a good reason to prefer Mesos over YARN for scheduling MR specifically? At what times would one prefer the other? On Fri, Jul 26, 2013 at 11:43 AM, Tsuyoshi OZAWA ozawa.tsuyo...@gmail.com wrote: Hi, Now, Apache Mesos, an distributed resource manager, is top-level apache project. Meanwhile, As you know, Hadoop has own resource manager - YARN. IMHO, we should make resource manager pluggable in MRv2, because there are their own field users of MapReduce would like to use. I think this work is useful for MapReduce users. On the other hand, this work can also be large, because MRv2's code base is tightly coupled with YARN currently. Thoughts? - Tsuyoshi -- Harsh J
