Vinod: thanks.
the queue is the correct way to go. but a small technical issue is that in our (as in most) environment, the most users share one "default" queue, and more importantly, even if they have a special queue, the queue limit is defined by ops team, not the user himself. i.e. he can not "self-police" his queue usage: if he has a queue with max 100 tasks running, he can not limit that down to 50 Yang On Mon, Apr 27, 2015 at 5:07 PM, Vinod Kumar Vavilapalli < vino...@hortonworks.com> wrote: > The MapReduce ApplicationMaster supports only one job. You can say that > (YARN ResourceManager + a bunch of MR ApplicationMasters (one per job) = > JobTracker). > > Tez does have a notion of multiple DAGs per YARN app. > > For your specific use-case, you can force that user to a queue and limit > how much he/she can access. > > Thanks > +Vinod > > On Apr 27, 2015, at 3:30 PM, Yang <teddyyyy...@gmail.com> wrote: > > > conceptually, the MR application master is similar to the old job > tracker. > > > > if so, can I submit multiple jobs to the same MR application master? it > looks like an odd use case, the context is that we have users generating > lots of MR jobs, and he currently has a little crude scheduler that > periodically launches jobs to the RM by just "hadoop jar ..." > > > > instead I was thinking to "carve out" a MR2 allocation in RM first, then > periodically submit to the "job tracker"/application master, so that all > the jobs are localized to this allocation. > > > > > > I was also thinking about using Tez instead of MR application master. > Tez replaces MR2 application master, not on top of it, right? > > > > Thanks > > Yang > >
