max number of application master in YARN
Is there any configuration in MR2 and YARN to limit concurrent max applications by setting max limit on ApplicationMasters in the cluster?
Re: max number of application master in YARN
Take a look at yarn.scheduler.capacity.maximum-am-resource-percent On Thu, Apr 30, 2015 at 11:38 AM, Shushant Arora shushantaror...@gmail.com wrote: Is there any configuration in MR2 and YARN to limit concurrent max applications by setting max limit on ApplicationMasters in the cluster?
Re: max number of application master in YARN
With Capacity Scheduler, the other useful param would be: yarn.scheduler.capacity.maximum-applications http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html On Thu, Apr 30, 2015 at 11:52 AM, Prashant Kommireddi prash1...@gmail.com wrote: Take a look at yarn.scheduler.capacity.maximum-am-resource-percent On Thu, Apr 30, 2015 at 11:38 AM, Shushant Arora shushantaror...@gmail.com wrote: Is there any configuration in MR2 and YARN to limit concurrent max applications by setting max limit on ApplicationMasters in the cluster?
Re: max number of application master in YARN
Thanks!! Is it for capacity scheduler only. Or applicable to fair scheduler also? On Fri, May 1, 2015 at 12:22 AM, Prashant Kommireddi prash1...@gmail.com wrote: Take a look at yarn.scheduler.capacity.maximum-am-resource-percent On Thu, Apr 30, 2015 at 11:38 AM, Shushant Arora shushantaror...@gmail.com wrote: Is there any configuration in MR2 and YARN to limit concurrent max applications by setting max limit on ApplicationMasters in the cluster?
Re: max number of application master in YARN
Hi Shushant, If you use fair scheduler, you can restrict number of AM by configuring queue: * maxRunningApps: limit the number of apps from the queue to run at once * maxAMShare: limit the fraction of the queue's fair share that can be used to run application masters. This property can only be used for leaf queues. For example, if set to 1.0f, then AMs in the leaf queue can take up to 100% of both the memory and CPU fair share. The value of -1.0f will disable this feature and the amShare will not be checked. The default value is 0.5f. http://hadoop.apache.org/docs/r2.6.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html Thanks, - Tsuyoshi On Fri, May 1, 2015 at 4:04 AM, Shushant Arora shushantaror...@gmail.com wrote: Thanks!! Is it for capacity scheduler only. Or applicable to fair scheduler also? On Fri, May 1, 2015 at 12:22 AM, Prashant Kommireddi prash1...@gmail.com wrote: Take a look at yarn.scheduler.capacity.maximum-am-resource-percent On Thu, Apr 30, 2015 at 11:38 AM, Shushant Arora shushantaror...@gmail.com wrote: Is there any configuration in MR2 and YARN to limit concurrent max applications by setting max limit on ApplicationMasters in the cluster?
