ontainers per 8-core node?
>
> John
>
> ** **
>
> ** **
>
> *From:* Sandy Ryza [mailto:sandy.r...@cloudera.com]
> *Sent:* Tuesday, July 02, 2013 1:26 PM
>
> *To:* user@hadoop.apache.org
> *Subject:* Re: Containers and CPU
>
> ** **
>
> Use
1:26 PM
To: user@hadoop.apache.org
Subject: Re: Containers and CPU
Use of cgroups for controlling CPU is off by default, but can be turned on as a
nodemanager configuration with
yarn.nodemanager.linux-container-executor.resources-handler.class. So it is
site-wide. If you want tasks to purely
have access to all CPU cores and simply fight it
> out in the OS thread scheduler.
>
> Thanks,
>
> john
>
> ** **
>
> *From:* Sandy Ryza [mailto:sandy.r...@cloudera.com]
> *Sent:* Tuesday, July 02, 2013 11:56 AM
> *To:* user@hadoop.apache.org
> *Subjec
org
Subject: RE: Containers and CPU
Sandy,
Sorry, I don't completely follow.
When you say "with cgroups on", is that an attribute of the AM, the Scheduler,
or the Site/RM? In other words is it site-wide or something that my
application can control?
With cgroups on, is there still a
or? I'd really
like all tasks to have access to all CPU cores and simply fight it out in the
OS thread scheduler.
Thanks,
john
From: Sandy Ryza [mailto:sandy.r...@cloudera.com]
Sent: Tuesday, July 02, 2013 11:56 AM
To: user@hadoop.apache.org
Subject: Re: Containers and CPU
CPU limits are only
CPU
> into account.
>
> ** **
>
> -Chuan
>
> ** **
>
> *From:* John Lilley [mailto:john.lil...@redpoint.net]
> *Sent:* Tuesday, July 02, 2013 8:57 AM
> *To:* user@hadoop.apache.org
> *Subject:* Containers and CPU
>
> ** **
>
> I have YARN tasks th
[mailto:john.lil...@redpoint.net]
Sent: Tuesday, July 02, 2013 8:57 AM
To: user@hadoop.apache.org
Subject: Containers and CPU
I have YARN tasks that benefit from multicore scaling. However, they don't
*always* use more than one core. I would like to allocate containers based
only on memory, and let each
I have YARN tasks that benefit from multicore scaling. However, they don't
*always* use more than one core. I would like to allocate containers based
only on memory, and let each task use as many cores as needed, without
allocating exclusive CPU "slots" in the scheduler. For example, on an 8-
