Hi Malcolm, Yes, the AM is just reporting what the RM specified as the maximum allowed request size.
I think 'yarn.scheduler.maximum-allocation-vcores' needs to be less than 'yarn.nodemanager.resource.cpu-vcores', since a container must fit on a single NM. Maybe the RM detected this and decided to default to 1? Can you try setting maximum-allocation-vcores lower? - Prateek On Mon, Apr 1, 2019 at 11:59 AM Malcolm McFarland <mmcfarl...@cavulus.com> wrote: > One other detail: I'm running YARN on ECS in AWS. Has anybody seen > issues with core allocation in this environment? I'm seeing this in > the samza log: > > "Got AM register response. The YARN RM supports container requests > with max-mem: 14336, max-cpu: 1" > > How does samza determine this? Looking at the Samza source on Github, > it appears to be information that's passed back to the AM when it > starts up. > > Cheers, > Malcolm > > On Mon, Apr 1, 2019 at 10:44 AM Malcolm McFarland > <mmcfarl...@cavulus.com> wrote: > > > > Hi Prateek, > > > > Sorry, meant to include these versions with my email; I'm running > > Samza 0.14 and Hadoop 2.6.1. I'm running three containers across 3 > > node managers, each with 16GB and 8 vcores. The other two containers > > are requesting 1 vcore each; even with the AMs running, that should be > > 4 for them in total, leaving plenty of processing power available. > > > > The error is in the application attempt diagnostics field: "The YARN > > cluster is unable to run your job due to unsatisfiable resource > > requirements. You asked for mem: 2048, and cpu: 2." I do not see this > > error with the same memory request, but a cpu count request of 1. > > > > Here are the configuration options pertaining to resource allocation: > > > > <?xml version="1.0"?> > > <configuration> > > <property> > > <name>yarn.resourcemanager.scheduler.class</name> > > > > <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler</value> > > </property> > > <property> > > <name>yarn.nodemanager.vmem-check-enabled</name> > > <value>false</value> > > </property> > > <property> > > <name>yarn.nodemanager.vmem-pmem-ratio</name> > > <value>2.1</value> > > </property> > > <property> > > <name>yarn.nodemanager.resource.memory-mb</name> > > <value>14336</value> > > </property> > > <property> > > <name>yarn.scheduler.minimum-allocation-mb</name> > > <value>256</value> > > </property> > > <property> > > <name>yarn.scheduler.maximum-allocation-mb</name> > > <value>14336</value> > > </property> > > <property> > > <name>yarn.scheduler.minimum-allocation-vcores</name> > > <value>1</value> > > </property> > > <property> > > <name>yarn.scheduler.maximum-allocation-vcores</name> > > <value>16</value> > > </property> > > <property> > > <name>yarn.nodemanager.resource.cpu-vcores</name> > > <value>8</value> > > </property> > > <property> > > <name>yarn.resourcemanager.cluster-id</name> > > <value>processor-cluster</value> > > </property> > > </configuration> > > > > Cheers, > > Malcolm > > > > On Mon, Apr 1, 2019 at 10:25 AM Prateek Maheshwari <prateek...@gmail.com> > wrote: > > > > > > Hi Malcolm, > > > > > > Just setting that configuration should be sufficient. We haven't seen > this > > > issue before. What Samza/YARN versions are you using? Can you also > include > > > the logs from where you get the error and your yarn configuration? > > > > > > - Prateek > > > > > > On Mon, Apr 1, 2019 at 2:33 AM Malcolm McFarland < > mmcfarl...@cavulus.com> > > > wrote: > > > > > > > Hey Folks, > > > > > > > > I'm having some issues getting multiple cores for containers in yarn. > > > > I seem to have my YARN settings correct, and the RM interface says > > > > that I have 24vcores available. However, when I set the > > > > cluster-manager.container.cpu.cores Samza setting to anything other > > > > than 1, I get a message about how the container is requesting more > > > > resources than it can allocate. With 1 core, everything is fine. Is > > > > there another Samza option I need to set? > > > > > > > > Cheers, > > > > Malcolm > > > > > > > > > > > > -- > > > > Malcolm McFarland > > > > Cavulus > > > > > > > > > > > > -- > > Malcolm McFarland > > Cavulus > > 1-800-760-6915 > > mmcfarl...@cavulus.com > > > > > > This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any > > unauthorized or improper disclosure, copying, distribution, or use of > > the contents of this message is prohibited. The information contained > > in this message is intended only for the personal and confidential use > > of the recipient(s) named above. If you have received this message in > > error, please notify the sender immediately and delete the original > > message. > > > > -- > Malcolm McFarland > Cavulus > 1-800-760-6915 > mmcfarl...@cavulus.com > > > This correspondence is from HealthPlanCRM, LLC, d/b/a Cavulus. Any > unauthorized or improper disclosure, copying, distribution, or use of > the contents of this message is prohibited. The information contained > in this message is intended only for the personal and confidential use > of the recipient(s) named above. If you have received this message in > error, please notify the sender immediately and delete the original > message. >
