Glad you were able to figure it out. FWIW, I had the same interpretation as
you. Let us know if you need anything else.
- Prateek
On Tue, Apr 2, 2019 at 4:55 PM Malcolm McFarland
wrote:
> Found the issue, and thank goodness it was a configuration issue on my end:
> I was setting the yarn.schedu
Found the issue, and thank goodness it was a configuration issue on my end:
I was setting the yarn.scheduler.maximum-allocation-vcores too low and
artificially constraining the cluster. This is (as the name implies) the
maximum allocation for the entire cluster; I had interpreted the
description fr
Interestingly, I just tried setting
yarn.scheduler.minimum-allocation-vcores=2 and restarting everything. On
startup, the RM now displays a Minimum Allocation of , but my application container still shows "Resource:4096 Memory,
1 VCores". The statistics page for the "default" queue shows "Used
Reso
Hi Prateek,
I'm not getting an error now, just an unyielding vcore allotment of 1.
I just verified that we're setting
yarn.resourcemanager.scheduler.class=org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
and
yarn.scheduler.capacity.resource-calculator=org.apache
And just to double check, you also changed the
yarn.resourcemanager.scheduler.class to CapacityScheduler?
On Tue, Apr 2, 2019 at 9:49 AM Prateek Maheshwari
wrote:
> Is it still the same message from the AM? The one that says: "Got AM
> register response. The YARN RM supports container requests w
Is it still the same message from the AM? The one that says: "Got AM
register response. The YARN RM supports container requests with max-mem:
14336, max-cpu: 1"
On Tue, Apr 2, 2019 at 12:09 AM Malcolm McFarland
wrote:
> Hey Prateek,
>
> The upgrade to Hadoop 2.7.6 went fine; everything seems to
Hey Prateek,
The upgrade to Hadoop 2.7.6 went fine; everything seems to be working, and
access to S3 via an access key/secret pair is working as well. However, my
requested tasks are still only getting allocated 1 core, despite requesting
more than that. Once again, I have a 3-node cluster that sh
One more thing -- fwiw, I actually also came across the possibility that I
would need to use the DominantResourceCalculator, but as you point out,
this doesn't seem to be available in Hadoop 2.6.
On Mon, Apr 1, 2019 at 5:27 PM Malcolm McFarland
wrote:
> That's quite helpful! I actually initiall
That's quite helpful! I actually initially tried using a version of
Hadoop > 2.6.x; when I did, it seemed like the AWS credentials in YARN
(fs.s3a.access.key, fs.s3a.secret.key) weren't being accessed, as I
received lots of "No AWS Credentials
provided by DefaultAWSCredentialsProviderChain" message
Hi Malcolm,
I think this is because in YARN 2.6 the FifoScheduler only accounts for
memory for 'maximumAllocation':
https://github.com/apache/hadoop/blob/branch-2.6.2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/r
Hi Prateek,
This still seems to be manifesting with the same problem. Since this seems
to be something in the hadoop codebase, and I've emailed the hadoop-dev
mailing list about it.
Cheers,
Malcolm
On Mon, Apr 1, 2019 at 1:51 PM Prateek Maheshwari
wrote:
> Hi Malcolm,
>
> Yes, the AM is just r
Hi Malcolm,
Yes, the AM is just reporting what the RM specified as the maximum allowed
request size.
I think 'yarn.scheduler.maximum-allocation-vcores' needs to be less than
'yarn.nodemanager.resource.cpu-vcores', since a container must fit on a
single NM. Maybe the RM detected this and decided t
One other detail: I'm running YARN on ECS in AWS. Has anybody seen
issues with core allocation in this environment? I'm seeing this in
the samza log:
"Got AM register response. The YARN RM supports container requests
with max-mem: 14336, max-cpu: 1"
How does samza determine this? Looking at the S
Hi Prateek,
Sorry, meant to include these versions with my email; I'm running
Samza 0.14 and Hadoop 2.6.1. I'm running three containers across 3
node managers, each with 16GB and 8 vcores. The other two containers
are requesting 1 vcore each; even with the AMs running, that should be
4 for them in
Hi Malcolm,
Just setting that configuration should be sufficient. We haven't seen this
issue before. What Samza/YARN versions are you using? Can you also include
the logs from where you get the error and your yarn configuration?
- Prateek
On Mon, Apr 1, 2019 at 2:33 AM Malcolm McFarland
wrote:
Hey Folks,
I'm having some issues getting multiple cores for containers in yarn.
I seem to have my YARN settings correct, and the RM interface says
that I have 24vcores available. However, when I set the
cluster-manager.container.cpu.cores Samza setting to anything other
than 1, I get a message ab
16 matches
Mail list logo
