Robert Grandl created YARN-1444:
-----------------------------------
Summary: RM crashes when force reducer to be node local
Key: YARN-1444
URL: https://issues.apache.org/jira/browse/YARN-1444
Project: Hadoop YARN
Issue Type: Bug
Components: client, resourcemanager
Reporter: Robert Grandl
Priority: Minor
I have tried to force reducers to execute on certain nodes. What I did is I
changed for reduce tasks, the
RMContainerRequestor#addResourceRequest(req.priority, ResourceRequest.ANY,
req.capability) to RMContainerRequestor#addResourceRequest(req.priority,
HOST_NAME, req.capability).
However, this change lead to RM crashes when reducers needs to be assigned with
the following exception:
FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:841)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:640)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:554)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:695)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:739)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:86)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:549)
at java.lang.Thread.run(Thread.java:722)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
