[
https://issues.apache.org/jira/browse/YARN-9164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733286#comment-16733286
]
Hudson commented on YARN-9164:
------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15697 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/15697/])
YARN-9164. Shutdown NM may cause NPE when opportunistic container (wwei: rev
cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestOpportunisticContainerAllocatorAMService.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/OpportunisticContainerAllocatorAMService.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerUtils.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
> Shutdown NM may cause NPE when opportunistic container scheduling is enabled
> ----------------------------------------------------------------------------
>
> Key: YARN-9164
> URL: https://issues.apache.org/jira/browse/YARN-9164
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: lujie
> Assignee: lujie
> Priority: Critical
> Fix For: 3.3.0, 3.2.1
>
> Attachments: YARN-9164-0.patch, YARN-9164-1.patch, YARN-9164-2.patch,
> hadoop-hires-resourcemanager-hadoop11.log
>
>
> We have meeted an NPE which can crash the whole cluster
> {code:java}
> 2018-12-31 22:18:11,924 FATAL org.apache.hadoop.yarn.event.EventDispatcher:
> Error in handling event type APP_ATTEMPT_REMOVED to the Event Dispatcher
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler.completedContainer(AbstractYarnScheduler.java:696)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.doneApplicationAttempt(CapacityScheduler.java:1123)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1827)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:171)
> at
> org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> this bug also happens in the latest trunk!
>
> workload is
> {code:java}
> $HADOOP_HOME/bin/hadoop jar
> $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-$VERSION.jar pi
> -Dmapreduce.job.num-opportunistic-maps-percent="100" 50 100
> {code}
> while job is running , shutdown one NM
> also need inject sleep before AbstractYarnScheduler.getNode()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
