[ https://issues.apache.org/jira/browse/YARN-9226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749921#comment-16749921 ]
lujie commented on YARN-9226: ----------------------------- Attatch the corresponding log. And this bugs will trigged while we shutdown a NM > NPE while YarnChild shudown > --------------------------- > > Key: YARN-9226 > URL: https://issues.apache.org/jira/browse/YARN-9226 > Project: Hadoop YARN > Issue Type: Bug > Reporter: lujie > Assignee: lujie > Priority: Major > Attachments: syslog > > > In YarnChild.main > {code:java} > try{ > logSyncer = TaskLog.createLogSyncer();//line 168 > .... > taskFinal.run(job, umbilical); //line 178 > }catch (Exception exception) {//line 187 > LOG.warn("Exception running child : " > + StringUtils.stringifyException(exception)); > ................. > task.taskCleanup(umbilical);// line 200 > }{code} > At line 178. it will initialize the task.committer, but the line168 may throw > exception, it will skip initialize the task.committer, hence task.committer > == null. Line 187 will catch this exception and do clean up(line 200), code > line 200 will use task.committer without null check, hence NPE happens > {code:java} > 2019-01-23 16:59:42,864 INFO [main] org.apache.hadoop.mapred.YarnChild: > Exception cleaning up: java.lang.NullPointerException > at org.apache.hadoop.mapred.Task.taskCleanup(Task.java:1458) > at org.apache.hadoop.mapred.YarnChild$3.run(YarnChild.java:200) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:197) > {code} > So why line168 may throw exception, below log give a example: > {code:java} > 2019-01-23 16:59:42,857 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : java.lang.IllegalStateException: Shutdown in > progress, cannot add a shutdownHook > at > org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:299) > at org.apache.hadoop.mapred.TaskLog.createLogSyncer(TaskLog.java:340) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org