[ https://issues.apache.org/jira/browse/MAPREDUCE-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-3808: ------------------------------------------- Attachment: MR-3808.txt This patch should fix the issue. It adds in a check for null on the output in several places when building paths and short curcits those methods to return null. It also adds in the a null output check in recoverTask and causes it to be a noop when output is null. Finally it adds in tests to validate that the FileOutputCommitter handles that case correctly. > NPE in FileOutputCommitter when running a 0 reduce job > ------------------------------------------------------ > > Key: MAPREDUCE-3808 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3808 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Affects Versions: 0.23.0, 0.24.0 > Reporter: Siddharth Seth > Assignee: Robert Joseph Evans > Priority: Blocker > Attachments: MR-3808.txt > > > This was while running LoadGen. > {noformat} > Error: java.lang.NullPointerException at > org.apache.hadoop.fs.Path.<init>(Path.java:67) > at org.apache.hadoop.fs.Path.<init>(Path.java:56) > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118) > > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167) > > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149) > > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185) > > at > org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209) > > at > org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100) > > at > org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94) > > at > org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176) > > at > org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248) > > at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955) > at org.apache.hadoop.mapred.Task.done(Task.java:912) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157) > > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142) > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira