[
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871732#action_12871732
]
Ravi Gummadi commented on MAPREDUCE-1813:
-----------------------------------------
This is not a duplicate of MAPREDUCE-576 and still exists after HADOOP-5623.
The issue here is NPE is seen because error thread's reporter is null. In
source code, this reporter is set only through startOutputThreads() which is
called from reduce(). When the input to reducer is empty, reduce() is not
called and thus reporter of error thread is not set --- causing NPE.
If the following perl script is reducer,
{code}
#!/usr/bin/perl
print STDERR "reporter:status: before\n";
while(<STDIN>) {
chomp;
}
print STDERR "reporter:status: after\n";
{code}
the first print statement causes this NPE even with non-empty input to reducer.
The second print statement causes this NPE if reducer's input is empty. Need to
fix both the cases.
The problem exists in current trunk also.
> NPE in PipeMapred.MRErrorThread
> -------------------------------
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/streaming
> Affects Versions: 0.20.1
> Reporter: Amareshwari Sriramadasu
> Fix For: 0.20.3
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
