[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-11 Thread Vinod K V (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod K V updated MAPREDUCE-1813:
-

  Status: Resolved  (was: Patch Available)
Release Note: Fixed an NPE in streaming that occurs when there is no input 
to reduce and the streaming reducer sends status updates by writing 
"reporter:status: xxx" statements to stderr.
  Resolution: Fixed

I just committed this to trunk. Thanks Ravi!

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.3.patch, 
> 1813.v1.4.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-10 Thread Vinod K V (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod K V updated MAPREDUCE-1813:
-

  Status: Patch Available  (was: Open)
Hadoop Flags: [Reviewed]

+1 for the patch. Submitting to Hudson.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.3.patch, 
> 1813.v1.4.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-10 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Attachment: 1813.v1.4.patch

Attaching new patch as I missed replacing one occurence of readOutput() call in 
earlier patch.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.3.patch, 
> 1813.v1.4.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-10 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Attachment: 1813.v1.3.patch

Attaching new patch with the testcases refactoring. Moved the validation of 
task status from testStreamingStatus() to testReporting() and removed 
testStreamingStatus() method. Modified the perl script of 
TestStreamingStatus.java to write to stdout(similar to that is there in 
TestStreamingEmptyInpnonemptyOut.java) and removed the file 
TestStreamingEmptyInpnonemptyOut.java.

As there are 3 copies of the method readOutput() in different files, removed 2 
of the 3 readOutput() methods and kept the one in MapReduceTestUtil.java.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.3.patch, 
> 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-10 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Status: Open  (was: Patch Available)

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-08 Thread Amareshwari Sriramadasu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1813:
---

Fix Version/s: 0.22.0
   (was: 0.20.3)

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.22.0
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-08 Thread Amareshwari Sriramadasu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1813:
---

Status: Patch Available  (was: Open)

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-08 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Attachment: 1813.v1.2.patch

Attaching patch by changing the testcases to have {perl script with empty input 
having "reporter:status:" lines and "reporter:counter:" lines written to 
stderr} instead of {StderrApp.class being used as streaming task}. Because, 
with nonempty input, earlier patch's testcase was not causing NPE 
consistently(fails based on timing) without the fix of the patch.

Please review and provide your comments.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch, 1813.v1.2.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-07 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Status: Open  (was: Patch Available)

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-06 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Attachment: 1813.v1.patch

Attaching patch that delays the starting of error thread till output thread is 
started so that we don't lose any stderr "report:status:" lines because of 
dummy reporter in case of having "report:status:" lines before consuming input 
in the reducer.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch, 1813.v1.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-06-01 Thread Amareshwari Sriramadasu (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated MAPREDUCE-1813:
---

Status: Patch Available  (was: Open)

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.



[jira] Updated: (MAPREDUCE-1813) NPE in PipeMapred.MRErrorThread

2010-05-27 Thread Ravi Gummadi (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravi Gummadi updated MAPREDUCE-1813:


Attachment: 1813.patch

Attaching patch for trunk fixing the issue.

This patch sets the reporter of MRErrorThread(in PipeMapRed.java) to a dummy 
reporter at the beginning of MRErrorThread.run() method just to avoid NPE. The 
actual reporter will be later set properly as usual and reports as needed. In 
the interim(before the actual reporter is set), the dummy reporter will just 
ignore the reporting lines.

This is the way MRErrorThread's reporter is set to dummy reporter in 
waitOutputThreads() for the case of empty input based reducer. See HADOOP-4620 
for details of the same.
But before control goes to waitOutputThreads(), if the MRErrorThread's reporter 
is accessed, we get NPE. So to avoid this NPE, am setting this dummy reporter.

This patch adds testcases which tests the perl script given in earlier comment 
as mapper and as reducer. Both test cases (1) perl script as streaming job's 
mapper and (2) perl script as streaming based reducer
  fail without the fix and pass with the fix.

Please review and provide your comments.

> NPE in PipeMapred.MRErrorThread
> ---
>
> Key: MAPREDUCE-1813
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1813
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: contrib/streaming
>Affects Versions: 0.20.1
>Reporter: Amareshwari Sriramadasu
>Assignee: Ravi Gummadi
> Fix For: 0.20.3
>
> Attachments: 1813.patch
>
>
> Some reduce tasks fail with following NPE
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:325)
> at 
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:540)
> at org.apache.hadoop.streaming.PipeReducer.close(PipeReducer.java:137)
> at 
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> Caused by: java.lang.NullPointerException
>at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.setStatus(PipeMapRed.java:517)
> at 
> org.apache.hadoop.streaming.PipeMapRed$MRErrorThread.run(PipeMapRed.java:449)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.