[ https://issues.apache.org/jira/browse/MAPREDUCE-3992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233139#comment-13233139 ]
Hudson commented on MAPREDUCE-3992: ----------------------------------- Integrated in Hadoop-Common-0.23-Commit #707 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/707/]) MAPREDUCE-3992. Reduce fetcher doesn't verify HTTP status code of response. Contributed by Todd Lipcon. (Revision 1302755) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1302755 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java > Reduce fetcher doesn't verify HTTP status code of response > ---------------------------------------------------------- > > Key: MAPREDUCE-3992 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3992 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1 > Affects Versions: 0.23.1, 0.24.0, 1.0.1 > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Fix For: 0.24.0, 0.23.3 > > Attachments: mr-3992.txt > > > Currently, the reduce fetch code doesn't check the HTTP status code of the > response. This can lead to the following situation: > - the map output servlet gets an IOException after setting the headers but > before the first call to flush() > - this causes it to send a response with a non-OK result code, including the > exception text as the response body (response.sendError() does this if the > response isn't committed) > - it will still include the response headers indicating it's a valid response > In the case of a merge-to-memory, the compression codec might then try to > interpret the HTML response as compressed data, resulting in either a huge > allocation (OOME) or some other nasty error. This bug seems to be present in > MR1, but haven't checked trunk/MR2 yet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira