[
https://issues.apache.org/jira/browse/MAPREDUCE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403505#comment-15403505
]
Hudson commented on MAPREDUCE-6724:
-----------------------------------
SUCCESS: Integrated in Hadoop-trunk-Commit #10190 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10190/])
MAPREDUCE-6724. Single shuffle to memory must not exceed (gera: rev
6890d5b472320fa7592ed1b08b623c55a27089c6)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/MergeManagerImpl.java
> Single shuffle to memory must not exceed Integer#MAX_VALUE
> ----------------------------------------------------------
>
> Key: MAPREDUCE-6724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Fix For: 2.9.0
>
> Attachments: MAPREDUCE-6724.009.patch, mapreduce6724.001.patch,
> mapreduce6724.002.patch, mapreduce6724.003.patch, mapreduce6724.004.patch,
> mapreduce6724.005.patch, mapreduce6724.006.patch, mapreduce6724.007.patch,
> mapreduce6724.008.patch
>
>
> When shuffle is done in memory, MergeManagerImpl converts the requested size
> to an int to allocate an instance of InMemoryMapOutput. This results in an
> overflow if the requested size is bigger than Integer.MAX_VALUE and
> eventually causes the reducer to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
