[
https://issues.apache.org/jira/browse/MAPREDUCE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389967#comment-15389967
]
Haibo Chen commented on MAPREDUCE-6724:
---------------------------------------
[~jira.shegalov] Can you elaborate a little big more on what you mean by
existing test covers regression? My understanding of testLargeMemoryLimits()
is that it checks memoryLimit and maxInMemReduceLimit, both of which can not be
tested with verifyReserveLargeMapOutput(). I could restore changes made to
testLargeMemoryLimits to minimize changes to existing tests. By "testing
MAX_VALUE", do you mean that we need to have another test case in which
singleSHuffleLimit is set to Integer.MAX_VAL?
> Unsafe conversion from long to int in MergeManagerImpl.unconditionalReserve()
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-6724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Attachments: mapreduce6724.001.patch, mapreduce6724.002.patch,
> mapreduce6724.003.patch, mapreduce6724.004.patch, mapreduce6724.005.patch,
> mapreduce6724.006.patch
>
>
> When shuffle is done in memory, MergeManagerImpl converts the requested size
> to an int to allocate an instance of InMemoryMapOutput. This results in an
> overflow if the requested size is bigger than Integer.MAX_VALUE and
> eventually causes the reducer to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
