[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825559#comment-17825559
]
ASF GitHub Bot commented on MAPREDUCE-7402:
-------------------------------------------
ayushtkn commented on code in PR #6622:
URL: https://github.com/apache/hadoop/pull/6622#discussion_r1521008831
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMerger.java:
##########
@@ -229,6 +232,14 @@ public void testInMemoryAndOnDiskMerger() throws Throwable
{
Assert.assertEquals(0, mergeManager.inMemoryMapOutputs.size());
Assert.assertEquals(0, mergeManager.inMemoryMergedMapOutputs.size());
Assert.assertEquals(0, mergeManager.onDiskMapOutputs.size());
+
+ jobConf.set("mapreduce.task.io.sort.factor", "1");
+ thrown.expectMessage("Invalid value for mapreduce.task.io.sort.factor: 1,"
+
+ " please set it to a number greater than 1");
+ thrown.expect(IllegalArgumentException.class);
+ new MergeManagerImpl<Text, Text>(reduceId2, jobConf, fs, lda,
Reporter.NULL, null,
+ null, null, null, null,
+ null, null, new Progress(), new MROutputFiles());
Review Comment:
We should use ``LambdaTestUtils.intercept`` rather than this ``thrown.``
thing
##########
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMerger.java:
##########
@@ -229,6 +232,14 @@ public void testInMemoryAndOnDiskMerger() throws Throwable
{
Assert.assertEquals(0, mergeManager.inMemoryMapOutputs.size());
Assert.assertEquals(0, mergeManager.inMemoryMergedMapOutputs.size());
Assert.assertEquals(0, mergeManager.onDiskMapOutputs.size());
+
+ jobConf.set("mapreduce.task.io.sort.factor", "1");
Review Comment:
can you use ``IO_SORT_FACTOR`` rather than hardcoding the config value
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -------------------------------------------------------------
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Jian Zhang
> Priority: Minor
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> !screenshot-1.png|width=600,height=400!
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
