[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang updated MAPREDUCE-7402:
--
Description:
*Describe：*
In the method long computeBytesInMerges(int factor, int inMem), when factor
is set to 1, and the initial numSegments>1, inMem=0, it will cause an infinite
loop of while. Although setting factor to 1 is meaningless, and it will not be
done in actual production, we need to remind users to avoid an infinite loop
due to carelessness.
!screenshot-1.png|width=600,height=400!
*How to reproduce:*
This bug can easily occur by setting
configuration.set("mapreduce.task.io.sort.factor", "1").
was:
*Describe：*
In the method long computeBytesInMerges(int factor, int inMem), when factor
is set to 1, and the initial numSegments>1, inMem=0, it will cause an infinite
loop of while. Although setting factor to 1 is meaningless, and it will not be
done in actual production, we need to remind users to avoid an infinite loop
due to carelessness.
!screenshot-1.png!
*How to reproduce:*
This bug can easily occur by setting
configuration.set("mapreduce.task.io.sort.factor", "1").
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> !screenshot-1.png|width=600,height=400!
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang updated MAPREDUCE-7402:
--
Attachment: screenshot-1.png
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang updated MAPREDUCE-7402:
--
Description:
*Describe：*
In the method long computeBytesInMerges(int factor, int inMem), when factor
is set to 1, and the initial numSegments>1, inMem=0, it will cause an infinite
loop of while. Although setting factor to 1 is meaningless, and it will not be
done in actual production, we need to remind users to avoid an infinite loop
due to carelessness.
!screenshot-1.png!
*How to reproduce:*
This bug can easily occur by setting
configuration.set("mapreduce.task.io.sort.factor", "1").
was:
*Describe：*
In the method long computeBytesInMerges(int factor, int inMem), when factor
is set to 1, and the initial numSegments>1, inMem=0, it will cause an infinite
loop of while. Although setting factor to 1 is meaningless, and it will not be
done in actual production, we need to remind users to avoid an infinite loop
due to carelessness.
*How to reproduce:*
This bug can easily occur by setting
configuration.set("mapreduce.task.io.sort.factor", "1").
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> !screenshot-1.png!
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825497#comment-17825497
]
ASF GitHub Bot commented on MAPREDUCE-7402:
---
KeeProMise commented on PR #6622:
URL: https://github.com/apache/hadoop/pull/6622#issuecomment-1989937453
@ayushtkn @tomicooler Hi, do you have time to help review this PR? thanks~
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
> Labels: pull-request-available
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825376#comment-17825376
]
ASF GitHub Bot commented on MAPREDUCE-7402:
---
hadoop-yetus commented on PR #6622:
URL: https://github.com/apache/hadoop/pull/6622#issuecomment-1988858639
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | compile | 0m 20s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 21s | | the patch passed |
| +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javac | 0m 18s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 12s | | the patch passed with JDK
Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | spotbugs | 0m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 29s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not
generate ASF License warnings. |
| | | 86m 12s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6622/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6622 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux b7f308976a60 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9
15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 540460637ad3e1dd115eb967eef11d4d6a9b321f |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6622/1/testReport/ |
| Max. process+thread count | 1125 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6622/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was
[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang updated MAPREDUCE-7402:
--
Description:
*Describe：*
In the method long computeBytesInMerges(int factor, int inMem), when factor
is set to 1, and the initial numSegments>1, inMem=0, it will cause an infinite
loop of while. Although setting factor to 1 is meaningless, and it will not be
done in actual production, we need to remind users to avoid an infinite loop
due to carelessness.
*How to reproduce:*
This bug can easily occur by setting
configuration.set("mapreduce.task.io.sort.factor", "1").
was:function with infinite loop:Merger$MergeQueue#computeBytesInMerges(int
factor, int inMem)，
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
> Labels: pull-request-available
>
> *Describe：*
> In the method long computeBytesInMerges(int factor, int inMem), when
> factor is set to 1, and the initial numSegments>1, inMem=0, it will cause an
> infinite loop of while. Although setting factor to 1 is meaningless, and it
> will not be done in actual production, we need to remind users to avoid an
> infinite loop due to carelessness.
> *How to reproduce:*
> This bug can easily occur by setting
> configuration.set("mapreduce.task.io.sort.factor", "1").
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[ https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825346#comment-17825346 ] ASF GitHub Bot commented on MAPREDUCE-7402: --- KeeProMise opened a new pull request, #6622: URL: https://github.com/apache/hadoop/pull/6622 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. > - > > Key: MAPREDUCE-7402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Jian Zhang >Priority: Minor > > function with infinite loop:Merger$MergeQueue#computeBytesInMerges(int > factor, int inMem)， -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[ https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated MAPREDUCE-7402: -- Labels: pull-request-available (was: ) > fix mapreduce.task.io.sort.factor=1 lead to an infinite loop. > - > > Key: MAPREDUCE-7402 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Jian Zhang >Priority: Minor > Labels: pull-request-available > > function with infinite loop:Merger$MergeQueue#computeBytesInMerges(int > factor, int inMem)， -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7402) fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang updated MAPREDUCE-7402:
--
Summary: fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
(was: In the merge phase, if configuration.set("mapreduce.task.io.sort.factor",
"1"), it may lead to an infinite loop)
> fix mapreduce.task.io.sort.factor=1 lead to an infinite loop.
> -
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
>
> function with infinite loop:Merger$MergeQueue#computeBytesInMerges(int
> factor, int inMem)，
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Reopened] (MAPREDUCE-7402) In the merge phase, if configuration.set("mapreduce.task.io.sort.factor", "1"), it may lead to an infinite loop
[
https://issues.apache.org/jira/browse/MAPREDUCE-7402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian Zhang reopened MAPREDUCE-7402:
---
> In the merge phase, if configuration.set("mapreduce.task.io.sort.factor",
> "1"), it may lead to an infinite loop
> ---
>
> Key: MAPREDUCE-7402
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7402
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Jian Zhang
>Priority: Minor
>
> function with infinite loop:Merger$MergeQueue#computeBytesInMerges(int
> factor, int inMem)，
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
