[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[ https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17605627#comment-17605627 ] ASF GitHub Bot commented on HDFS-16737: --- ZanderXu commented on PR #4784: URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1248953308 The failed UT `hadoop.hdfs.server.namenode.TestFileTruncate` works well locally. @Hexiaoqiao Sir, can you help me review this patch too? > Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume > --- > > Key: HDFS-16737 > URL: https://issues.apache.org/jira/browse/HDFS-16737 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is > elastic right now, make it fixed. > Presently the corePoolSize is set to 1 and maximumPoolSize is set to > maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue > doesn't tend to get full and threads are always confined to 1 irrespective of > maxNumThreadsPerVolume. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[
https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600699#comment-17600699
]
ASF GitHub Bot commented on HDFS-16737:
---
hadoop-yetus commented on PR #4784:
URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1237915530
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 8s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 17s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 22s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 35s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 43s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 347m 45s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not
generate ASF License warnings. |
| | | 466m 2s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestFileTruncate |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4784 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 0e0ad1a30cba 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b13903175a636d20c0a993cdbb61608c9d706bb2 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/3/testReport/ |
| Max. process+thread count | 2245 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-p
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[ https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17600562#comment-17600562 ] ASF GitHub Bot commented on HDFS-16737: --- ZanderXu commented on PR #4784: URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1237614839 @ayushtkn @jianghuazhu Sir, I added one UT to test the active count and core pool size of the executer. Can you help me review it again. I found this issue because we encountered many stacked pending deletion in our prod environment and caused ReplicationNotFoundException. So this PR is used to increase the active thread count of the executer. And we will raise another PR to resolve ReplicationNotFoundException. @ferhui @haiyang1987 Masters, can you help me review this PR? > Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume > --- > > Key: HDFS-16737 > URL: https://issues.apache.org/jira/browse/HDFS-16737 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is > elastic right now, make it fixed. > Presently the corePoolSize is set to 1 and maximumPoolSize is set to > maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue > doesn't tend to get full and threads are always confined to 1 irrespective of > maxNumThreadsPerVolume. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[
https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597413#comment-17597413
]
ASF GitHub Bot commented on HDFS-16737:
---
hadoop-yetus commented on PR #4784:
URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1230901326
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 17s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 327m 52s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 444m 28s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4784 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 5122abd2c850 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 845f3331ff2dbe3f367a7a0681c3a37f4ee67779 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/2/testReport/ |
| Max. process+thread count | 2283 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[ https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597223#comment-17597223 ] ASF GitHub Bot commented on HDFS-16737: --- ZanderXu commented on PR #4784: URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1230394854 @ayushtkn Thanks for your review. About [HDFS-16386](https://issues.apache.org/jira/browse/HDFS-16386), the increased memory maybe caused by the stacked async deletion takes, increasing or decreasing the maximum number of threads may cannot fix it. Can increase the corePoolSize of the threadPoolExecutor maybe can reduce the memory. cc @jianghuazhu I have updated the patch, please help me review it again. Thanks > Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume > --- > > Key: HDFS-16737 > URL: https://issues.apache.org/jira/browse/HDFS-16737 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is > elastic right now, make it fixed. > Presently the corePoolSize is set to 1 and maximumPoolSize is set to > maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue > doesn't tend to get full and threads are always confined to 1 irrespective of > maxNumThreadsPerVolume. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[
https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17584291#comment-17584291
]
ASF GitHub Bot commented on HDFS-16737:
---
ayushtkn commented on code in PR #4784:
URL: https://github.com/apache/hadoop/pull/4784#discussion_r953896405
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:
##
@@ -151,9 +151,9 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
public static final longDFS_DATANODE_MAX_LOCKED_MEMORY_DEFAULT = 0;
public static final String
DFS_DATANODE_FSDATASETCACHE_MAX_THREADS_PER_VOLUME_KEY =
"dfs.datanode.fsdatasetcache.max.threads.per.volume";
public static final int
DFS_DATANODE_FSDATASETCACHE_MAX_THREADS_PER_VOLUME_DEFAULT = 4;
- public static final String
DFS_DATANODE_FSDATASETASYNCDISK_MAX_THREADS_PER_VOLUME_KEY =
- "dfs.datanode.fsdatasetasyncdisk.max.threads.per.volume";
- public static final int
DFS_DATANODE_FSDATASETASYNCDISK_MAX_THREADS_PER_VOLUME_DEFAULT = 4;
+ public static final String
DFS_DATANODE_FSDATASETASYNCDISK_THREADS_PER_VOLUME_KEY =
+ "dfs.datanode.fsdatasetasyncdisk.threads.per.volume";
+ public static final int
DFS_DATANODE_FSDATASETASYNCDISK_THREADS_PER_VOLUME_DEFAULT = 1;
Review Comment:
you can't change the config name if that is there in a released version.
> Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
> ---
>
> Key: HDFS-16737
> URL: https://issues.apache.org/jira/browse/HDFS-16737
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
>
> The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is
> elastic right now, make it fixed.
> Presently the corePoolSize is set to 1 and maximumPoolSize is set to
> maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue
> doesn't tend to get full and threads are always confined to 1 irrespective of
> maxNumThreadsPerVolume.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[ https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583911#comment-17583911 ] ASF GitHub Bot commented on HDFS-16737: --- ZanderXu commented on PR #4784: URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1225011225 @ayushtkn Sir, can you help me review this patch? > Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume > --- > > Key: HDFS-16737 > URL: https://issues.apache.org/jira/browse/HDFS-16737 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > > The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is > elastic right now, make it fixed. > Presently the corePoolSize is set to 1 and maximumPoolSize is set to > maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue > doesn't tend to get full and threads are always confined to 1 irrespective of > maxNumThreadsPerVolume. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[
https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583222#comment-17583222
]
ASF GitHub Bot commented on HDFS-16737:
---
hadoop-yetus commented on PR #4784:
URL: https://github.com/apache/hadoop/pull/4784#issuecomment-1223150269
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 2s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 21s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | compile | 1m 17s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 333m 53s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 452m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4784 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 5c60f210dffb 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4987d2a33c28fffa706d48724d1f264e48aec89a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4784/1/testReport/ |
| Max. process+thread count | 2326 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job
[jira] [Commented] (HDFS-16737) Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume
[ https://issues.apache.org/jira/browse/HDFS-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582999#comment-17582999 ] ASF GitHub Bot commented on HDFS-16737: --- ZanderXu opened a new pull request, #4784: URL: https://github.com/apache/hadoop/pull/4784 ### Description of PR The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is elastic right now, make it fixed. Presently the corePoolSize is set to 1 and maximumPoolSize is set to maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue doesn't tend to get full and threads are always confined to 1 irrespective of maxNumThreadsPerVolume. > Fix number of threads in FsDatasetAsyncDiskService#addExecutorForVolume > --- > > Key: HDFS-16737 > URL: https://issues.apache.org/jira/browse/HDFS-16737 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > > The number of threads in FsDatasetAsyncDiskService#addExecutorForVolume is > elastic right now, make it fixed. > Presently the corePoolSize is set to 1 and maximumPoolSize is set to > maxNumThreadsPerVolume, but since the size of Queue is Integer.MAX, the queue > doesn't tend to get full and threads are always confined to 1 irrespective of > maxNumThreadsPerVolume. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
