[jira] [Comment Edited] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1778#comment-1778 ] Yanlei Yu edited comment on HDFS-17093 at 7/19/23 6:08 AM: --- Jus

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1778#comment-1778 ] Yanlei Yu commented on HDFS-17093: -- Just to add to that,Our cluster configuration is df

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1775#comment-1775 ] Yanlei Yu commented on HDFS-17093: -- [~xinglin] ，I think you modify some more reasonable,

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1774#comment-1774 ] Yanlei Yu commented on HDFS-17093: -- {quote} Would you try to dig the log information at

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744437#comment-17744437 ] Xing Lin commented on HDFS-17093: - Hi [~yuyanlei], Thanks for sharing! I don't fully und

[jira] [Comment Edited] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744429#comment-17744429 ] Xiaoqiao He edited comment on HDFS-17097 at 7/19/23 3:54 AM: -

[jira] [Commented] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744429#comment-17744429 ] Xiaoqiao He commented on HDFS-17097: [~FuzzingTeam] Thanks for your report. We receiv

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744424#comment-17744424 ] Xiaoqiao He commented on HDFS-17093: [~yuyanlei] Thanks for your report, Would you tr

[jira] [Commented] (HDFS-17091) Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744418#comment-17744418 ] ASF GitHub Bot commented on HDFS-17091: --- YuanbenWang commented on PR #5849: URL: ht

[jira] [Updated] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanlei Yu updated HDFS-17093: - Issue Type: Bug (was: Improvement) > In the case of all datanodes sending FBR when the namenode restarts

[jira] [Resolved] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan resolved HDFS-17088. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve the debug verifyE

[jira] [Commented] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744408#comment-17744408 ] ASF GitHub Bot commented on HDFS-17088: --- slfan1989 commented on PR #5846: URL: http

[jira] [Commented] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744407#comment-17744407 ] ASF GitHub Bot commented on HDFS-17088: --- slfan1989 merged PR #5846: URL: https://gi

[jira] [Resolved] (HDFS-17091) Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan resolved HDFS-17091. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Blocks on DECOMMISSIONING

[jira] [Commented] (HDFS-17091) Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744406#comment-17744406 ] ASF GitHub Bot commented on HDFS-17091: --- slfan1989 commented on PR #5849: URL: http

[jira] [Commented] (HDFS-17091) Blocks on DECOMMISSIONING DNs should be sorted properly in LocatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744405#comment-17744405 ] ASF GitHub Bot commented on HDFS-17091: --- slfan1989 merged PR #5849: URL: https://gi

[jira] [Commented] (HDFS-15901) Solve the problem of DN repeated block reports occupying too many RPCs during Safemode

[ https://issues.apache.org/jira/browse/HDFS-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744402#comment-17744402 ] Yanlei Yu commented on HDFS-15901: -- I submitted a patch to [HDFS-18093|https://issues.a

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744401#comment-17744401 ] Yanlei Yu commented on HDFS-17093: -- The problem is similar to HDFS-15901 > In the case

[jira] [Updated] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanlei Yu updated HDFS-17093: - Attachment: HDFS-17093.patch > In the case of all datanodes sending FBR when the namenode restarts (large

[jira] [Commented] (HDFS-15925) The lack of packet-level mirrorError state synchronization in BlockReceiver$PacketResponder can cause the HDFS client to hang

[ https://issues.apache.org/jira/browse/HDFS-15925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744350#comment-17744350 ] ASF GitHub Bot commented on HDFS-15925: --- hadoop-yetus commented on PR #2821: URL: h

[jira] [Created] (HDFS-17106) NameNode can't successfully launched when set dfs.datanode.volumes.replica-add.threadpool.size

ConfX created HDFS-17106: Summary: NameNode can't successfully launched when set dfs.datanode.volumes.replica-add.threadpool.size Key: HDFS-17106 URL: https://issues.apache.org/jira/browse/HDFS-17106 Project:

[jira] [Created] (HDFS-17105) mistakenly purge editLogs even after it is empty in NNStorageRetentionManager

ConfX created HDFS-17105: Summary: mistakenly purge editLogs even after it is empty in NNStorageRetentionManager Key: HDFS-17105 URL: https://issues.apache.org/jira/browse/HDFS-17105 Project: Hadoop HDFS

[jira] [Created] (HDFS-17104) permission deprivation in HDFS without proper check or restoration

ConfX created HDFS-17104: Summary: permission deprivation in HDFS without proper check or restoration Key: HDFS-17104 URL: https://issues.apache.org/jira/browse/HDFS-17104 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-17103) messy file system cleanup in TestNameEditsConfigs

ConfX created HDFS-17103: Summary: messy file system cleanup in TestNameEditsConfigs Key: HDFS-17103 URL: https://issues.apache.org/jira/browse/HDFS-17103 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-17102) Timeout encountered when running TestDataNodeOutlierDetectionViaMetrics

ConfX created HDFS-17102: Summary: Timeout encountered when running TestDataNodeOutlierDetectionViaMetrics Key: HDFS-17102 URL: https://issues.apache.org/jira/browse/HDFS-17102 Project: Hadoop HDFS

[jira] [Created] (HDFS-17101) HostsFileWriter throws a mysterious AssertionError after permission misconfiguration

ConfX created HDFS-17101: Summary: HostsFileWriter throws a mysterious AssertionError after permission misconfiguration Key: HDFS-17101 URL: https://issues.apache.org/jira/browse/HDFS-17101 Project: Hadoop HD

[jira] [Commented] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744292#comment-17744292 ] ASF GitHub Bot commented on HDFS-17094: --- hadoop-yetus commented on PR #5854: URL: h

[jira] [Created] (HDFS-17100) Group.GroupCacheLoader does not properly throw Exception with message

ConfX created HDFS-17100: Summary: Group.GroupCacheLoader does not properly throw Exception with message Key: HDFS-17100 URL: https://issues.apache.org/jira/browse/HDFS-17100 Project: Hadoop HDFS Is

[jira] [Updated] (HDFS-17099) Null Pointer Exception when stop namesystem in HDFS

[ https://issues.apache.org/jira/browse/HDFS-17099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17099: - Description: h2. What happend: Got NullPointerException when stop namesystem in HDFS. h2. Buggy code:   {code:j

[jira] [Created] (HDFS-17099) Null Pointer Exception when stop namesystem in HDFS

ConfX created HDFS-17099: Summary: Null Pointer Exception when stop namesystem in HDFS Key: HDFS-17099 URL: https://issues.apache.org/jira/browse/HDFS-17099 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-17098) DatanodeManager does not handle null storage type properly

ConfX created HDFS-17098: Summary: DatanodeManager does not handle null storage type properly Key: HDFS-17098 URL: https://issues.apache.org/jira/browse/HDFS-17098 Project: Hadoop HDFS Issue Type: Bu

[jira] [Updated] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17097: - Description: h2. What happened: When setting {{dfs.namenode.blockreport.queue.size}} to a large number, {{Block

[jira] [Updated] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17097: - Description: h2. What happened: When setting {{dfs.namenode.blockreport.queue.size}} to a large number, {{Block

[jira] [Created] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

ConfX created HDFS-17097: Summary: Out of Memory when set dfs.namenode.blockreport.queue.size to a large number Key: HDFS-17097 URL: https://issues.apache.org/jira/browse/HDFS-17097 Project: Hadoop HDFS

[jira] [Created] (HDFS-17096) Out of memory exception when mistakenly set ipc.server.read.threadpool.size to a large value

ConfX created HDFS-17096: Summary: Out of memory exception when mistakenly set ipc.server.read.threadpool.size to a large value Key: HDFS-17096 URL: https://issues.apache.org/jira/browse/HDFS-17096 Project:

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Updated] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

[ https://issues.apache.org/jira/browse/HDFS-17095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HDFS-17095: - Description: h2. What happened: When setting {{file.bytes-per-checksum}} to a large number, {{ChecksumFileSyste

[jira] [Created] (HDFS-17095) Out of Memory when mistakenly set file.bytes-per-checksum to a large number

ConfX created HDFS-17095: Summary: Out of Memory when mistakenly set file.bytes-per-checksum to a large number Key: HDFS-17095 URL: https://issues.apache.org/jira/browse/HDFS-17095 Project: Hadoop HDFS

[jira] [Commented] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744210#comment-17744210 ] ASF GitHub Bot commented on HDFS-17094: --- Hexiaoqiao commented on code in PR #5854:

[jira] [Commented] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744183#comment-17744183 ] ASF GitHub Bot commented on HDFS-17088: --- hadoop-yetus commented on PR #5846: URL: h

[jira] [Commented] (HDFS-16991) Fix testMkdirsRaceWithObserverRead

[ https://issues.apache.org/jira/browse/HDFS-16991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744173#comment-17744173 ] ASF GitHub Bot commented on HDFS-16991: --- fanluoo commented on PR #5591: URL: https:

[jira] [Updated] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuyan Zhang updated HDFS-17094: Description: When a block recovery occurs, `RecoveryTaskStriped` in datanode expects `rBlock.getLoc

[jira] [Assigned] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuyan Zhang reassigned HDFS-17094: --- Assignee: Shuyan Zhang > EC: Fix bug in block recovery when there are stale datanodes >

[jira] [Commented] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744159#comment-17744159 ] ASF GitHub Bot commented on HDFS-17094: --- zhangshuyan0 opened a new pull request, #5

[jira] [Updated] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

[ https://issues.apache.org/jira/browse/HDFS-17094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-17094: -- Labels: pull-request-available (was: ) > EC: Fix bug in block recovery when there are stale da

[jira] [Created] (HDFS-17094) EC: Fix bug in block recovery when there are stale datanodes

Shuyan Zhang created HDFS-17094: --- Summary: EC: Fix bug in block recovery when there are stale datanodes Key: HDFS-17094 URL: https://issues.apache.org/jira/browse/HDFS-17094 Project: Hadoop HDFS

[jira] [Created] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

Yanlei Yu created HDFS-17093: Summary: In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting Key: HDFS-17093 URL: https://issues.apach

[jira] [Commented] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744140#comment-17744140 ] ASF GitHub Bot commented on HDFS-17088: --- haiyang1987 commented on PR #5846: URL: ht

[jira] [Commented] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[ https://issues.apache.org/jira/browse/HDFS-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744130#comment-17744130 ] ASF GitHub Bot commented on HDFS-17088: --- haiyang1987 commented on PR #5846: URL: ht

[jira] [Commented] (HDFS-17087) Add Throttler for datanode reading block

[ https://issues.apache.org/jira/browse/HDFS-17087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744125#comment-17744125 ] ASF GitHub Bot commented on HDFS-17087: --- haiyang1987 commented on PR #5845: URL: ht