[jira] [Resolved] (HDFS-16781) setReplication方法不能设置整个目录的副本吗?whole folder?

2022-09-25 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16781. Resolution: Invalid [~zdltvxq] Thanks for your report. IIRC, setReplication api is kept for long

[jira] [Commented] (HDFS-16781) setReplication方法不能设置整个目录的副本吗?whole folder?

2022-09-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17609358#comment-17609358 ] Xiaoqiao He commented on HDFS-16781: Great, welcome and look forwards to more discussion and

[jira] [Updated] (HDFS-16798) SerialNumberMap should decrease current counter if the item exist

2022-10-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16798: --- Summary: SerialNumberMap should decrease current counter if the item exist (was: SerialNumberMap

[jira] [Resolved] (HDFS-16798) SerialNumberMap should decrease current counter if the item exist

2022-10-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16798. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > SerialNumberMap should

[jira] [Updated] (HDFS-16783) Remove the redundant lock in deepCopyReplica and getFinalizedBlocks

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16783: --- Summary: Remove the redundant lock in deepCopyReplica and getFinalizedBlocks (was: Remove the

[jira] [Resolved] (HDFS-16783) Remove the redundant lock in deepCopyReplica and getFinalizedBlocks

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16783. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove the redundant

[jira] [Updated] (HDFS-16787) Remove the redundant lock in DataSetLockManager#removeLock.

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16787: --- Summary: Remove the redundant lock in DataSetLockManager#removeLock. (was: Remove redundant lock in

[jira] [Updated] (HDFS-16787) Remove the redundant lock in DataSetLockManager#removeLock.

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16787: --- Parent: HDFS-15382 Issue Type: Sub-task (was: Improvement) > Remove the redundant lock in

[jira] [Resolved] (HDFS-16787) Remove the redundant lock in DataSetLockManager#removeLock.

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16787. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove the redundant

[jira] [Updated] (HDFS-16783) Remove the redundant lock in deepCopyReplica

2022-10-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16783: --- Parent: HDFS-15382 Issue Type: Sub-task (was: Improvement) > Remove the redundant lock in

[jira] [Resolved] (HDFS-16593) Correct inaccurate BlocksRemoved metric on DataNode side

2022-09-05 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16593. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. >

[jira] [Resolved] (HDFS-16704) Datanode return empty response instead of NPE for GetVolumeInfo during restarting

2022-08-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16704. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk.

[jira] [Commented] (HDFS-16868) Audit log duplicate problem when an ACE occurs in FSNamesystem.

2022-12-11 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17645937#comment-17645937 ] Xiaoqiao He commented on HDFS-16868: [~chino71] Great catch! Thanks for your report. Would you like

[jira] [Resolved] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

2022-12-12 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16868. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix audit log

[jira] [Updated] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

2022-12-12 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16868: --- Issue Type: Improvement (was: Bug) > Fix audit log duplicate issue when an ACE occurs in

[jira] [Updated] (HDFS-16868) Fix audit log duplicate issue when an ACE occurs in FSNamesystem.

2022-12-12 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16868: --- Summary: Fix audit log duplicate issue when an ACE occurs in FSNamesystem. (was: Audit log

[jira] [Commented] (HDFS-16855) Remove the redundant write lock in addBlockPool

2022-11-29 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641058#comment-17641058 ] Xiaoqiao He commented on HDFS-16855: [~dingshun]Thanks for your catches. Sorry I didn't get the

[jira] [Commented] (HDFS-16855) Remove the redundant write lock in addBlockPool

2022-11-30 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641171#comment-17641171 ] Xiaoqiao He commented on HDFS-16855: [~dingshun] Thanks for the detailed explanation. IIRC, at the

[jira] [Commented] (HDFS-16836) StandbyCheckpointer can still trigger rollback fs image after RU is finalized

2022-11-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634619#comment-17634619 ] Xiaoqiao He commented on HDFS-16836: {quote}What we have seen is #1 succeeded but #2 failed so

[jira] [Commented] (HDFS-16836) StandbyCheckpointer can still trigger rollback fs image after RU is finalized

2022-11-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631431#comment-17631431 ] Xiaoqiao He commented on HDFS-16836: [~Lei Yang] Great catch and thanks for your report.

[jira] [Commented] (HDFS-16836) StandbyCheckpointer can still trigger rollback fs image after RU is finalized

2022-11-14 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634144#comment-17634144 ] Xiaoqiao He commented on HDFS-16836: {quote}I mean doCheckpoint can fail and throw exception hence

[jira] [Comment Edited] (HDFS-16836) StandbyCheckpointer can still trigger rollback fs image after RU is finalized

2022-11-14 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634144#comment-17634144 ] Xiaoqiao He edited comment on HDFS-16836 at 11/15/22 4:10 AM: -- {quote}I mean

[jira] [Updated] (HDFS-16785) Avoid to hold write lock to improve performance when add volume.

2022-11-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16785: --- Summary: Avoid to hold write lock to improve performance when add volume. (was: DataNode hold BP

[jira] [Updated] (HDFS-16785) Avoid to hold write lock to improve performance when add volume.

2022-11-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16785: --- Parent: HDFS-15382 Issue Type: Sub-task (was: Improvement) > Avoid to hold write lock to

[jira] [Resolved] (HDFS-16785) Avoid to hold write lock to improve performance when add volume.

2022-11-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16785. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk.

[jira] [Updated] (HDFS-16958) EC: Fix bug in processing EC excess redundancy

2023-03-17 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16958: --- Summary: EC: Fix bug in processing EC excess redundancy (was: Fix bug in processing EC excess

[jira] [Resolved] (HDFS-16948) Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer

2023-03-23 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16948. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Update log of

[jira] [Updated] (HDFS-16948) Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer

2023-03-16 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16948: --- Summary: Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by

[jira] [Commented] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

2023-03-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17698373#comment-17698373 ] Xiaoqiao He commented on HDFS-16939: [~ste...@apache.org] Thanks for your reminder, [~zhangshuyan] is

[jira] [Updated] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

2023-03-11 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16939: --- Fix Version/s: 3.3.5 > Fix the thread safety bug in LowRedundancyBlocks >

[jira] [Commented] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

2023-03-11 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699191#comment-17699191 ] Xiaoqiao He commented on HDFS-16939: [~ste...@apache.org] This PR has been committed to branch-3.3

[jira] [Resolved] (HDFS-16939) Fix the thread safety bug in LowRedundancyBlocks

2023-03-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16939. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix the thread safety

[jira] [Resolved] (HDFS-16986) EC: Fix locationBudget in getListing()

2023-04-24 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16986. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > EC: Fix locationBudget

[jira] [Resolved] (HDFS-16997) Set the locale to avoid printing useless logs in BlockSender

2023-05-02 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16997. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Set the locale to

[jira] [Resolved] (HDFS-16964) Improve processing of excess redundancy after failover

2023-03-28 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16964. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve processing of

[jira] [Resolved] (HDFS-16974) Consider volumes average load of each DataNode when choosing target.

2023-04-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16974. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Consider volumes

[jira] [Updated] (HDFS-16974) Consider volumes average load of each DataNode when choosing target.

2023-04-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16974: --- Summary: Consider volumes average load of each DataNode when choosing target. (was: Consider load

[jira] [Assigned] (HDFS-16898) Make write lock fine-grain in processCommandFromActor method

2023-02-05 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HDFS-16898: -- Assignee: ZhangHB > Make write lock fine-grain in processCommandFromActor method >

[jira] [Commented] (HDFS-16898) Make write lock fine-grain in processCommandFromActor method

2023-02-05 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684410#comment-17684410 ] Xiaoqiao He commented on HDFS-16898: Add ZhangHB to contributors group, and assign this ticket to

[jira] [Updated] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

2023-02-07 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16898: --- Summary: Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

[jira] [Resolved] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

2023-02-07 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16898. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove write lock for

[jira] [Commented] (HDFS-17046) Add metrics or delete directly when path can not parsed in Trash.

2023-06-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732023#comment-17732023 ] Xiaoqiao He commented on HDFS-17046: Thanks [~zhanghaobo] for your report. Do you mean that there

[jira] [Commented] (HDFS-17071) Erasure coding: delete duplicated ec block with same block group id on one datanode.

2023-07-07 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740925#comment-17740925 ] Xiaoqiao He commented on HDFS-17071: I found that this JIRA report the same issue as HDFS-17050 which

[jira] [Commented] (HDFS-17072) DFSAdmin: add a triggerVolumeScanner command

2023-07-07 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740926#comment-17740926 ] Xiaoqiao He commented on HDFS-17072: Thanks to involve me here. +1 from my side. > DFSAdmin: add a

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

2023-07-18 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744424#comment-17744424 ] Xiaoqiao He commented on HDFS-17093: [~yuyanlei] Thanks for your report, Would you try to dig the log

[jira] [Commented] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

2023-07-18 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744429#comment-17744429 ] Xiaoqiao He commented on HDFS-17097: [~FuzzingTeam] Thanks for your report. We received many JIRAs

[jira] [Comment Edited] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

2023-07-18 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744429#comment-17744429 ] Xiaoqiao He edited comment on HDFS-17097 at 7/19/23 3:54 AM: - [~FuzzingTeam]

[jira] [Resolved] (HDFS-17073) Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute

2023-07-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17073. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Enhance the warning

[jira] [Resolved] (HDFS-17033) Update fsck to display stale state info of blocks accurately

2023-07-10 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17033. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Update fsck to display

[jira] [Created] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

2023-07-16 Thread Xiaoqiao He (Jira)
Xiaoqiao He created HDFS-17090: -- Summary: Decommission will be stuck for long time when restart because overlapped process Register and BlockReport. Key: HDFS-17090 URL:

[jira] [Commented] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

2023-07-17 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743790#comment-17743790 ] Xiaoqiao He commented on HDFS-17090: We mark `DatanodeProtocol#registerDatanode` interface as

[jira] [Commented] (HDFS-17093) In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting

2023-07-19 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744526#comment-17744526 ] Xiaoqiao He commented on HDFS-17093: {quote}# DN wants to send 12 reports but only sent 1 report. #

[jira] [Updated] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

2023-05-25 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17026: --- Summary: RBF: NamenodeHeartbeatService should update JMX report with configurable frequency (was:

[jira] [Commented] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

2023-05-25 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726451#comment-17726451 ] Xiaoqiao He commented on HDFS-17026: {quote}We should configure the NamenodeHeartbeatService so it

[jira] [Resolved] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

2023-05-31 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17026. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF:

[jira] [Commented] (HDFS-17026) RBF: NamenodeHeartbeatService should update JMX report with configurable frequency

2023-05-31 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728183#comment-17728183 ] Xiaoqiao He commented on HDFS-17026: [~hchaverri] Please submit this PR for branch-3.3 via Github, It

[jira] [Commented] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

2023-06-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17730500#comment-17730500 ] Xiaoqiao He commented on HDFS-17003: cherry-pick to branch-3.3. > Erasure Coding: invalidate wrong

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

2023-06-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Fix Version/s: 3.3.6 > Erasure Coding: invalidate wrong block after reporting bad blocks from >

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

2023-06-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Component/s: namenode > Erasure Coding: invalidate wrong block after reporting bad blocks from >

[jira] [Resolved] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

2023-06-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17003. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Erasure Coding:

[jira] [Updated] (HDFS-17003) Erasure Coding: invalidate wrong block after reporting bad blocks from datanode

2023-06-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17003: --- Summary: Erasure Coding: invalidate wrong block after reporting bad blocks from datanode (was:

[jira] [Resolved] (HDFS-17037) Consider nonDfsUsed when running balancer

2023-06-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17037. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Consider nonDfsUsed

[jira] [Updated] (HDFS-17037) Consider nonDfsUsed when running balancer

2023-06-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17037: --- Component/s: balamcer Issue Type: Improvement (was: Bug) > Consider nonDfsUsed when running

[jira] [Updated] (HDFS-17037) Consider nonDfsUsed when running balancer

2023-06-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17037: --- Component/s: balancer & mover (was: balamcer) > Consider nonDfsUsed when

[jira] [Updated] (HDFS-16909) Improve ReplicaMap#mergeAll method.

2023-05-21 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16909: --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Affects Version/s: (was: 3.3.4)

[jira] [Commented] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

2023-06-11 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17731448#comment-17731448 ] Xiaoqiao He commented on HDFS-16898: Committed to branch-3.3. > Remove write lock for

[jira] [Updated] (HDFS-16898) Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat

2023-06-11 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16898: --- Fix Version/s: 3.3.7 > Remove write lock for processCommandFromActor of DataNode to reduce impact on

[jira] [Commented] (HDFS-17138) RBF: We changed the hadoop.security.auth_to_local configuration of one router, the other routers stopped working

2023-08-02 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750537#comment-17750537 ] Xiaoqiao He commented on HDFS-17138: Thanks. IMO, it may be one common issue, not affect to DFSRouter

[jira] [Resolved] (HDFS-17134) RBF: Fix duplicate results of getListing through Router.

2023-08-01 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17134. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF: Fix duplicate

[jira] [Updated] (HDFS-17134) RBF: Fix duplicate results of getListing through Router.

2023-08-01 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17134: --- Component/s: rbf > RBF: Fix duplicate results of getListing through Router. >

[jira] [Resolved] (HDFS-17112) Show decommission duration in JMX and HTML

2023-07-24 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17112. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Show decommission

[jira] [Updated] (HDFS-17112) Show decommission duration in JMX and HTML

2023-07-24 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17112: --- Component/s: namenode > Show decommission duration in JMX and HTML >

[jira] [Commented] (HDFS-17097) Out of Memory when set dfs.namenode.blockreport.queue.size to a large number

2023-07-23 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746189#comment-17746189 ] Xiaoqiao He commented on HDFS-17097: [~FuzzingTeam] Great, I try to search JIRA from your report,

[jira] [Updated] (HDFS-17090) Decommission will be stuck for long time when restart because overlapped process Register and BlockReport.

2023-07-24 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17090: --- Description: I met one corner case recently, which decommission DataNode impact performance of

[jira] [Commented] (HDFS-17138) RBF: We changed the hadoop.security.auth_to_local configuration of one router, the other routers stopped working

2023-08-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17751456#comment-17751456 ] Xiaoqiao He commented on HDFS-17138: {quote}Other modules are also using subclasses of this

[jira] [Assigned] (HDFS-17140) Optimize the BPOfferService.reportBadBlocks() method

2023-08-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HDFS-17140: -- Assignee: Liangjun He > Optimize the BPOfferService.reportBadBlocks() method >

[jira] [Assigned] (HDFS-17141) Optimize the default parameters for the FileUtil.isRegularFile() method

2023-08-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HDFS-17141: -- Assignee: Liangjun He > Optimize the default parameters for the FileUtil.isRegularFile()

[jira] [Updated] (HDFS-17137) Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication.

2023-08-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17137: --- Summary: Standby/Observer NameNode skip to handle redundant replica block logic when set decrease

[jira] [Updated] (HDFS-17137) Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication.

2023-08-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17137: --- Component/s: namenode > Standby/Observer NameNode skip to handle redundant replica block logic when

[jira] [Resolved] (HDFS-17137) Standby/Observer NameNode skip to handle redundant replica block logic when set decrease replication.

2023-08-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17137. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Standby/Observer

[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.

2023-08-09 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17752284#comment-17752284 ] Xiaoqiao He commented on HDFS-17149: Hi [~zhanghaobo], Please check if HDFS-15079 can solve this

[jira] [Updated] (HDFS-17117) Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17117: --- Component/s: namenode > Print reconstructionQueuesInitProgress periodically when BlockManager >

[jira] [Resolved] (HDFS-17117) Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17117. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Print

[jira] [Updated] (HDFS-17117) Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17117: --- Summary: Print reconstructionQueuesInitProgress periodically when BlockManager

[jira] [Updated] (HDFS-17116) RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService.

2023-07-28 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17116: --- Summary: RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService. (was:

[jira] [Resolved] (HDFS-17116) RBF: Update invoke millisecond time as monotonicNow() in RouterSafemodeService.

2023-07-28 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17116. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF: Update invoke

[jira] [Commented] (HDFS-6994) libhdfs3 - A native C/C++ HDFS client

2023-06-21 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735663#comment-17735663 ] Xiaoqiao He commented on HDFS-6994: --- Hi [~wangzw],[~cmccabe], Thanks for your great works. This is very

[jira] [Resolved] (HDFS-17049) EC: Fix duplicate block group IDs generated by SequentialBlockGroupIdGenerator

2023-06-20 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17049. Resolution: Not A Problem > EC: Fix duplicate block group IDs generated by

[jira] [Resolved] (HDFS-17052) Improve BlockPlacementPolicyRackFaultTolerant to avoid choose nodes failed when no enough Rack.

2023-07-02 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17052. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Committed to trunk. And

[jira] [Resolved] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

2023-06-27 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17044. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Set size of non-exist

[jira] [Updated] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

2023-06-27 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17044: --- Component/s: namenode > Set size of non-exist block to NO_ACK when process FBR or IBR to avoid >

[jira] [Updated] (HDFS-17044) Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from DataNode

2023-06-27 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17044: --- Summary: Set size of non-exist block to NO_ACK when process FBR or IBR to avoid useless report from

[jira] [Commented] (HDFS-17048) FSNamesystem.delete() maybe cause data residue when active namenode crash or shutdown

2023-06-14 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732434#comment-17732434 ] Xiaoqiao He commented on HDFS-17048: [~liuguanghua] Thanks for your report. Sorry I didn't understand

[jira] [Commented] (HDFS-17003) Erasure coding: invalidate wrong block after reporting bad blocks from datanode

2023-05-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720752#comment-17720752 ] Xiaoqiao He commented on HDFS-17003: [~zhanghaobo] Thanks for your report. Any more information? The

[jira] [Resolved] (HDFS-16999) Fix wrong use of processFirstBlockReport()

2023-05-08 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16999. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix wrong use of

[jira] [Resolved] (HDFS-16985) Fix data missing issue when delete local block file.

2023-05-14 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-16985. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Target Version/s: (was: 3.3.1)

[jira] [Updated] (HDFS-16985) Fix data missing issue when delete local block file.

2023-05-14 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16985: --- Summary: Fix data missing issue when delete local block file. (was: delete local block file when

[jira] [Updated] (HDFS-16663) EC: Allow block reconstruction pending timeout refreshable to increase decommission performance

2024-01-23 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-16663: --- Summary: EC: Allow block reconstruction pending timeout refreshable to increase decommission

[jira] [Resolved] (HDFS-17270) RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to get token in some case

2023-12-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-17270. Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > RBF: Fix

[jira] [Updated] (HDFS-17270) RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to get token in some case

2023-12-06 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HDFS-17270: --- Component/s: rbf > RBF: Fix ZKDelegationTokenSecretManagerImpl use closed zookeeper client to >

<    5   6   7   8   9   10   11   >