[jira] [Resolved] (HDFS-17599) EC: Fix the mismatch between locations and indices for mover

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-17599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-17599.
---
Resolution: Fixed

> EC: Fix the mismatch between locations and indices for mover
> 
>
> Key: HDFS-17599
> URL: https://issues.apache.org/jira/browse/HDFS-17599
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: balancer & mover
>Affects Versions: 3.3.0, 3.4.0
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.5.0
>
> Attachments: image-2024-08-03-17-59-08-059.png, 
> image-2024-08-03-18-00-01-950.png
>
>
> We set the EC policy to (6+3) and also have nodes that were in state 
> ENTERING_MAINTENANCE.
>  
> When we move the data of some directories from SSD to HDD, some blocks move 
> fail due to disk full, as shown in the figure below 
> (blk_-9223372033441574269).
> We tried to move again and found the following error "{color:#ff}Replica 
> does not exist{color}".
> Observing the information of fsck, it can be found that the wrong 
> blockid(blk_-9223372033441574270) was found when moving block.
>  
> {*}Mover Logs{*}:
> !image-2024-08-03-17-59-08-059.png|width=741,height=85!
>  
> {*}FSCK Info{*}:
> !image-2024-08-03-18-00-01-950.png|width=738,height=120!
>  
> {*}Root Cause{*}:
> Similar to this HDFS-16333, when mover is initialized, only the `LIVE` node 
> is processed. As a result, the datanode in the `ENTERING_MAINTENANCE` state 
> in the locations is filtered when initializing `DBlockStriped`, but the 
> indices are not adapted, resulting in a mismatch between the location and 
> indices lengths. Finally, ec block calculates the wrong blockid when getting 
> internal block (see `DBlockStriped#getInternalBlock`).
>  
> We added debug logs, and a few key messages are shown below. 
> {color:#ff}The result is an incorrect correspondence: xx.xx.7.31 -> 
> -9223372033441574270{color}.
> {code:java}
> DBlock getInternalBlock(StorageGroup storage) {
>   // storage == xx.xx.7.31
>   // idxInLocs == 1 (location ([xx.xx.,85.29:DISK, xx.xx.7.31:DISK, 
> xx.xx.207.22:DISK, xx.xx.8.25:DISK, xx.xx.79.30:DISK, xx.xx.87.21:DISK, 
> xx.xx.8.38:DISK]), xx.xx.179.31 is in the ENTERING_MAINTENANCE state is 
> filtered)
>   int idxInLocs = locations.indexOf(storage);
>   if (idxInLocs == -1) {
> return null;
>   }
>   // idxInGroup == 2 (indices is [1,2,3,4,5,6,7,8])   
>   byte idxInGroup = indices[idxInLocs];
>   // blkId: -9223372033441574272 + 2 = -9223372033441574270
>   long blkId = getBlock().getBlockId() + idxInGroup;
>   long numBytes = getInternalBlockLength(getNumBytes(), cellSize,
>   dataBlockNum, idxInGroup);
>   Block blk = new Block(getBlock());
>   blk.setBlockId(blkId);
>   blk.setNumBytes(numBytes);
>   DBlock dblk = new DBlock(blk);
>   dblk.addLocation(storage);
>   return dblk;
> } {code}
> {*}Solution{*}:
> When initializing DBlockStriped, if any location is filtered out, we need to 
> remove the corresponding element in the indices to do the adaptation.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17564) Erasure Coding: Fix the issue of inaccurate metrics when decommission mark busy DN

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17564:
-

 Summary: Erasure Coding: Fix the issue of inaccurate metrics when 
decommission mark busy DN
 Key: HDFS-17564
 URL: https://issues.apache.org/jira/browse/HDFS-17564
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17559) Fix the uuid as null in NameNodeMXBean

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17559:
-

 Summary: Fix the uuid as null in NameNodeMXBean
 Key: HDFS-17559
 URL: https://issues.apache.org/jira/browse/HDFS-17559
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu


If there is datanode info in includes, but the datanode service is not 
currently started, the uuid of the datanode will be null. When getting the 
DeadNodes DeadNodes metric, the following exception will occur:
{code:java}
2024-06-26 17:06:49,698 ERROR jmx.JMXJsonServlet 
(JMXJsonServlet.java:writeAttribute(345)) [qtp1107412069-7704] - getting 
attribute DeadNodes of Hadoop:service=NameNode,name=NameNodeInfo threw an 
exception javax.management.RuntimeMBeanException: 
java.lang.NullPointerException: null value in entry: uuid=null
        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
        at 
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
        at 
org.apache.hadoop.jmx.JMXJsonServlet.writeAttribute(JMXJsonServlet.java:338)
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17557) Fix bug for TestRedundancyMonitor#testChooseTargetWhenAllDataNodesStop

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17557:
-

 Summary: Fix bug for 
TestRedundancyMonitor#testChooseTargetWhenAllDataNodesStop
 Key: HDFS-17557
 URL: https://issues.apache.org/jira/browse/HDFS-17557
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-17464) Improve some logs output in class FsDatasetImpl

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-17464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-17464.
---
Fix Version/s: 3.5.0
   Resolution: Resolved

> Improve some logs output in class FsDatasetImpl
> ---
>
> Key: HDFS-17464
> URL: https://issues.apache.org/jira/browse/HDFS-17464
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Affects Versions: 3.4.0
>Reporter: farmmamba
>Assignee: farmmamba
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 3.5.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-17451) RBF: fix spotbugs for redundant nullcheck of dns.

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-17451.
---
Fix Version/s: 3.5.0
 Hadoop Flags: Reviewed
   Resolution: Fixed

> RBF: fix spotbugs for redundant nullcheck of dns.
> -
>
> Key: HDFS-17451
> URL: https://issues.apache.org/jira/browse/HDFS-17451
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Jian Zhang
>Assignee: Jian Zhang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.5.0
>
>
> h2. Dodgy code Warnings
> ||Code||Warning||
> |RCN|Redundant nullcheck of dns, which is known to be non-null in 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getCachedDatanodeReport(HdfsConstants$DatanodeReportType)|
> | |[Bug type RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE (click for 
> details)|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6655/8/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf-warnings.html#RCN_REDUNDANT_NULLCHECK_OF_NONNULL_VALUE]
> In class org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer
> In method 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.getCachedDatanodeReport(HdfsConstants$DatanodeReportType)
> Value loaded from dns
> Return value of 
> org.apache.hadoop.thirdparty.com.google.common.cache.LoadingCache.get(Object) 
> of type Object
> Redundant null check at RouterRpcServer.java:[line 1093]|



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [ANNOUNCE] New Hadoop Committer - Haiyang Hu

[haiyang hu]

Thanks!  Looking forward to participating more in the Apache Hadoop
community!

Yuanbo Liu  于2024年4月22日周一 15:26写道：

> Congratulations
>
> On Mon, Apr 22, 2024 at 12:14 PM Ayush Saxena  wrote:
>
>> Congratulations Haiyang!!!
>>
>> -Ayush
>>
>> > On 22 Apr 2024, at 9:41 AM, Xiaoqiao He  wrote:
>> >
>> > I am pleased to announce that Haiyang Hu has been elected as
>> > a committer on the Apache Hadoop project. We appreciate all of
>> > Haiyang's work, and look forward to her/his continued contributions.
>> >
>> > Congratulations and Welcome, Haiyang!
>> >
>> > Best Regards,
>> > - He Xiaoqiao
>> > (On behalf of the Apache Hadoop PMC)
>> >
>> > -
>> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
>> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>>
>>

[jira] [Created] (HDFS-17461) Fix spotbugs in PeerCache#getInternal

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17461:
-

 Summary: Fix spotbugs in PeerCache#getInternal
 Key: HDFS-17461
 URL: https://issues.apache.org/jira/browse/HDFS-17461
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


Fix spotbugs in PeerCache#getInternal 

https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6710/4/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17455) Fix Client throw IndexOutOfBoundsException in DFSInputStream#fetchBlockAt

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17455:
-

 Summary: Fix Client throw IndexOutOfBoundsException in 
DFSInputStream#fetchBlockAt
 Key: HDFS-17455
 URL: https://issues.apache.org/jira/browse/HDFS-17455
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17448) Enhance the stability of the unit test TestDiskBalancerCommand

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17448:
-

 Summary:  Enhance the stability of the unit test 
TestDiskBalancerCommand 
 Key: HDFS-17448
 URL: https://issues.apache.org/jira/browse/HDFS-17448
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu


TestDiskBalancerCommand#testDiskBalancerQueryWithoutSubmitAndMultipleNodes  
frequently fails tests, such as:

https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1540/testReport/junit/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/

https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6637/1/testReport/org.apache.hadoop.hdfs.server.diskbalancer.command/TestDiskBalancerCommand/testDiskBalancerQueryWithoutSubmitAndMultipleNodes/

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17431) Fix log format for BlockRecoveryWorker#recoverBlocks

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17431:
-

 Summary: Fix log format for BlockRecoveryWorker#recoverBlocks
 Key: HDFS-17431
 URL: https://issues.apache.org/jira/browse/HDFS-17431
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


Fix log format for BlockRecoveryWorker#recoverBlocks



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17430) RecoveringBlock will skip no live replicas when get block recovery command.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17430:
-

 Summary: RecoveringBlock will skip no live replicas when get block 
recovery command.
 Key: HDFS-17430
 URL: https://issues.apache.org/jira/browse/HDFS-17430
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17426) Remove Invalid FileSystemECReadStats logic in DFSInputStream

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17426:
-

 Summary: Remove Invalid FileSystemECReadStats logic in 
DFSInputStream
 Key: HDFS-17426
 URL: https://issues.apache.org/jira/browse/HDFS-17426
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17406) Suppress UnresolvedPathException in hdfs router log

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17406:
-

 Summary: Suppress UnresolvedPathException in hdfs router log
 Key: HDFS-17406
 URL: https://issues.apache.org/jira/browse/HDFS-17406
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


UnresolvedPathException as a normal process of resolving symlinks, the router 
server doesn't need to be logged at all .





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Discussion about NameNode Fine-grained locking

[haiyang hu]

Thank you for raising the issue of this long-standing bottleneck, this will
be a very important improvement!

Hopefully can participate and push forward together.

Best Regards~

Brahma Reddy Battula  于2024年2月3日周六 00:40写道：

> Thanks for bringing this and considering all the history around this.
> One of the outstanding bottleneck(global lock) from a long time.
>
> Hopefully we can push forward this time.
>
>
> On Fri, Feb 2, 2024 at 12:23 PM Hui Fei  wrote:
>
> > Thanks for driving this. It's very meaningful. The performance
> improvement
> > looks very good.
> >
> > Many users are facing the write performance issue. As far as I know, some
> > companies already implemented the similar idea on their internal
> branches.
> > But the internal branch is very different from the community one. So it's
> > very hard to be in sync with the community. If this improvement can be
> > involved in the community, that would be great to both end-user and the
> > community.
> >
> > It is very worth doing.
> >
> > Zengqiang XU  于2024年2月2日周五 11:07写道：
> >
> > > Hi everyone
> > >
> > > I have started a discussion about NameNode Fine-grained Locking to
> > improve
> > > performance of write operations in NameNode.
> > >
> > > I started this discussion again for serval main reasons:
> > > 1. We have implemented it and gained nearly 7x performance improvement
> in
> > > our prod environment
> > > 2. Many other companies made similar improvements based on their
> internal
> > > branch.
> > > 3. This topic has been discussed for a long time, but still without any
> > > results.
> > >
> > > I hope we can push this important improvement in the community so that
> > all
> > > end-users can enjoy this significant improvement.
> > >
> > > I'd really appreciate you can join in and work with me to push this
> > feature
> > > forward.
> > >
> > > Thanks very much.
> > >
> > > Ticket: HDFS-17366 
> > > Design: NameNode Fine-grained locking based on directory tree
> > > <
> > >
> >
> https://docs.google.com/document/d/1bVBQcI4jfzS0UrczB7UhsrQTXmrERGvBV-a9W3HCCjk/edit?usp=sharing
> > > >
> > >
> >
>

[jira] [Created] (HDFS-17369) Add uuid into datanode info for NameNodeMXBean

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17369:
-

 Summary: Add uuid into datanode info for NameNodeMXBean
 Key: HDFS-17369
 URL: https://issues.apache.org/jira/browse/HDFS-17369
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


Add uuid into datanode info for NameNodeMXBean and convenient to locate issues.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17361) DiskBalancer: Query command support with multiple nodes

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17361:
-

 Summary: DiskBalancer: Query command support with multiple nodes
 Key: HDFS-17361
 URL: https://issues.apache.org/jira/browse/HDFS-17361
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17352) Add configuration to control whether DN delete this replica from disk when client requests a missing block

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17352:
-

 Summary: Add configuration to control whether DN delete this 
replica from disk when client requests a missing block 
 Key: HDFS-17352
 URL: https://issues.apache.org/jira/browse/HDFS-17352
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: datanode
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17346) Fix DirectoryScanner check mark the normal blocks as corrupt.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17346:
-

 Summary: Fix DirectoryScanner check mark the normal blocks as 
corrupt.
 Key: HDFS-17346
 URL: https://issues.apache.org/jira/browse/HDFS-17346
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17342) Fix DataNode may invalidates normal block causing missing block

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17342:
-

 Summary: Fix DataNode may invalidates normal block causing missing 
block
 Key: HDFS-17342
 URL: https://issues.apache.org/jira/browse/HDFS-17342
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


When users read an append file, occasional exceptions may occur, such as 
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: xxx.

This can happen if one thread is reading the block while writer thread is 
finalizing it simultaneously.

*Root cause:*

The reader thread obtains a RBW replica from VolumeMap, such as: 
blk_xxx_xxx[RBW] and  the data file should be in /XXX/rbw/blk_xxx.

Simultaneously, the writer thread will finalize this block, moving it from the 
RBW directory to the FINALIZE directory. the data file is move from 
/XXX/rbw/block_xxx to /XXX/finalize/block_xxx.
The reader thread attempts to open this data input stream but encounters a 
FileNotFoundException because the data file /XXX/rbw/blk_xxx or meta file 
/XXX/rbw/blk_xxx_xxx doesn't exist at this moment.
The reader thread  will treats this block as corrupt, removes the replica from 
the volume map, and the DataNode reports the deleted block to the NameNode.
The NameNode removes this replica for the block.
If the current file replication is 1, this file will cause a missing block 
issue until this DataNode executes the DirectoryScanner again.


As described above, when the reader thread encountered FileNotFoundException is 
as expected, because the file is moved.
So we need to add a double check to the invalidateMissingBlock logic to verify 
whether the data file or meta file exists to avoid similar cases.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17310) DiskBalancer: Enhance the log message

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17310:
-

 Summary: DiskBalancer: Enhance the log message
 Key: HDFS-17310
 URL: https://issues.apache.org/jira/browse/HDFS-17310
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17298) Fix NPE in DataNode.handleBadBlock and BlockSender

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17298:
-

 Summary: Fix NPE in DataNode.handleBadBlock and BlockSender
 Key: HDFS-17298
 URL: https://issues.apache.org/jira/browse/HDFS-17298
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17297) The NameNode should remove block from the BlocksMap if the block is marked as deleted.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17297:
-

 Summary: The NameNode should remove block from the BlocksMap if 
the block is marked as deleted.
 Key: HDFS-17297
 URL: https://issues.apache.org/jira/browse/HDFS-17297
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


When call internalReleaseLease method:

{code:java}
boolean internalReleaseLease(
...
int minLocationsNum = 1;
if (lastBlock.isStriped()) {
  minLocationsNum = ((BlockInfoStriped) lastBlock).getRealDataBlockNum();
}

if (uc.getNumExpectedLocations() < minLocationsNum &&
lastBlock.getNumBytes() == 0) {
  // There is no datanode reported to this block.
  // may be client have crashed before writing data to pipeline.
  // This blocks doesn't need any recovery.
  // We can remove this block and close the file.
  pendingFile.removeLastBlock(lastBlock);
  finalizeINodeFileUnderConstruction(src, pendingFile,
  iip.getLatestSnapshotId(), false); 
...
}
{code}
 if the condition uc.getNumExpectedLocations() < minLocationsNum && 
lastBlock.getNumBytes() == 0 is met during the execution of UNDER_RECOVERY 
logic, the block is removed from the block list in the inode file and marked as 
deleted. 
However it is not removed from the BlocksMap, it may cause memory leak.
Therefore it is necessary to remove the block from the BlocksMap at this point 
as well.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17279) RBF: Fix link to Fedbalance document

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17279:
-

 Summary: RBF: Fix link to Fedbalance document 
 Key: HDFS-17279
 URL: https://issues.apache.org/jira/browse/HDFS-17279
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: rbf
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17250) EditLogTailer#triggerActiveLogRoll should handle thread Interrupted

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17250:
-

 Summary: EditLogTailer#triggerActiveLogRoll should handle thread 
Interrupted
 Key: HDFS-17250
 URL: https://issues.apache.org/jira/browse/HDFS-17250
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


*Issue:*
When the NameNode attempts to trigger a log roll and the cachedActiveProxy is a 
"shut down NameNode," it is unable to establish a network connection. This 
results in a timeout during the socket connection phase,
which has a set timeout of 90 seconds. Since the asynchronous call for 
"Triggering log roll" has a waiting time of 60 seconds,
it triggers a timeout and initiates a "cancel" operation, causing the executing 
thread to receive an "Interrupted" signal and throwing a 
"java.io.InterruptedIOException" exception.

Currently, the logic not to handle interrupted signal, and the 
"getActiveNodeProxy" method hasn't reached the maximum retry limit, the overall 
execution process doesn't exit and it continues to attempt to
call the "rollEditLog" on the next NameNode in the list. However when a socket 
connection is established, it throws a 
"java.nio.channels.ClosedByInterruptException" exception due to the thread being
in an "Interrupted" state.
this cycle repeats until it reaches the maximum retry limit (nnCount * 
maxRetries) will exits.

However in the next cycle of "Triggering log roll," it continues to traverse 
the NameNode list and encounters the same issue and the cachedActiveProxy is 
still a "shut down NameNode."
This eventually results in the NameNode being unable to successfully complete 
the "Triggering log roll" operation.

To optimize this, we need to handle the thread being interrupted and exit the 
execution



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17247:
-

 Summary: Improve 
AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic 
 Key: HDFS-17247
 URL: https://issues.apache.org/jira/browse/HDFS-17247
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: namanode
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17243) Add the parameter storage type for getBlocks method

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17243:
-

 Summary: Add the parameter storage type for getBlocks method
 Key: HDFS-17243
 URL: https://issues.apache.org/jira/browse/HDFS-17243
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: balamcer
Reporter: Haiyang Hu
Assignee: Haiyang Hu


When Balancer is running, it is found that there are many logs, such as 

{code:java}
INFO  balancer.Dispatcher (Dispatcher.java:markMovedIfGoodBlock(306)) - No 
striped internal block on source xxx:50010:SSD, block blk_-xxx_xxx 
size=982142783. Skipping.
{code}
these logs show that Balancer cannot to balancer SSD type source， and it causes 
that Balancer will frequently get blocks from NN through getBlocks RPC.

The main reason is the storage type in the current Source is SSD, but now 
getBlocks obtains all list of blocks belonging to datanode, so need add the 
parameter storage type for getBlocks method







--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17235) Fix javadoc errors in BlockManager

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17235:
-

 Summary: Fix javadoc errors in BlockManager
 Key: HDFS-17235
 URL: https://issues.apache.org/jira/browse/HDFS-17235
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: documentation
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17217) Add lifeline RPC start up log when NameNode#startCommonServices

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17217:
-

 Summary: Add lifeline RPC start up  log when 
NameNode#startCommonServices
 Key: HDFS-17217
 URL: https://issues.apache.org/jira/browse/HDFS-17217
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


If start up the lifeline RPC server in NameNode and need add lifeline RPC start 
up log when NameNode#startCommonServices



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17208) Add the metrics PendingAsyncDiskOperations in datanode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17208:
-

 Summary: Add the metrics PendingAsyncDiskOperations  in datanode 
 Key: HDFS-17208
 URL: https://issues.apache.org/jira/browse/HDFS-17208
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: datanode
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17205) HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17205:
-

 Summary: HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be 
configurable
 Key: HDFS-17205
 URL: https://issues.apache.org/jira/browse/HDFS-17205
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17194) Enhance the log message for striped block recovery

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17194:
-

 Summary: Enhance the log message for striped block recovery
 Key: HDFS-17194
 URL: https://issues.apache.org/jira/browse/HDFS-17194
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17184) In BlockReceiver method need throw DiskOutOfSpaceException

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17184:
-

 Summary: In BlockReceiver method need throw DiskOutOfSpaceException
 Key: HDFS-17184
 URL: https://issues.apache.org/jira/browse/HDFS-17184
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17177) ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17177:
-

 Summary: ErasureCodingWork reconstruct ignore the block length is 
Long.MAX_VALUE.
 Key: HDFS-17177
 URL: https://issues.apache.org/jira/browse/HDFS-17177
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17172) Support FSNamesystemLock Parameters reconfigurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17172:
-

 Summary: Support FSNamesystemLock Parameters reconfigurable
 Key: HDFS-17172
 URL: https://issues.apache.org/jira/browse/HDFS-17172
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


For Namesystem lock some parameters such as 
"dfs.namenode.lock.detailed-metrics.enabled", 
"dfs.namenode.read-lock-reporting-threshold-ms", 
"dfs.namenode.write-lock-reporting-threshold-ms"

we will support reconfigurable these parameters without namenode restart is 
convenient for us to check the metrics operation of FSNamesystemLock.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17137) Standby/Observer NameNode should not handle redundant replica block logic when set decrease replication

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17137:
-

 Summary:  Standby/Observer NameNode should not  handle redundant 
replica block logic  when set decrease replication
 Key: HDFS-17137
 URL: https://issues.apache.org/jira/browse/HDFS-17137
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17135) Fsck -blockId display excess state info of blocks accurately

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17135:
-

 Summary: Fsck -blockId display excess state info of blocks 
accurately
 Key: HDFS-17135
 URL: https://issues.apache.org/jira/browse/HDFS-17135
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17117) BlockManager#processMisReplicatesAsync can periodically print reconstructionQueuesInitProgress

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17117:
-

 Summary: BlockManager#processMisReplicatesAsync can periodically 
print reconstructionQueuesInitProgress
 Key: HDFS-17117
 URL: https://issues.apache.org/jira/browse/HDFS-17117
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17116) Reset startupTime and enterSafeModeTime if check time interval is negative during router safe mode exit check

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17116:
-

 Summary: Reset startupTime and enterSafeModeTime if check time 
interval is negative during router safe mode exit check
 Key: HDFS-17116
 URL: https://issues.apache.org/jira/browse/HDFS-17116
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17088) Improve the debug verifyEC and dfsrouteradmin commands in HDFSCommands.md

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17088:
-

 Summary: Improve the debug verifyEC and dfsrouteradmin commands in 
HDFSCommands.md
 Key: HDFS-17088
 URL: https://issues.apache.org/jira/browse/HDFS-17088
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17087) Add Throttler for datanode reading block

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17087:
-

 Summary: Add Throttler for datanode reading block 
 Key: HDFS-17087
 URL: https://issues.apache.org/jira/browse/HDFS-17087
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17086) Fix the parameter settings in TestDiskspaceQuotaUpdate#updateCountForQuota.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17086:
-

 Summary: Fix the parameter settings in 
TestDiskspaceQuotaUpdate#updateCountForQuota.
 Key: HDFS-17086
 URL: https://issues.apache.org/jira/browse/HDFS-17086
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17082) Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17082:
-

 Summary: Add documentation for provisionSnapshotTrash command to 
HDFSCommands.md  and HdfsSnapshots.md
 Key: HDFS-17082
 URL: https://issues.apache.org/jira/browse/HDFS-17082
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17081) Append ec file check if a block is replicated to at least the minimum replication need consider ec block

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17081:
-

 Summary: Append ec file check if a block is replicated to at least 
the minimum replication need consider ec block
 Key: HDFS-17081
 URL: https://issues.apache.org/jira/browse/HDFS-17081
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17076) Remove the unused method isSlownodeByNameserviceId in DataNode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17076:
-

 Summary: Remove the unused method isSlownodeByNameserviceId in 
DataNode
 Key: HDFS-17076
 URL: https://issues.apache.org/jira/browse/HDFS-17076
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17075) Reconfig disk balancer parameters for datanode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17075:
-

 Summary: Reconfig disk balancer parameters for datanode
 Key: HDFS-17075
 URL: https://issues.apache.org/jira/browse/HDFS-17075
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17073) Enhance the warning message output for BlockGroupNonStripedChecksumComputer#compute

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17073:
-

 Summary: Enhance the warning message output for 
BlockGroupNonStripedChecksumComputer#compute
 Key: HDFS-17073
 URL: https://issues.apache.org/jira/browse/HDFS-17073
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17064) Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17064:
-

 Summary: Document the usage of the new Balancer "sortTopNodes" and 
"hotBlockTimeInterval" parameter
 Key: HDFS-17064
 URL: https://issues.apache.org/jira/browse/HDFS-17064
 Project: Hadoop HDFS
  Issue Type: Improvement
        Reporter: Haiyang Hu
        Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17057) Add DataNode maintenance states to Federation UI

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17057:
-

 Summary: Add DataNode maintenance states to Federation UI 
 Key: HDFS-17057
 URL: https://issues.apache.org/jira/browse/HDFS-17057
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu


Add DataNode maintenance states to Federation UI 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17044) Process reported block toInvalidate logic should set the block size to NO_ACK

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17044:
-

 Summary: Process reported block toInvalidate logic should set the 
block size to NO_ACK
 Key: HDFS-17044
 URL: https://issues.apache.org/jira/browse/HDFS-17044
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17019) Optimize the logic for reconfigure slow peer enable for Namenode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17019:
-

 Summary:  Optimize the logic for reconfigure slow peer enable for 
Namenode
 Key: HDFS-17019
 URL: https://issues.apache.org/jira/browse/HDFS-17019
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-17017) Fix the issue of parameter length limit in report command in DFSAdmin.

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-17017:
-

 Summary: Fix the issue of parameter length limit in report command 
in DFSAdmin.
 Key: HDFS-17017
 URL: https://issues.apache.org/jira/browse/HDFS-17017
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16927) Command rollingUpgrade query need to get current rolling upgrade status of each namenode in the nameservice

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16927:
-

 Summary: Command rollingUpgrade query need to get current rolling 
upgrade status of each namenode in the nameservice
 Key: HDFS-16927
 URL: https://issues.apache.org/jira/browse/HDFS-16927
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16926) Fix Typo of ImageServlet#Put()

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16926:
-

 Summary: Fix Typo of ImageServlet#Put()
 Key: HDFS-16926
 URL: https://issues.apache.org/jira/browse/HDFS-16926
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16910) Fix incorrectly initializing RandomAccessFile caused PRC performance decreased for JN

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16910:
-

 Summary: Fix incorrectly initializing RandomAccessFile caused PRC 
performance decreased for JN
 Key: HDFS-16910
 URL: https://issues.apache.org/jira/browse/HDFS-16910
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16892) Fix method name of RPC.Builder#setnumReaders

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16892:
-

 Summary: Fix method name  of RPC.Builder#setnumReaders
 Key: HDFS-16892
 URL: https://issues.apache.org/jira/browse/HDFS-16892
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16888) BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16888:
-

 Summary: BlockManager#maxReplicationStreams, 
replicationStreamsHardLimit, blocksReplWorkMultiplier and 
PendingReconstructionBlocks#timeout should be volatile
 Key: HDFS-16888
 URL: https://issues.apache.org/jira/browse/HDFS-16888
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu


BlockManager#maxReplicationStreams, replicationStreamsHardLimit, 
blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout these 
variables may be  writen by NameNode#reconfReplicationParameters then while 
read by the other threads. 
Thus they should be declared as volatile to make sure the "happens-before" 
consistency.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16885) Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16885:
-

 Summary: Fix 
TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed
 Key: HDFS-16885
 URL: https://issues.apache.org/jira/browse/HDFS-16885
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16884) Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock failed

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16884:
-

 Summary: Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock 
failed
 Key: HDFS-16884
 URL: https://issues.apache.org/jira/browse/HDFS-16884
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16879) EC : Fsck -blockId shows number of redundant internal block replicas for EC files

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16879:
-

 Summary: EC : Fsck -blockId shows number of redundant internal 
block replicas for EC files
 Key: HDFS-16879
 URL: https://issues.apache.org/jira/browse/HDFS-16879
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16841) Enhance the function of DebugAdmin#VerifyECCommand

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16841:
-

 Summary: Enhance the function of DebugAdmin#VerifyECCommand
 Key: HDFS-16841
 URL: https://issues.apache.org/jira/browse/HDFS-16841
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16840) Add sp and ec policy opiton description about oiv in HDFSCommands.md

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16840:
-

 Summary: Add sp and ec policy opiton description about oiv  in 
HDFSCommands.md
 Key: HDFS-16840
 URL: https://issues.apache.org/jira/browse/HDFS-16840
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-16810) Support to make dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock reconfigurable

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-16810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-16810.
---
Resolution: Won't Do

> Support to make 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock 
> reconfigurable 
> -
>
> Key: HDFS-16810
> URL: https://issues.apache.org/jira/browse/HDFS-16810
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
>
> When the Backoff monitor is enabled, the parameter 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock can be 
> dynamically adjusted to determines release the namenode write lock after the 
> numbe of blocks to process when moving blocks to pendingReplication .



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Reopened] (HDFS-16810) Support to make dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock reconfigurable

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-16810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu reopened HDFS-16810:
---

> Support to make 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock 
> reconfigurable 
> -
>
> Key: HDFS-16810
> URL: https://issues.apache.org/jira/browse/HDFS-16810
> Project: Hadoop HDFS
>  Issue Type: Improvement
>    Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
>
> When the Backoff monitor is enabled, the parameter 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock can be 
> dynamically adjusted to determines release the namenode write lock after the 
> numbe of blocks to process when moving blocks to pendingReplication .



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-16810) Support to make dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock reconfigurable

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-16810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-16810.
---
Resolution: Done

> Support to make 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock 
> reconfigurable 
> -
>
> Key: HDFS-16810
> URL: https://issues.apache.org/jira/browse/HDFS-16810
> Project: Hadoop HDFS
>  Issue Type: Improvement
>    Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
>
> When the Backoff monitor is enabled, the parameter 
> dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock can be 
> dynamically adjusted to determines release the namenode write lock after the 
> numbe of blocks to process when moving blocks to pendingReplication .



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16819) Remove the redundant write lock in FsDatasetImpl#createTemporary

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16819:
-

 Summary:  Remove the redundant write lock in 
FsDatasetImpl#createTemporary 
 Key: HDFS-16819
 URL: https://issues.apache.org/jira/browse/HDFS-16819
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16817) Remove useless DataNode lock related configuration

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16817:
-

 Summary: Remove useless DataNode lock related configuration
 Key: HDFS-16817
 URL: https://issues.apache.org/jira/browse/HDFS-16817
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16813) Remove parameter validation logic such as dfs.namenode.decommission.blocks.per.interval in DatanodeAdminManager#activate

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16813:
-

 Summary: Remove parameter validation logic such as 
dfs.namenode.decommission.blocks.per.interval in DatanodeAdminManager#activate
 Key: HDFS-16813
 URL: https://issues.apache.org/jira/browse/HDFS-16813
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16812) Support refresh the decommission manager at runtime

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16812:
-

 Summary: Support refresh the decommission manager at runtime
 Key: HDFS-16812
 URL: https://issues.apache.org/jira/browse/HDFS-16812
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu


Now if we want to switch 'dfs.namenode.decommission.monitor.class' we need to 
restart the NameNode. It would be convenient if we can switch it at runtime. 
Can dynamic Adjustment between DatanodeAdminDefaultMonitor and 
DatanodeAdminBackoffMonitor as needed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16811) 5. Support to make dfs.namenode.decommission.backoff.monitor.pending.limit reconfigurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16811:
-

 Summary: 5. Support to make 
dfs.namenode.decommission.backoff.monitor.pending.limit reconfigurable 
 Key: HDFS-16811
 URL: https://issues.apache.org/jira/browse/HDFS-16811
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16810) 4. Support to make dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock reconfigurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16810:
-

 Summary: 4. Support to make 
dfs.namenode.decommission.backoff.monitor.pending.blocks.per.lock 
reconfigurable 
 Key: HDFS-16810
 URL: https://issues.apache.org/jira/browse/HDFS-16810
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16805) Support set the number of RPC Readers according to different RPC Servers in NameNode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16805:
-

 Summary: Support set the number of RPC Readers according to 
different RPC Servers in NameNode
 Key: HDFS-16805
 URL: https://issues.apache.org/jira/browse/HDFS-16805
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-16786) RBF: Create -addNamespace method in dfsrouteradmin

[Haiyang Hu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HDFS-16786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haiyang Hu resolved HDFS-16786.
---
Resolution: Duplicate

> RBF: Create -addNamespace method in dfsrouteradmin
> --
>
> Key: HDFS-16786
> URL: https://issues.apache.org/jira/browse/HDFS-16786
> Project: Hadoop HDFS
>  Issue Type: Improvement
>    Reporter: Haiyang Hu
>        Assignee: Haiyang Hu
>Priority: Major
>
> When add Namespaces to the existing MountTable. 
> there are the following 2 methods：
> 1.hdfs dfsrouteradmin -add
> for example:
> hdfs dfsrouteradmin -add /user/project newNs /user/project
> this method problem is if no '-order' is specified, 
> the order will set the default HASH and overwrite the existing MountTable 
> order value，
> if the previously set order value is another DestinationOrder so may cause 
> problems use of this MountTable.
> 2.hdfs dfsrouteradmin -update
> for example:
> hdfs dfsrouteradmin -update /user/project  ns1,ns2,newNs /user/project -order 
> HASH_ALL
> this method problem is need to get the existing ns lists, and the correct 
> '-order' also needs to be specified,
> if there is problem with the specified parameters, which will also may cause 
> problems use of this MountTable.
> so currently consider add new method,
> for example:
> hdfs dfsrouteradmin -addNamespace /user/project newNs 
> to satisfy add Namespaces to the existing MountTable and facilitate daily 
> maintenance of MountTable



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16786) Create addNamespace method in dfsrouteradmin

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16786:
-

 Summary: Create addNamespace method in dfsrouteradmin
 Key: HDFS-16786
 URL: https://issues.apache.org/jira/browse/HDFS-16786
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16775) Improve BlockPlacementPolicyRackFaultTolerant's chooseOnce

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16775:
-

 Summary: Improve BlockPlacementPolicyRackFaultTolerant's chooseOnce
 Key: HDFS-16775
 URL: https://issues.apache.org/jira/browse/HDFS-16775
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16774) Improve async delete replica on datanode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16774:
-

 Summary: Improve async delete replica on datanode
 Key: HDFS-16774
 URL: https://issues.apache.org/jira/browse/HDFS-16774
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16314) Support to makedfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16314:
-

 Summary: Support to 
makedfs.namenode.block-placement-policy.exclude-slow-nodes.enabled 
reconfigurable
 Key: HDFS-16314
 URL: https://issues.apache.org/jira/browse/HDFS-16314
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16306) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16306:
-

 Summary: Stale record should be remove when 
DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson
 Key: HDFS-16306
 URL: https://issues.apache.org/jira/browse/HDFS-16306
 Project: Hadoop HDFS
  Issue Type: Bug
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16287:
-

 Summary: Support to make dfs.namenode.avoid.read.slow.datanode and 
dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable
 Key: HDFS-16287
 URL: https://issues.apache.org/jira/browse/HDFS-16287
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16247) RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16247:
-

 Summary: RBF: Fix the ProcessingAvgTime and ProxyAvgTime code 
comments and document metrics describe ms unit
 Key: HDFS-16247
 URL: https://issues.apache.org/jira/browse/HDFS-16247
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-16231:
-

 Summary: Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
 Key: HDFS-16231
 URL: https://issues.apache.org/jira/browse/HDFS-16231
 Project: Hadoop HDFS
  Issue Type: Task
Reporter: Haiyang Hu
Assignee: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-15998) Fix NullPointException In listOpenFiles

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-15998:
-

 Summary: Fix NullPointException In listOpenFiles
 Key: HDFS-15998
 URL: https://issues.apache.org/jira/browse/HDFS-15998
 Project: Hadoop HDFS
  Issue Type: Bug
Affects Versions: 3.2.0
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-15889) Erasure Coding: Limit the number of concurrent EC Reconstruct blocks in NameNode

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-15889:
-

 Summary: Erasure Coding: Limit the number of concurrent EC 
Reconstruct blocks in NameNode
 Key: HDFS-15889
 URL: https://issues.apache.org/jira/browse/HDFS-15889
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

[Haiyang Hu (Jira)]

Haiyang Hu created HDFS-15877:
-

 Summary: BlockReconstructionWork should resetTargets() before 
BlockManager#validateReconstructionWork return false
 Key: HDFS-15877
 URL: https://issues.apache.org/jira/browse/HDFS-15877
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Haiyang Hu






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org