[ https://issues.apache.org/jira/browse/HDFS-16316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17781202#comment-17781202 ]
姚凡 edited comment on HDFS-16316 at 10/31/23 3:36 AM: ----------------------------------------------------- public static boolean isRegularFile(File file, boolean allowLinks) { if (file != null) { if (allowLinks) { return Files.isRegularFile(file.toPath()); } return Files.isRegularFile(file.toPath(), LinkOption.NOFOLLOW_LINKS); } return true; } When a meta file that does not exist is transferred, the blk is incorrectly set to corrupt. You need to check whether the file exists. was (Author: yaofan): !image-2023-10-31-11-24-39-387.png! When a meta file that does not exist is transferred, the blk is incorrectly set to corrupt. You need to check whether the file exists. > Improve DirectoryScanner: add regular file check related block > -------------------------------------------------------------- > > Key: HDFS-16316 > URL: https://issues.apache.org/jira/browse/HDFS-16316 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode > Affects Versions: 2.9.2 > Reporter: JiangHua Zhu > Assignee: JiangHua Zhu > Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: screenshot-1.png, screenshot-2.png, screenshot-3.png, > screenshot-4.png > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Something unusual happened in the online environment. > The DataNode is configured with 11 disks (${dfs.datanode.data.dir}). It is > normal for 10 disks to calculate the used capacity, and the calculated value > for the other 1 disk is much larger, which is very strange. > This is about the live view on the NameNode: > !screenshot-1.png! > This is about the live view on the DataNode: > !screenshot-2.png! > We can look at the view on linux: > !screenshot-3.png! > There is a big gap here, regarding'/mnt/dfs/11/data'. This situation should > be prohibited from happening. > I found that there are some abnormal block files. > There are wrong blk_xxxx.meta in some subdir directories, causing abnormal > computing space. > Here are some abnormal block files: > !screenshot-4.png! > Such files should not be used as normal blocks. They should be actively > identified and filtered, which is good for cluster stability. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org