[
https://issues.apache.org/jira/browse/HDFS-16059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17436427#comment-17436427
]
Ayush Saxena commented on HDFS-16059:
-------------------------------------
Thanx [~aajisaka] for the report. Was just casually trying this out. Can we not
leverage the code for listing all open files and just have a filter for files
blocking decommission, rather than trying out each datanode? I couldn't think
of any reason where it would cause a problem or why it wasn't done that way
initially.
I have uploaded a WIP patch with the said approach. Tried out running the
related tests, they passed for me. But couldn't repro the exact scenario,
reproducing that seems tough, post we change the approach, so relied on the
existing tests :)
> dfsadmin -listOpenFiles -blockingDecommission can miss some files
> -----------------------------------------------------------------
>
> Key: HDFS-16059
> URL: https://issues.apache.org/jira/browse/HDFS-16059
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: dfsadmin
> Reporter: Akira Ajisaka
> Priority: Major
> Attachments: HDFS-16059-WIP-01.patch, HDFS-16059-regression-test.patch
>
>
> While reviewing HDFS-13671, I found "dfsadmin -listOpenFiles
> -blockingDecommission" can drop some files.
> [https://github.com/apache/hadoop/pull/3065#discussion_r647396463]
> {quote}If the DataNodes have the following open files and we want to list all
> the open files:
> DN1: [1001, 1002, 1003, ... , 2000]
> DN2: [1, 2, 3, ... , 1000]
> At first getFilesBlockingDecom(0, "/") is called and it returns [1001, 1002,
> ... , 2000] because it reached max size (=1000), and next
> getFilesBlockingDecom(2000, "/") is called because the last inode Id of the
> previous result is 2000. That way the open files of DN2 is missed
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
