[
https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813619#comment-16813619
]
Daryn Sharp commented on HDFS-14419:
------------------------------------
Seems like a more straightforward and durable approach may be to use an
iterator so hasMore = iter.hasNext().
> Avoid repeated calls to the listOpenFiles function
> --------------------------------------------------
>
> Key: HDFS-14419
> URL: https://issues.apache.org/jira/browse/HDFS-14419
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, performance
> Affects Versions: 3.3.0
> Reporter: HuangTao
> Assignee: HuangTao
> Priority: Minor
> Fix For: 3.3.0
>
> Attachments: HDFS-14419.001.patch
>
>
> `hdfs dfsadmin -listOpenFiles -path /any/path` will request all opened files.
> In the NameNode side, the function
> LeaseManager.java#getUnderConstructionFiles will be called.
> When there are only N(<maxListOpenFilesResponses, default is 100) files meet
> the conditions, but the leaseManager contains M(>maxListOpenFilesResponses)
> files, we will scan all leases. Finally, the hasMore will be set true and the
> openFileEntries contains N files, the scenario will cause listOpenFiles() be
> called again.
> If M is greater more than N, the two calls to getUnderConstructionFiles will
> impact the NameNode performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
