[
https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876099#comment-14876099
]
Colin Patrick McCabe commented on HDFS-8873:
--------------------------------------------
bq. Question about the rest of your comments: Except for the last one, the rest
of your comments apply to code that I touched only by indenting it. I didn't
modify any of that code that I didn't have to, in the name of simplifying
review by diff. Sounds like the going rule is, "you touch it, you own it," even
if it's just a whitespace change. Am I reading that correctly?
That's a fair point. If you want to leave the d and m variables as-is to
simplify the diff and backporting, that's fine. We should fix them at some
point, but it doesn't have to be right now.
> throttle directoryScanner
> -------------------------
>
> Key: HDFS-8873
> URL: https://issues.apache.org/jira/browse/HDFS-8873
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Nathan Roberts
> Assignee: Daniel Templeton
> Attachments: HDFS-8873.001.patch, HDFS-8873.002.patch,
> HDFS-8873.003.patch
>
>
> The new 2-level directory layout can make directory scans expensive in terms
> of disk seeks (see HDFS-8791) for details.
> It would be good if the directoryScanner() had a configurable duty cycle that
> would reduce its impact on disk performance (much like the approach in
> HDFS-8617).
> Without such a throttle, disks can go 100% busy for many minutes at a time
> (assuming the common case of all inodes in cache but no directory blocks
> cached, 64K seeks are required for full directory listing which translates to
> 655 seconds)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
