[ https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kihwal Lee updated HDFS-8873: ----------------------------- Labels: 2.7.2-candidate (was: ) > Allow the directoryScanner to be rate-limited > --------------------------------------------- > > Key: HDFS-8873 > URL: https://issues.apache.org/jira/browse/HDFS-8873 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode > Affects Versions: 2.7.1 > Reporter: Nathan Roberts > Assignee: Daniel Templeton > Labels: 2.7.2-candidate > Fix For: 2.8.0 > > Attachments: HDFS-8873.001.patch, HDFS-8873.002.patch, > HDFS-8873.003.patch, HDFS-8873.004.patch, HDFS-8873.005.patch, > HDFS-8873.006.patch, HDFS-8873.007.patch, HDFS-8873.008.patch, > HDFS-8873.009.patch > > > The new 2-level directory layout can make directory scans expensive in terms > of disk seeks (see HDFS-8791) for details. > It would be good if the directoryScanner() had a configurable duty cycle that > would reduce its impact on disk performance (much like the approach in > HDFS-8617). > Without such a throttle, disks can go 100% busy for many minutes at a time > (assuming the common case of all inodes in cache but no directory blocks > cached, 64K seeks are required for full directory listing which translates to > 655 seconds) -- This message was sent by Atlassian JIRA (v6.3.4#6332)