[
https://issues.apache.org/jira/browse/HDFS-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14805021#comment-14805021
]
Yi Liu commented on HDFS-8873:
------------------------------
{quote}
We've seen this before and never managed to track it down. It seems to be a bug
in our Jenkins integration, possibly related to having multiple maven
invocations going on at once sharing the same .m2 directory. I will re-trigger
the build.
{quote}
Yes, I think so, I see similar issue several times. If I have a HDFS patch, and
also add new class in Hadoop common, then it easily fails because other builds
may overwrite the hadoop common jar of my build installed. Unless we use
different virtual machine, maybe through docker container to solve this
problem? Maybe need support from Infra?
> throttle directoryScanner
> -------------------------
>
> Key: HDFS-8873
> URL: https://issues.apache.org/jira/browse/HDFS-8873
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.7.1
> Reporter: Nathan Roberts
> Assignee: Daniel Templeton
> Attachments: HDFS-8873.001.patch, HDFS-8873.002.patch,
> HDFS-8873.003.patch
>
>
> The new 2-level directory layout can make directory scans expensive in terms
> of disk seeks (see HDFS-8791) for details.
> It would be good if the directoryScanner() had a configurable duty cycle that
> would reduce its impact on disk performance (much like the approach in
> HDFS-8617).
> Without such a throttle, disks can go 100% busy for many minutes at a time
> (assuming the common case of all inodes in cache but no directory blocks
> cached, 64K seeks are required for full directory listing which translates to
> 655 seconds)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
