[
https://issues.apache.org/jira/browse/HDFS-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851274#comment-13851274
]
Konstantin Shvachko commented on HDFS-5453:
-------------------------------------------
Daryn, just wanted to remind an issue HADOOP-3860, where we decided that
fine-grained locking is not worth pursuing some years ago. [See summary
here|https://issues.apache.org/jira/browse/HADOOP-3860?focusedCommentId=12618038&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12618038]
There are three reasons:
# Performance gains are minimal. ~2-15% is almost within the measurement
threshold, and write operations (those using the write lock) are IO bound, not
cpu, because of journalling.
# Increased code complexity: more locks means more errors.
# The actual NameNode load on a busy cluster never reaches its maximum
execution power. The load used to be two orders of magnitude lower than NN can
handle.
Things changed since then in performance numbers and use cases, but the
arguments may still be valid.
Checking peak and average load on one of your clusters could answer some of
such questions.
> Support fine grain locking in FSNamesystem
> ------------------------------------------
>
> Key: HDFS-5453
> URL: https://issues.apache.org/jira/browse/HDFS-5453
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The namesystem currently uses a course grain lock to control access. This
> prevents concurrent writers in different branches of the tree, and prevents
> readers from accessing branches that writers aren't using.
> Features that introduce latency to namesystem operations, such as cold
> storage of inodes, will need fine grain locking to avoid degrading the entire
> namesystem's throughput.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
