[
https://issues.apache.org/jira/browse/HDFS-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852114#comment-13852114
]
Suresh Srinivas commented on HDFS-5453:
---------------------------------------
I agree with [~shv].
In namenode write operation spends a small time updating the memory (t1) and
much larger time depositing the editlog record onto disk (t2). By making t1
more efficient with fine grained locking, I do not think much performance will
be gained or CPU cores can be kept busy, since the processing time is dominated
by t2.
We could do benchmark with say namenode synchronization turned off (multiple
threads are doing operations on diff parts of namespace) and editlog writing to
/dev/null. My guess would be, the second one will do much better. If that is
the case, we could try optimizing the editlog records using inode ID instead of
paths etc.
> Support fine grain locking in FSNamesystem
> ------------------------------------------
>
> Key: HDFS-5453
> URL: https://issues.apache.org/jira/browse/HDFS-5453
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The namesystem currently uses a course grain lock to control access. This
> prevents concurrent writers in different branches of the tree, and prevents
> readers from accessing branches that writers aren't using.
> Features that introduce latency to namesystem operations, such as cold
> storage of inodes, will need fine grain locking to avoid degrading the entire
> namesystem's throughput.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
