[
https://issues.apache.org/jira/browse/HDFS-5453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822613#comment-13822613
]
Daryn Sharp commented on HDFS-5453:
-----------------------------------
I've got an implementation of fine grain fsn locking that can be phased in with
no compatibility issue. The changes to fsn are absolutely minimal. The use of
course (existing) or fine grain locking can be controlled via a conf. If we
clearly mark the conf as a non-public experimental feature, then no risk will
be introduced.
However... Although readers and writers can enter disjoint branches of the
namespace, the fsdir lock reserializes batches of readers with writers. The
fsdir lock is being used to protect various non-thread safe subsystems and data
structures. Changes to eliminate this bottleneck may require a feature branch
if addressing the bottlenecks proves too risky. I've been juggling too many
tasks to gauge this complexity.
> Support fine grain locking in FSNamesystem
> ------------------------------------------
>
> Key: HDFS-5453
> URL: https://issues.apache.org/jira/browse/HDFS-5453
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: namenode
> Affects Versions: 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
>
> The namesystem currently uses a course grain lock to control access. This
> prevents concurrent writers in different branches of the tree, and prevents
> readers from accessing branches that writers aren't using.
> Features that introduce latency to namesystem operations, such as cold
> storage of inodes, will need fine grain locking to avoid degrading the entire
> namesystem's throughput.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
