[
https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039209#comment-17039209
]
Stephen O'Donnell commented on HDFS-15180:
------------------------------------------
We have some work on going related to HDFS-15150 to move the DN lock to a
ReentrantReadWrite lock. There is further work to do to move various code paths
to use the read lock to reduce contention. HDFS-15160 is a start on this, but
there will need to be more Jiras.
These changes are relatively simple and safe. I think there could be further
gains made with finer grained locking, but those changes could be difficult to
prove there are no race conditions.
I would suggest we try to complete moving various code paths to use the new
read lock and remove any places where the lock is held during disk IO, and then
see how things look. That may give good improvements, but its hard to say until
we do some benchmarks and see some real world use. Then we can consider finer
grained locking.
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
> -----------------------------------------------------------
>
> Key: HDFS-15180
> URL: https://issues.apache.org/jira/browse/HDFS-15180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 3.2.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
>
> Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in
> big cluster. If we can split the FsDatasetImpl datasetLock via blockpool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
