[
https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051952#comment-17051952
]
Xiaoqiao He commented on HDFS-15180:
------------------------------------
[~zhuqi] Thanks for your proposal and involve me here.
It is very valuable suggestion, actually I [~sodonnell] have done some work,
ref. HDFS-15150 introduce read write lock and HDFS-15160 is in progress
currently. Of course, HDFS-14997 (please with HDFS-15113 together if backport)
is another way to avoid heavy IO to impact interactive with NN.
Beside these works, I believe there are some other ways to split the global
lock. My colleague [~Aiphag0] try to use {{BlockPoolLockManager}} to split
{{dataLock}} more fine-grained. {{BlockPoolLockManager}} represents rwlock pool
with many rwlocks and it is more convenient for different BlockPools and
different Disks to acquire lock and improve parallel read and write. This work
is nearly finished recently, and gray deploy in our produce cluster. HDFS-15000
will trace this work.
Thanks [~zhuqi] again.
> DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.
> -----------------------------------------------------------
>
> Key: HDFS-15180
> URL: https://issues.apache.org/jira/browse/HDFS-15180
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.2.0
> Reporter: zhuqi
> Assignee: zhuqi
> Priority: Major
>
> Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in
> big cluster. If we can split the FsDatasetImpl datasetLock via blockpool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
