[ https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204434#comment-17204434 ]
Xiaoqiao He commented on HDFS-15382: ------------------------------------ Thanks [~sodonnell],[~Jiang Xin] for your comments. HDFS-15150 and HDFS-15160 is very interesting improvement for DataNode, and I think the result is also impressive. But this solution does not solve coupling issue between different BlockPools and different Volumes when enable Federation feature. Especially one of BlockPool/Volume's load is very high, other BlockPools/Volumes read/write operation will be blocked since still some IO operation in Lock which could hold for long time, such as #updateReplicaUnderRecovery. In our inner branch, this issue is very critical. Please reference https://drive.google.com/file/d/1eaE8vSEhIli0H3j2eDiPJNYuKAC0MFgu/view?usp=sharing if interesting details. IMO, the key of this improvement is decoupling BlockPools and Volumes and try to improve performance further. with HDFS-15150 and HDFS-15160, it will get a better result. About the demo patch, if agreement, we will split to some subtask to push this feature forwards. cc [~Aiphag0] Thanks [~sodonnell] and [~LiJinglun] again. Welcome more discussion and suggestions. > Split FsDatasetImpl from blockpool lock to blockpool volume lock > ----------------------------------------------------------------- > > Key: HDFS-15382 > URL: https://issues.apache.org/jira/browse/HDFS-15382 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Aiphago > Assignee: Aiphago > Priority: Major > Attachments: HDFS-15382-sample.patch, image-2020-06-02-1.png, > image-2020-06-03-1.png > > > In HDFS-15180 we split lock to blockpool grain size.But when one volume is in > heavy load and will block other request which in same blockpool but different > volume.So we split lock to two leval to avoid this happend.And to improve > datanode performance. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org