[
https://issues.apache.org/jira/browse/HDFS-15000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000742#comment-17000742
]
Aiphago commented on HDFS-15000:
--------------------------------
Hi [~sodonnell] ,Thanks for your comment.I have some question with your idea.
??I wonder if it would be possible to refactor things so we do steps 1, 2, 3
and 5, drop the lock and then do the IO operation to actually create the file.
In the event the IO fails, re-take the lock and clean up the volume map.??
If we do the IO after step 5,and now the volume map have the replica info but
actually the IO may not done.If some error happend with IO,you should get lock
again so volume map will have this replica for a long time,this may cause
consistency problem when other thread get this replica.
and my thought is
# Not change the step in one method.
# Make the IO async and release the lock wait util other thread signal this
thread when finish the IO.
# Keep the IO operation is in order in different methods as #finalizeBlock,
#finalizeReplica, #createRbw.....
# after IO operation change then change the volume map.
> Improve FsDatasetImpl to avoid IO operation in datasetLock
> ----------------------------------------------------------
>
> Key: HDFS-15000
> URL: https://issues.apache.org/jira/browse/HDFS-15000
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Xiaoqiao He
> Assignee: Aiphago
> Priority: Major
>
> As HDFS-14997 mentioned, some methods in #FsDatasetImpl such as
> #finalizeBlock, #finalizeReplica, #createRbw includes IO operation in the
> datasetLock, It will block some logic when IO load is very high. We should
> reduce grain fineness or move IO operation out of datasetLock.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
