[
https://issues.apache.org/jira/browse/HDFS-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633319#comment-13633319
]
Tsz Wo (Nicholas), SZE commented on HDFS-4702:
----------------------------------------------
Hi Chris,
I agree that we could remove namesystem lock from
DatanodeManager#fetchDatanodes if we make sure that datanodeMap is always
synchronized. Right now, refreshNodes only acquires namesystem.writeLock but
not datanodeMap lock. There are also some other places such as getDatanode,
getDatanodeCyclicIteration and newStorageID read datanodeMap without
datanodeMap lock. We may need to fix them as well.
Sometime ago, there is no DatanodeManager and everything is in FSNamesystem.
That's why it uses namesystem lock. After some refactoring, I believe we can
change DatanodeManager so that it has its only read-write lock to synchronize
all datanode related operations. See HDFS-2206.
> remove namesystem lock from DatanodeManager#fetchDatanodes
> ----------------------------------------------------------
>
> Key: HDFS-4702
> URL: https://issues.apache.org/jira/browse/HDFS-4702
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
>
> {{DatanodeManager#fetchDatanodes}} currently holds the namesystem read lock
> while iterating through data nodes. This method is called from the namenode
> web UI. HDFS-3990 reported a performance problem in this code path. This is
> a follow-up jira to investigate whether or not we can remove the lock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
