[ https://issues.apache.org/jira/browse/HDFS-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13147231#comment-13147231 ]
Tsz Wo (Nicholas), SZE commented on HDFS-2495: ---------------------------------------------- Tom and Hairong, thanks for the great work! > Increase granularity of write operations in ReplicationMonitor thus reducing > contention for write lock > ------------------------------------------------------------------------------------------------------ > > Key: HDFS-2495 > URL: https://issues.apache.org/jira/browse/HDFS-2495 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: name-node > Reporter: Tomasz Nykiel > Assignee: Tomasz Nykiel > Fix For: 0.24.0 > > Attachments: replicationMon.patch, replicationMon.patch-1 > > > For processing blocks in ReplicationMonitor > (BlockManager.computeReplicationWork), we first obtain a list of blocks to be > replicated by calling chooseUnderReplicatedBlocks, and then for each block > which was found, we call computeReplicationWorkForBlock. The latter processes > a block in three stages, acquiring the writelock twice per call: > 1. obtaining block related info (livenodes, srcnode, etc.) under lock > 2. choosing target for replication > 3. scheduling replication (under lock) > We would like to change this behaviour and decrease contention for the write > lock, by batching blocks and executing 1,2,3, for sets of blocks, rather than > for each one separately. This would decrease the number of writeLock to 2, > from 2*numberofblocks. > Also, the info level logging can be pushed outside the writelock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira