[
https://issues.apache.org/jira/browse/HDFS-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo (Nicholas), SZE updated HDFS-2495:
-----------------------------------------
Status: Patch Available (was: Open)
> Increase granularity of write operations in ReplicationMonitor thus reducing
> contention for write lock
> ------------------------------------------------------------------------------------------------------
>
> Key: HDFS-2495
> URL: https://issues.apache.org/jira/browse/HDFS-2495
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Reporter: Tomasz Nykiel
> Assignee: Tomasz Nykiel
> Attachments: replicationMon.patch, replicationMon.patch-1
>
>
> For processing blocks in ReplicationMonitor
> (BlockManager.computeReplicationWork), we first obtain a list of blocks to be
> replicated by calling chooseUnderReplicatedBlocks, and then for each block
> which was found, we call computeReplicationWorkForBlock. The latter processes
> a block in three stages, acquiring the writelock twice per call:
> 1. obtaining block related info (livenodes, srcnode, etc.) under lock
> 2. choosing target for replication
> 3. scheduling replication (under lock)
> We would like to change this behaviour and decrease contention for the write
> lock, by batching blocks and executing 1,2,3, for sets of blocks, rather than
> for each one separately. This would decrease the number of writeLock to 2,
> from 2*numberofblocks.
> Also, the info level logging can be pushed outside the writelock.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
