[
https://issues.apache.org/jira/browse/HDFS-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148728#comment-17148728
]
Hudson commented on HDFS-15160:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18393 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18393/])
HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various (weichiu:
rev 2a67e2b1a0e3a5f91056f5b977ef9c4c07ba6718)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/ReplicaMap.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
> ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl
> methods should use datanode readlock
> -----------------------------------------------------------------------------------------------------------
>
> Key: HDFS-15160
> URL: https://issues.apache.org/jira/browse/HDFS-15160
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.3.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Fix For: 3.4.0
>
> Attachments: HDFS-15160.001.patch, HDFS-15160.002.patch,
> HDFS-15160.003.patch, HDFS-15160.004.patch, HDFS-15160.005.patch,
> HDFS-15160.006.patch, HDFS-15160.007.patch, HDFS-15160.008.patch,
> image-2020-04-10-17-18-08-128.png, image-2020-04-10-17-18-55-938.png
>
>
> Now we have HDFS-15150, we can start to move some DN operations to use the
> read lock rather than the write lock to improve concurrence. The first step
> is to make the changes to ReplicaMap, as many other methods make calls to it.
> This Jira switches read operations against the volume map to use the readLock
> rather than the write lock.
> Additionally, some methods make a call to replicaMap.replicas() (eg
> getBlockReports, getFinalizedBlocks, deepCopyReplica) and only use the result
> in a read only fashion, so they can also be switched to using a readLock.
> Next is the directory scanner and disk balancer, which only require a read
> lock.
> Finally (for this Jira) are various "low hanging fruit" items in BlockSender
> and fsdatasetImpl where is it fairly obvious they only need a read lock.
> For now, I have avoided changing anything which looks too risky, as I think
> its better to do any larger refactoring or risky changes each in their own
> Jira.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
