[
https://issues.apache.org/jira/browse/HDFS-15382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123379#comment-17123379
]
Aiphago commented on HDFS-15382:
After improve our du in cache copy time is very low.And we make improve just
copy replica in each BlockPoolSlice.
{code:java}
2020-06-02 12:44:16,586 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, blockPoolId: BP-xxx, replicas size: 665, duration: 0ms
2020-06-02 12:44:16,586 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Refresh dfs used, bpid: BP-xxx,replicas size: 665, dfsUsed: 15925882188 on
volume: DS-4f1f820a-460f-4fa9-89be-49caed604a52, duration: 0ms , isopen
hardlink false
2020-06-02 12:44:16,586 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, blockPoo
lId: BP-xxx, replicas size: 699, duration: 1ms
2020-06-02 12:44:16,586 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, blockPoo
lId: BP-xxx, replicas size: 698, duration: 1ms
2020-06-02 12:44:16,587 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, blockPoo
lId: BP-xxx, replicas size: 638, duration: 0ms
2020-06-02 12:44:16,587 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Refresh dfs used, bpid: BP-xxx, replicas size: 638, dfsUsed: 16519661992 on
volume: DS-b2eb6423-d0bd-493e-a102-d317e55815ce, duration: 0ms , isopen
hardlink false
2020-06-02 12:44:16,588 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, blockPoo
lId: BP-xxx, replicas size: 644, duration: 0ms
2020-06-02 12:44:16,588 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Refresh dfs used, bpid: BP-xxx, replicas size: 644, dfsUsed: 16636348641 on
volume: DS-83a2deeb-2389-4036-9f13-df61fc6b35f6, duration: 0ms , isopen
hardlink false
2020-06-02 12:44:16,588 INFO
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReplicaCachingGetSpaceUsed:
Copy replica infos, BP-xxx,
replicas size: 663, duration: 0ms{code}
> Split DataNode FsDatasetImpl lock to blockpool volume lock
> ---
>
> Key: HDFS-15382
> URL: https://issues.apache.org/jira/browse/HDFS-15382
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Aiphago
>Assignee: Aiphago
>Priority: Major
> Fix For: 3.2.1
>
> Attachments: image-2020-06-02-1.png
>
>
> In HDFS-15180 we split lock to blockpool grain size.But when one volume is in
> heavy load and will block other request which in same blockpool but different
> volume.So we split lock to two leval to avoid this happend.And to improve
> datanode performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
