[
https://issues.apache.org/jira/browse/HDDS-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221206#comment-17221206
]
Lokesh Jain commented on HDDS-4344:
-----------------------------------
HDDS-4297 gives better deletion speed for deleting a milllion keys.
Without PR - SCM sends total 180 blocks for deletion in a minute. This number
does not increase even after increasing configs. Datanode deletion speed was
very slow. I monitored a datanode disk space and it deleted less than a GB even
after 5 minutes.
With PR - SCM sends 6000 blocks for deletion. Datanode was deleting 1 GB of
data every minute.
I used the following configs.
ozone.block.deleting.container.limit.per.interval: "1000"
ozone.key.deleting.limit.per.task: "10000"
> Block Deletion Performance Improvements
> ---------------------------------------
>
> Key: HDDS-4344
> URL: https://issues.apache.org/jira/browse/HDDS-4344
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Attachments: Block Deletion Performance.pdf
>
>
> In cluster deployments it was observed that block deletion can be slow. For
> example if a user writes a million keys in Ozone, the time it takes for those
> million keys to be deleted from datanodes can be high. The jira would cover
> various improvements which can be made for better deletion speeds.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
