Weiwei Yang created HDFS-12443: ---------------------------------- Summary: Ozone: Improve SCM block deletion throttling algorithm Key: HDFS-12443 URL: https://issues.apache.org/jira/browse/HDFS-12443 Project: Hadoop HDFS Issue Type: Sub-task Components: ozone, scm Reporter: Weiwei Yang Assignee: Weiwei Yang
Currently SCM scans delLog to send deletion transactions to datanode periodically, the throttling algorithm is simple, it scans at most {{BLOCK_DELETE_TX_PER_REQUEST_LIMIT}} (by default 50) at a time. This is non-optimal, worst case it might cache 50 TXs for 50 different DNs so each DN will only get 1 TX to proceed in an interval, this will make the deletion slow. An improvement to this is to make this throttling by datanode, e.g 50 TXs per datanode per interval. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org