captainzmc opened a new pull request #814: HDDS-3286. BasicOzoneFileSystem support batchDelete and batchRename. URL: https://github.com/apache/hadoop-ozone/pull/814 ## What changes were proposed in this pull request? Currently delete file is to get all the keys in the directory, and then delete one by one. And the same with rename. This makes for poor performance. By tested the deletion directory with 100,000 files, which took 3718.70 sec. And rename it took 7327.936 sec. Using this PR, when batch-size is set to 100, the time of delete and rename's directory of 100,000 files is 62.498 sec and 46.002 sec. Performance improved nearly 100 times。 https://issues.apache.org/jira/browse/HDDS-3286 ## How was this patch tested? Set the conf in your code or ozone-site.xml, The default value of ozone.fs.iterate.batch-size is 1. Configuration conf = new Configuration(); conf.setInt("ozone.fs.iterate.batch-size",100);
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
