[ https://issues.apache.org/jira/browse/HDDS-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822092#comment-16822092 ]
Steve Loughran commented on HDDS-1301: -------------------------------------- All the work in mapreduce for the s3a committer can be applied to Ozone too: you can define a new implementation of PathOutputCommitProtocol, and then declare the factory class for it {{PathOutputCommitterFactory}} , and when an MR client (or suitably configured Spark app) looks for an output committer for a FileOutputFormat, you can serve up your committer instead of the legacy "let's commit by rename" algorithm. Happy to talk about how to do this/point you at the s3a impl. Key point: for your store, no matter what the applications think, rename is optional > Optimize recursive ozone filesystem apis > ---------------------------------------- > > Key: HDDS-1301 > URL: https://issues.apache.org/jira/browse/HDDS-1301 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task > Reporter: Lokesh Jain > Assignee: Lokesh Jain > Priority: Major > Labels: pull-request-available > Attachments: HDDS-1301.001.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > > This Jira aims to optimise recursive apis in ozone file system. These are the > apis which have a recursive flag which requires an operation to be performed > on all the children of the directory. The Jira would add support forĀ > recursive apis in Ozone manager in order to reduce the number of rpc calls to > Ozone Manager. Also currently these operations are not atomic. This Jira > would make all the operations in ozone filesystem atomic. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org