[ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Mackrory updated HADOOP-13760: ----------------------------------- Attachment: HADOOP-13760-HADOOP-13345.003.patch This patch has 2 tests failing without S3Guard that I still need to look into. Just posting as a sort of checkpoint before the weekend. I've incorporated all of Fabbri's feedback, and rebased on top of Mingliang's change to add S3Guard support to listFiles\*. In that change, the behavior changed to start including intermediate directories, and that did indeed cause a lot of tests to fail (copyFile failed when attempting to move an empty blob that didn't exist), so I've implemented a new iterator that only returns leaf nodes, since that is best for all current uses of listFilesAndDirectories. I also renamed it to listFilesAndEmptyDirectories to reflect the intention. > S3Guard: add delete tracking > ---------------------------- > > Key: HADOOP-13760 > URL: https://issues.apache.org/jira/browse/HADOOP-13760 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Aaron Fabbri > Assignee: Sean Mackrory > Attachments: HADOOP-13760-HADOOP-13345.001.patch, > HADOOP-13760-HADOOP-13345.002.patch, HADOOP-13760-HADOOP-13345.003.patch > > > Following the S3AFileSystem integration patch in HADOOP-13651, we need to add > delete tracking. > Current behavior on delete is to remove the metadata from the MetadataStore. > To make deletes consistent, we need to add a {{isDeleted}} flag to > {{PathMetadata}} and check it when returning results from functions like > {{getFileStatus()}} and {{listStatus()}}. In HADOOP-13651, I added TODO > comments in most of the places these new conditions are needed. The work > does not look too bad. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org