[ https://issues.apache.org/jira/browse/HADOOP-16380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886293#comment-16886293 ]
Gabor Bota commented on HADOOP-16380: ------------------------------------- The following should be fixed to wrap the findings up fully: * HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface * HADOOP-16433. Filter expired entries and tombstones when listing with MetadataStore#listChildren > S3A non-recursive deletion of root to return false > -------------------------------------------------- > > Key: HADOOP-16380 > URL: https://issues.apache.org/jira/browse/HADOOP-16380 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test > Affects Versions: 3.2.0, 3.0.3, 3.3.0, 3.1.2 > Reporter: Steve Loughran > Assignee: Gabor Bota > Priority: Major > > If S3AFileSystem does an S3 LIST restricted to a single object to see if a > directory is empty, and the single entry found has a tombstone marker (either > from an inconsistent DDB Table or from an eventually consistent LIST) then it > will consider the directory empty, _even if there is 1+ entry which is not > deleted_ > We need to make sure the calculation of whether a directory is empty or not > is resilient to this, efficiently. > It surfaces as an issue two places > * delete(path) (where it may make things worse) > * rename(src, dest), where a check is made for dest != an empty directory. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org