[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15851645#comment-15851645 ]
Steve Loughran commented on HADOOP-13936: ----------------------------------------- Is a TTL going to need an extra column on the DB? That's going to be a non-traumatic change as far as ddb is concerned, if the client code sets some default timestamp on entries without them. > S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation > ------------------------------------------------------------------------- > > Key: HADOOP-13936 > URL: https://issues.apache.org/jira/browse/HADOOP-13936 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Rajesh Balamohan > Priority: Minor > > As a part of {{S3AFileSystem.delete}} operation {{innerDelete}} is invoked, > which deletes keys from S3 in batches (default is 1000). But DynamoDB is > updated only at the end of this operation. This can cause issues when > deleting large number of keys. > E.g, it is possible to get exception after deleting 1000 keys and in such > cases dynamoDB would not be updated. This can cause DynamoDB to go out of > sync. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org