[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562579#comment-16562579 ]
Gabor Bota edited comment on HADOOP-14154 at 7/30/18 10:05 PM: --------------------------------------------------------------- Uploaded patch v001 for this issue. * I had to fix {{MetadataStoreTestBase#testPruneUnsetsAuthoritative}} because it did not test anything. (had to add a dir which is authoritative, but a child element gets removed so the parent dir loses the flag) * I've added {{testPrunePreservesAuthoritative}} to test that the authoritative flag is preserved when no child elements pruned from the dir. * Added {{PathMetadata#isAuthoritativeDir}} because {{DynamoDBMetadataStore}} stores {{PathMetadata}} in ddb, so that was a logical choice. * This is not the final patch I'd like to be committed - the final patch should also include .md docs about this change. Big thanks to [~fabbri] for the continuous help, and for the original idea on how to solve this issue. was (Author: gabor.bota): Uploaded patch v001 for this issue. * I had to fix {{MetadataStoreTestBase#testPruneUnsetsAuthoritative}} because it did not test anything. (had to add a dir which is authoritative, but a child element gets removed so the parent dir loses the flag) * I've added {{testPrunePreservesAuthoritative}} to test that the authoritative flag is preserved when no child elements pruned from the dir. * Added {{PathMetadata#isAuthoritativeDir}} because {{DynamoDBMetadataStore}} stores {{PathMetadata}} in ddb, so that was a logical choice. Big thanks to [~fabbri] for the continuous help, and for the original idea on how to solve this issue. > Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support) > ----------------------------------------------------------------------------- > > Key: HADOOP-14154 > URL: https://issues.apache.org/jira/browse/HADOOP-14154 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.0.0-beta1 > Reporter: Rajesh Balamohan > Assignee: Gabor Bota > Priority: Minor > Attachments: HADOOP-14154-HADOOP-13345.001.patch, > HADOOP-14154-HADOOP-13345.002.patch, HADOOP-14154-spec-001.pdf, > HADOOP-14154-spec-002.pdf, HADOOP-14154.001.patch > > > Add support for "authoritative mode" for DynamoDBMetadataStore. > The missing feature is to persist the bit set in > {{DirListingMetadata.isAuthoritative}}. > This topic has been super confusing for folks so I will also file a > documentation Jira to explain the design better. > We may want to also rename the DirListingMetadata.isAuthoritative field to > .isFullListing to eliminate the multiple uses and meanings of the word > "authoritative". > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org