[jira] [Comment Edited] (HADOOP-13756) LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash.
[ https://issues.apache.org/jira/browse/HADOOP-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456485#comment-16456485 ] Gabor Bota edited comment on HADOOP-13756 at 4/27/18 2:01 PM: -- Thanks [~fabbri]. I've created https://issues.apache.org/jira/browse/HADOOP-15423 for the Path -> tuple(DirListingMetadata, PathMetadata) change. was (Author: gabor.bota): Thanks [~fabbri]. I've issed https://issues.apache.org/jira/browse/HADOOP-15423 for the Path -> tuple(DirListingMetadata, PathMetadata) change. > LocalMetadataStore#put(DirListingMetadata) should also put file metadata into > fileHash. > --- > > Key: HADOOP-13756 > URL: https://issues.apache.org/jira/browse/HADOOP-13756 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.0.0-beta1 >Reporter: Lei (Eddy) Xu >Assignee: Gabor Bota >Priority: Major > Fix For: 3.2.0 > > Attachments: HADOOP-13756.001.patch > > > {{LocalMetadataStore#put(DirListingMetadata)}} only puts the metadata into > {{dirHash}}, thus all {{FileStatus}} s are missing from > {{LocalMedataStore#fileHash()}}, which makes it confuse to use. > So in the current way, to correctly put file status into the store (and also > set {{authoriative}} flag), you need to run {code} > List metas = new ArrayList(); > boolean authorizative = true; > for (S3AFileStatus status : files) { >PathMetadata meta = new PathMetadata(status); >store.put(meta); > } > DirListingMetadata dirMeta = new DirMeta(parent, metas, authorizative); > store.put(dirMeta); > {code} > Since solely calling {{store.put(dirMeta)}} is not correct, and calling > {{store.put(dirMeta);}} after putting all sub-file {{FileStatuss}} does the > repetitive jobs. Can we just use a {{put(PathMetadata)}} and a > {{get/setAuthorative()}} in the MetadataStore interface instead? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13756) LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash.
[ https://issues.apache.org/jira/browse/HADOOP-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16446879#comment-16446879 ] Gabor Bota edited comment on HADOOP-13756 at 4/21/18 4:26 PM: -- Test & verify runs on us-west-2 successfully for the patch. was (Author: gabor.bota): Test & verify ran on us-west-2 successfully for the patch. > LocalMetadataStore#put(DirListingMetadata) should also put file metadata into > fileHash. > --- > > Key: HADOOP-13756 > URL: https://issues.apache.org/jira/browse/HADOOP-13756 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.0.0-beta1 >Reporter: Lei (Eddy) Xu >Assignee: Gabor Bota >Priority: Major > Attachments: HADOOP-13756.001.patch > > > {{LocalMetadataStore#put(DirListingMetadata)}} only puts the metadata into > {{dirHash}}, thus all {{FileStatus}} s are missing from > {{LocalMedataStore#fileHash()}}, which makes it confuse to use. > So in the current way, to correctly put file status into the store (and also > set {{authoriative}} flag), you need to run {code} > List metas = new ArrayList(); > boolean authorizative = true; > for (S3AFileStatus status : files) { >PathMetadata meta = new PathMetadata(status); >store.put(meta); > } > DirListingMetadata dirMeta = new DirMeta(parent, metas, authorizative); > store.put(dirMeta); > {code} > Since solely calling {{store.put(dirMeta)}} is not correct, and calling > {{store.put(dirMeta);}} after putting all sub-file {{FileStatuss}} does the > repetitive jobs. Can we just use a {{put(PathMetadata)}} and a > {{get/setAuthorative()}} in the MetadataStore interface instead? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org