[ https://issues.apache.org/jira/browse/HADOOP-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Fabbri updated HADOOP-13448: ---------------------------------- Attachment: HADOOP-13448-HADOOP-13345.002.patch I'm posting a v2 patch which evolves the initial patch some. @cnauorth is on vacation, but I wanted to keep the process moving with some suggested modifications I made (for my work on subtask HADOOP-13452): - Define concrete types PathMetadata and DirListingMetadata. - Both are "containers" of FileStatuses. I use a type bound to allow them to hold any subclass of FileStatus, e.g. S3AFileStatus. - DirListingMetadata is a collection of FileStatus plus an isAuthoritative flag as recently discussed in HADOOP-13345. - I thought about having a common base class but it felt contrived, as things like isAuthoratitive(), lookup/get/delete() only make sense for directory listings, not single path metadata. That is, I didn't see a common "is a" relationship. - I removed MetadataStore's dependency on S3AFileSystem for the Logger. I'm hoping this code can eventually be factored out of s3a and become useful for other FS clients. - Added another put() method to MetadataStore that takes a DirListingMetadata. I explain the need for the isAuthorative bit on put() in my recent comment on HADOOP-13345 Question / Next Steps: - Should MetadataStore be an interface? I left it an abstract class but it looks more like an Interface at the moment. Perhaps we'll want to add back a common Logger object, that doesn't depend on S3AFileSystem? - How do folks feel about this API consuming/producing FileStatus'? I think it simplifies the client code quite a bit, which is one of my top concerns. I'm fine with changing it if it becomes awkward. - I'm wondering if DirListingMetadata will become a collection of PathMetadata instead of FileStatus'. For now I did the latter, thinking of what FS clients will need: FileStatus'. If we add things like content hashes or etags in future I could see this becoming a collection of PathMetadata instead. > S3Guard: Define MetadataStore interface. > ---------------------------------------- > > Key: HADOOP-13448 > URL: https://issues.apache.org/jira/browse/HADOOP-13448 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Chris Nauroth > Assignee: Chris Nauroth > Attachments: HADOOP-13448-HADOOP-13345.001.patch, > HADOOP-13448-HADOOP-13345.002.patch > > > Define the common interface for metadata store operations. This is the > interface that any metadata back-end must implement in order to integrate > with S3Guard. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org