[
https://issues.apache.org/jira/browse/HADOOP-14020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Mackrory updated HADOOP-14020:
-----------------------------------
Attachment: HADOOP-14020-HADOOP-13345.002.patch
Attaching a patch that just uses authoritative mode to enable / disable the new
logic, and also fixing some javadoc errors.
Not sure what was up with my failure to build all of a sudden yesterday.
hadoop-kms was not building the classes or tests artifacts, and the failure to
find them got blamed on the DynamoDB repo. Rebasing my local branch on the
latest trunk did the trick...
> Optimize dirListingUnion
> ------------------------
>
> Key: HADOOP-14020
> URL: https://issues.apache.org/jira/browse/HADOOP-14020
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Sean Mackrory
> Assignee: Sean Mackrory
> Attachments: HADOOP-14020-HADOOP-13345.001.patch,
> HADOOP-14020-HADOOP-13345.002.patch
>
>
> There's a TODO in dirListingUnion:
> {quote}// TODO optimize for when allowAuthoritative = false{quote}
> There will be cases when we can intelligently avoid a round trip: if S3A
> results are a subset or the metadatastore results (including them being equal
> or empty) then writing back will do nothing (although perhaps that should set
> the authoritative flag if it isn't set already).
> There may also be cases where users want to just skip that altogether. It's
> wasted work if authoritative mode is disabled, so perhaps we want to trigger
> a skip if that's false, or perhaps it should be a separate property. First
> one makes for simpler config, second is more flexible...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
