[ https://issues.apache.org/jira/browse/HADOOP-14020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Fabbri updated HADOOP-14020: ---------------------------------- Status: Open (was: Patch Available) > Optimize dirListingUnion > ------------------------ > > Key: HADOOP-14020 > URL: https://issues.apache.org/jira/browse/HADOOP-14020 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Sean Mackrory > Assignee: Sean Mackrory > Attachments: HADOOP-14020-HADOOP-13345.001.patch, > HADOOP-14020-HADOOP-13345.002.patch, HADOOP-14020-HADOOP-13345.003.patch, > HADOOP-14020-HADOOP-13345.004.patch > > > There's a TODO in dirListingUnion: > {quote}// TODO optimize for when allowAuthoritative = false{quote} > There will be cases when we can intelligently avoid a round trip: if S3A > results are a subset or the metadatastore results (including them being equal > or empty) then writing back will do nothing (although perhaps that should set > the authoritative flag if it isn't set already). > There may also be cases where users want to just skip that altogether. It's > wasted work if authoritative mode is disabled, so perhaps we want to trigger > a skip if that's false, or perhaps it should be a separate property. First > one makes for simpler config, second is more flexible... -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org