[ https://issues.apache.org/jira/browse/MAPREDUCE-6760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sunil Govindan updated MAPREDUCE-6760: -------------------------------------- Target Version/s: 3.3.0 (was: 3.2.0) Bulk update: moved all 3.2.0 non-blocker issues, please move back if it is a blocker. > LocatedFileStatusFetcher to use listFiles(recursive) > ---------------------------------------------------- > > Key: MAPREDUCE-6760 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6760 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 > Affects Versions: 2.8.0 > Reporter: Steve Loughran > Priority: Major > > {{LocatedFileStatusFetcher}} does parallelized path listing, but it does make > recursive calls on every subdir. > If we could switch it to use {{FileSystem.listFiles(recursive)}}, object > stores that have high-performance implementations of that operation would see > significant speedup. > HADOOP-13208 implements that for S3A; Azure, swift &c can do the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org