Steve Loughran created HADOOP-17275: ---------------------------------------
Summary: Implement Incremental globStatus() -> RemoteIterator class Key: HADOOP-17275 URL: https://issues.apache.org/jira/browse/HADOOP-17275 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 3.3.0 Reporter: Steve Loughran globStatus/globber is inefficient, not incremental and doesn't provide a way to query any IOStatistics. Proposed: * new IncrementalGlobber class which takes an interface to the file source (+impls for FS and FC) which offer the incremental ops * use incremental listStatusIncremental calls * or, if no wildcards, full deep listFiles(recursive) * collect IOStatistics from source ops, aggregate and serve Potentially a lot of work...need to look carefully here -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org