As we have seen multiple times in the mailing list, users want to have the capability of getting all files out of a multi-level directory structure.
4/1/2008: http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200804.mbox/%[email protected]%3e 2/3/2009: http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200902.mbox/%[email protected]%3e 6/2/2009: http://mail-archives.apache.org/mod_mbox/hadoop-common-user/200906.mbox/%[email protected]%3e One solution that our users had is to write a new FileInputFormat, but that means all existing FileInputFormat subclasses need to be changed in order to support this feature. We can easily provide a JobConf option (which defaults to false) to FileInputFormat.listStatus(...) to recursively go into directory structure. See https://issues.apache.org/jira/browse/MAPREDUCE-1501 for details. I cced several people who were involved in the earlier discussions. Can somebody help review the patch? Zheng
