Cheolsoo Park created PIG-2856:
----------------------------------
Summary: AvroStorage doesn't load files in the directories when a
glob pattern matches both files and directories.
Key: PIG-2856
URL: https://issues.apache.org/jira/browse/PIG-2856
Project: Pig
Issue Type: Bug
Components: piggybank
Affects Versions: 0.10.0
Reporter: Cheolsoo Park
Assignee: Cheolsoo Park
This is a regression from PIG-2492.
When a glob pattern such as '*' matches not only files but also directories,
AvroStorage does not load files in the directories. This is a bug in
getAllSubDirs() that can be fixed as follows:
{code}
static boolean getAllSubDirs(Path path, Job job, Set<Path> paths)
...
FileStatus[] matchedFiles = fs.globStatus(path, PATH_FILTER);
...
for (FileStatus file : matchedFiles) {
if (file.isDir()) {
- for (FileStatus sub : fs.listStatus(path)) {
+ for (FileStatus sub : fs.listStatus(file.getPath())) {
getAllSubDirs(sub.getPath(), job, paths);
}
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
