[ https://issues.apache.org/jira/browse/MAPREDUCE-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Dere updated MAPREDUCE-5756: ---------------------------------- Attachment: MAPREDUCE-5756.2.patch Patch v2 adds unit test > CombineFileInputFormat.getSplits() including directories in its results > ----------------------------------------------------------------------- > > Key: MAPREDUCE-5756 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5756 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Jason Dere > Attachments: MAPREDUCE-5756.1.patch, MAPREDUCE-5756.2.patch > > > Trying to track down HIVE-6401, where we see some "is not a file" errors > because getSplits() is giving us directories. I believe the culprit is > FileInputFormat.listStatus(): > {code} > if (recursive && stat.isDirectory()) { > addInputPathRecursively(result, fs, stat.getPath(), > inputFilter); > } else { > result.add(stat); > } > {code} > Which seems to be allowing directories to be added to the results if > recursive is false. Is this meant to return directories? If not, I think it > should look like this: > {code} > if (stat.isDirectory()) { > if (recursive) { > addInputPathRecursively(result, fs, stat.getPath(), > inputFilter); > } > } else { > result.add(stat); > } > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)