Hi all,
In case it is useful to some of you:
I have a big batch that needs to use globs (*.parquet for example) to
read input files. It seems that globs do not work out of the box (see
https://issues.apache.org/jira/browse/FLINK-6417)
But there is a workaround:
final FileInputFormat inputFormat =new FileInputFormat(new
Path(extractDir(filePath)));/* or any subclass of FileInputFormat*/ /*extact
parent dir*/
inputFormat.setFilesFilter(new GlobFilePathFilter(Collections.singletonList(filePath), Collections.emptyList()));/*filePath contains glob, the whole path needs to be provided to
GlobFilePathFilter*/
inputFormat.setNestedFileEnumeration(true);
Hope, it helps some people
Etienne Chauchot
