Cheng Lian created SPARK-16317:
----------------------------------
Summary: Add file filtering interface for FileFormat
Key: SPARK-16317
URL: https://issues.apache.org/jira/browse/SPARK-16317
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.0.0
Reporter: Cheng Lian
{{FileFormat}} data sources like Parquet and Avro (provided by spark-avro) have
customized file filtering logics. For example, Parquet needs to filter out
summary files, while Avro provides a Hadoop configuration option to filter out
all files whose names don't end with ".avro".
It would be nice to have a general file filtering interface in {{FileFormat}}
to handle similar requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
