Xiao Li created SPARK-16010: ------------------------------- Summary: Code Refactoring, Test Case Improvement and Description Updates for SQLConf spark.sql.parquet.filterPushdown Key: SPARK-16010 URL: https://issues.apache.org/jira/browse/SPARK-16010 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 2.0.0 Reporter: Xiao Li
Starting Spark 2.0, vectorized decoding is introduced for parquet reading. This feature changes the filter pushdown behavior of parquet reading. Thus, we need to update the out-of-dated description of two external `SQLConf`: `spark.sql.parquet.filterPushdown` and `spark.sql.parquet.enableVectorizedReader`. The code for building `parquetReader` can be slightly simplified for easy maintenance. The current test cases do not verify the behavior when `spark.sql.parquet.filterPushdown` is set to `false`, a test case for improving the test case coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org