Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/21741#discussion_r201233974
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -378,6 +378,15 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
+ val PARQUET_FILTER_PUSHDOWN_TIMESTAMP_ENABLED =
+ buildConf("spark.sql.parquet.filterPushdown.timestamp")
+ .doc("If true, enables Parquet filter push-down optimization for
Timestamp. " +
+ "This configuration only has an effect when
'spark.sql.parquet.filterPushdown' is " +
+ "enabled and Timestamp stored as TIMESTAMP_MICROS or
TIMESTAMP_MILLIS type.")
+ .internal()
+ .booleanConf
+ .createWithDefault(true)
--- End diff --
May be default should be `false`. because `PARQUET_OUTPUT_TIMESTAMP_TYPE`
default is `INT96`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
