Maxim Gekk created SPARK-31318:
----------------------------------
Summary: Split Parquet/Avro configs for rebasing dates/timestamps
in read and in write
Key: SPARK-31318
URL: https://issues.apache.org/jira/browse/SPARK-31318
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0
Reporter: Maxim Gekk
Currently, Spark provides 2 SQL configs to control rebasing of dates/timestamps
in Parquet and Avro datasource:
spark.sql.legacy.parquet.rebaseDateTime.enabled
spark.sql.legacy.avro.rebaseDateTime.enabled
The configs control rebasing in read and in write. That's can be inconvenient
for users who want to read files saved by Spark 2.4 and earlier versions, and
save dates/timestamps without rebasing.
The ticket aims to split the configs, and introduce separate SQL configs for
read and for write.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
