gengliangwang commented on PR #45621:
URL: https://github.com/apache/spark/pull/45621#issuecomment-2011010486
@RussellSpitzer The same pattern can happen on iceberg if there are multiple
Spark versions in the pipeline
On Spark 3.3+
```
// The following can be Replace table as select as well
spark.read.parquet(...).write.mode("overwrite").format("iceberg").saveAsTable("iceberg_table")
```
On Spark 3.2 and earlier:
```
> select * from iceberg_ts
java.lang.IllegalArgumentException: Cannot handle timestamp without timezone
fields in Spark. Spark does not natively support this type but if you would
like to handle all timestamps as timestamp with timezone set
'spark.sql.iceberg.handle-timestamp-without-timezone' to true. This will not
change the underlying values stored but will change their displayed values in
Spark. For more information please see
https://docs.databricks.com/spark/latest/dataframes-datasets/dates-timestamps.html#ansi-sql-and-spark-sql-timestamps
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
