zzzzming95 opened a new pull request, #9905: URL: https://github.com/apache/iceberg/pull/9905
### What changes were proposed in this pull request? A user was attempting to convert an ORC backed external table in hive to a Iceberg table using the migrate command but was immediately met with a "Can not promote TIMESTAMP to TIMESTAMP" error. This occurs because our Spark -> Iceberg conversion code always converts to a Timestamp.withZone. it relate issue : https://github.com/apache/iceberg/issues/9784 ``` spark.sql("CREATE EXTERNAL TABLE mytable (foo timestamp) STORED AS orc LOCATION '/Users/russellspitzer/Temp/foo'") spark.sql("INSERT INTO mytable VALUES (now())") spark.sql("CALL spark_catalog.system.migrate('mytable')") spark.sql("SELECT * FROM mytable") ``` ``` java.lang.IllegalArgumentException: Can not promote TIMESTAMP type to TIMESTAMP at org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:441) at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:301) at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:275) at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:258) ``` add a config , `iceberg.orc.convert.timestamptz` , when set is as true , it will auto convert orc `TIMESTAMP` as orc `TIMESTAMP_INSTANT` , so we can fix this issue. ### Why are the changes needed? fix issue. ### Does this PR introduce any user-facing change? No ### How was this patch tested? add a UT -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
