[
https://issues.apache.org/jira/browse/HUDI-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu updated HUDI-3125:
-----------------------------
Sprint: Hudi-Sprint-0.10.1
> Spark SQL writing timestamp type don't need to disable
> `spark.sql.datetime.java8API.enabled` manually
> -----------------------------------------------------------------------------------------------------
>
> Key: HUDI-3125
> URL: https://issues.apache.org/jira/browse/HUDI-3125
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Spark Integration
> Reporter: Yann Byron
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.11.0, 0.10.1
>
>
> {code:java}
> create table h0_p(id int, name string, price double, dt timestamp) using hudi
> partitioned by(dt) options(type = 'cow', primaryKey = 'id');
> insert into h0_p values (3, 'a1', 10, cast('2021-05-08 00:00:00' as
> timestamp)); {code}
> By default, that run the sql above will throw exception:
> {code:java}
> Caused by: java.lang.ClassCastException: java.time.Instant cannot be cast to
> java.sql.Timestamp
> at
> org.apache.hudi.AvroConversionHelper$.$anonfun$createConverterToAvro$8(AvroConversionHelper.scala:306)
> at
> org.apache.hudi.AvroConversionHelper$.$anonfun$createConverterToAvro$8$adapted(AvroConversionHelper.scala:306)
> at scala.Option.map(Option.scala:230) {code}
> We need disable `spark.sql.datetime.java8API.enabled` manually to make it
> work:
> {code:java}
> set spark.sql.datetime.java8API.enabled=false; {code}
> And the command must be executed in the runtime. It can't work if provide
> this by spark-sql command: `spark-sql --conf
> spark.sql.datetime.java8API.enabled=false`. That's because this config is
> forced to enable when launch spark-sql.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
