joeyutong commented on issue #9374: URL: https://github.com/apache/gravitino/issues/9374#issuecomment-3624449060
@shashihoushengqia @FANNG1 From my tests, the spark connector doesn't support Mysql's `DateTime` type currently. `DateTime` type will be converted to `timestamp without time zone` In [`MysqlTypeConverter`](https://github.com/apache/gravitino/blob/main/catalogs/catalog-jdbc-mysql/src/main/java/org/apache/gravitino/catalog/mysql/converter/MysqlTypeConverter.java#L88), which will not be handled in [SparkTypeConverter](https://github.com/apache/gravitino/blob/main/spark-connector/spark-common/src/main/java/org/apache/gravitino/spark/connector/SparkTypeConverter.java#L158). An Exception is thrown in this case ``` java.lang.UnsupportedOperationException: Not support TimestampType{withTimeZone=false, precision=0} at org.apache.gravitino.spark.connector.SparkTypeConverter.toSparkType(SparkTypeConverter.java:191) at org.apache.gravitino.spark.connector.jdbc.SparkJdbcTypeConverter.toSparkType(SparkJdbcTypeConverter.java:37) at org.apache.gravitino.spark.connector.utils.GravitinoTableInfoHelper.lambda$schema$0(GravitinoTableInfoHelper.java:89) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at org.apache.gravitino.spark.connector.utils.GravitinoTableInfoHelper.schema(GravitinoTableInfoHelper.java:93) at org.apache.gravitino.spark.connector.jdbc.SparkJdbcTable.schema(SparkJdbcTable.java:64) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
