[ https://issues.apache.org/jira/browse/HUDI-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
sivabalan narayanan updated HUDI-1453: -------------------------------------- Labels: pull-request-available sev:high user-support-issues (was: pull-request-available user-support-issues) > Throw Exception when input data schema is not equal to the hoodie table schema > ------------------------------------------------------------------------------ > > Key: HUDI-1453 > URL: https://issues.apache.org/jira/browse/HUDI-1453 > Project: Apache Hudi > Issue Type: Improvement > Components: Writer Core > Reporter: pengzhiwei > Assignee: pengzhiwei > Priority: Major > Labels: pull-request-available, sev:high, user-support-issues > Fix For: 0.8.0 > > > The hoodie table *h0's* schema is : > {code:java} > (id long, price double){code} > when I write the *dataframe* to *h0* with the follow schema: > {code:java} > (id long, price int){code} > An Exception is threw as follow: > {code:java} > at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:136)at > org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:136) at > org.apache.hudi.common.util.ParquetReaderIterator.hasNext(ParquetReaderIterator.java:49) > at > org.apache.hudi.common.util.queue.IteratorBasedQueueProducer.produce(IteratorBasedQueueProducer.java:45) > at > org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$0(BoundedInMemoryExecutor.java:102) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ... 4 > moreCaused by: java.lang.UnsupportedOperationException: > org.apache.parquet.avro.AvroConverters$FieldIntegerConverter at > org.apache.parquet.io.api.PrimitiveConverter.addDouble(PrimitiveConverter.java:84) > at > org.apache.parquet.column.impl.ColumnReaderImpl$2$2.writeValue(ColumnReaderImpl.java:228) > at > org.apache.parquet.column.impl.ColumnReaderImpl.writeCurrentValueToConverter(ColumnReaderImpl.java:367) > at > org.apache.parquet.io.RecordReaderImplementation.read(RecordReaderImplementation.java:406) > at > org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:226) > ... 11 more > {code} > I have enable the *AVRO_SCHEMA_VALIDATE,* it *can pass the schema validate > in HoodieTable#validateUpsertSchema,* so it is right to write the "int" data > to the "double" field in hoodie. -- This message was sent by Atlassian Jira (v8.3.4#803005)