hudi-bot opened a new issue, #14617:
URL: https://github.com/apache/hudi/issues/14617
I am using Hudi 0.5.0 and writing to COW table using Spark. Consecutive
writes fails with below error.
Caused by: java.lang.ClassCastException: optional int32
trli_sequence_number_list is not a groupCaused by:
java.lang.ClassCastException: optional int32 trli_sequence_number_list is not a
group at org.apache.parquet.schema.Type.asGroupType(Type.java:202) at
org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:206)
at
org.apache.parquet.avro.AvroRecordConverter.<init>(AvroRecordConverter.java:112)
at
org.apache.parquet.avro.AvroRecordConverter.<init>(AvroRecordConverter.java:79)
at
org.apache.parquet.avro.AvroRecordMaterializer.<init>(AvroRecordMaterializer.java:33)
at
org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:132)
at
org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:175)
at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:149)
at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:125) at
org.apache.hudi.func.ParquetReaderIterator.hasNext(P
arquetReaderIterator.java:47) at
org.apache.hudi.common.util.queue.IteratorBasedQueueProducer.produce(IteratorBasedQueueProducer.java:44)
at
org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$0(BoundedInMemoryExecutor.java:91)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ... 4
more
Corresponding schema is as below.
{
"name" : "trli_sequence_number_list",
"type" : [ {
"type" : "array",
"items" : [ "string", "null" ]
}, "null" ]
},
I have multiple columns having array data type.
{
"name" : "rli_invoice_number_list",
"type" : [ {
"type" : "array",
"items" : [ "string", "null" ]
}, "null" ]
}, {
"name" : "trli_sequence_number_list",
"type" : [ {
"type" : "array",
"items" : [ "string", "null" ]
}, "null" ]
},
Is there a way to avoid this error?
Similarly there is an another error log on the same run.
Caused by: java.lang.ClassCastException: optional binary
app_application_list (UTF8) is not a group
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-1057
- Type: Bug
---
## Comments
29/Jun/20 02:33;shenhong;Did you use hivemeta? Whether the int32 type is
defined in hivemeta, or the data is int32 type？;;;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
