[ https://issues.apache.org/jira/browse/HUDI-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenning Ding reassigned HUDI-1181: ---------------------------------- Assignee: Wenning Ding > Decimal type display issue for record key field > ----------------------------------------------- > > Key: HUDI-1181 > URL: https://issues.apache.org/jira/browse/HUDI-1181 > Project: Apache Hudi > Issue Type: Bug > Reporter: Wenning Ding > Assignee: Wenning Ding > Priority: Major > Labels: pull-request-available > > When using *fixed_len_byte_array* decimal type as Hudi record key, Hudi would > not correctly display the decimal value, instead, Hudi would display it as a > byte array. > During the Hudi writing phase, Hudi would save the parquet source data into > Avro Generic Record. For example, the source parquet data has a column with > decimal type: > > {code:java} > optional fixed_len_byte_array(16) OBJ_ID (DECIMAL(38,0));{code} > > Then Hudi will convert it into the following avro decimal type: > {code:java} > { > "name" : "OBJ_ID", > "type" : [ { > "type" : "fixed", > "name" : "fixed", > "namespace" : "hoodie.hudi_ln.hudi_ln_record.OBJ_ID", > "size" : 16, > "logicalType" : "decimal", > "precision" : 38, > "scale" : 0 > }, "null" ] > } > {code} > This decimal field would be stored as a fixed length bytes array. And in the > reading phase, Hudi will convert this bytes array back to a readable decimal > value through this > [converter|https://github.com/apache/hudi/blob/master/hudi-spark/src/main/scala/org/apache/hudi/AvroConversionHelper.scala#L58]. > However, the problem is, when setting decimal type as record keys, Hudi would > read the value from Avro Generic Record and then directly convert it into > String type(See > [here|https://github.com/apache/hudi/blob/master/hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java#L76]). > As a result, what shows in the _hoodie_record_key field would be something > like: LN_LQDN_OBJ_ID:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 40, 95, -71].So > we need to handle this special case to convert bytes array back before > converting to String. -- This message was sent by Atlassian Jira (v8.3.4#803005)