[GitHub] [hudi] askldjd commented on issue #4233: [SUPPORT] Hudi parquet INT16 conversion loses LogicalType information

2021-12-09 Thread GitBox
askldjd commented on issue #4233: URL: https://github.com/apache/hudi/issues/4233#issuecomment-990086967 For those who are affected by this bug, our workaround is to convert int16 fields to int32 before giving it to hudi. In our use case, the parquet is generated by AWS DMS, and we just ap

[GitHub] [hudi] askldjd commented on issue #4233: [SUPPORT] Hudi parquet INT16 conversion loses LogicalType information

2021-12-09 Thread GitBox
askldjd commented on issue #4233: URL: https://github.com/apache/hudi/issues/4233#issuecomment-990076527 I did a bit of tracing and I think this is where we lost the ShortType information. `AvroConversionUtils.convertStructTypeToAvroSchema` https://github.com/apache/hudi/blob/rel