lokeshj1703 opened a new issue, #19621:
URL: https://github.com/apache/hudi/issues/19621
When clustering (or `bulk_insert`) sorts by a column of BINARY/bytes type on
the Spark record path (`HoodieSparkRecord`), the job fails with:
```
java.lang.ClassCastException: [B cannot be cast to java.lang.Comparable
at
org.apache.hudi.common.util.collection.FlatLists.ofComparableArray(FlatLists.java:55)
```
`SortUtils.getComparableSortColumns` builds the sort key by reading each
sort column's value and passing it to `FlatLists.ofComparableArray`, which
casts every element to `Comparable`. On the Spark record path a binary column's
value is a raw `byte[]`, which is not `Comparable`, so the cast throws and
every clustering / bulk-insert attempt on that column fails.
The Avro record path does not hit this: `HoodieAvroUtils.getNestedFieldVal`
returns a `java.nio.ByteBuffer` (which is `Comparable`) for the same column.
### Expected
A BINARY sort column should be usable as a clustering / bulk-insert sort
key, ordered consistently with the Avro path.
### Environment
Hudi 1.x (master), Spark record type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
