danny0405 commented on code in PR #8300:
URL: https://github.com/apache/hudi/pull/8300#discussion_r1165101064
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/execution/bulkinsert/JavaCustomColumnsSortPartitioner.java:
##########
@@ -51,9 +52,13 @@ public JavaCustomColumnsSortPartitioner(String[]
columnNames, Schema schema, boo
public List<HoodieRecord<T>> repartitionRecords(
List<HoodieRecord<T>> records, int outputPartitions) {
return records.stream().sorted((o1, o2) -> {
- Object values1 =
HoodieAvroUtils.getRecordColumnValues((HoodieAvroRecord)o1, sortColumnNames,
schema, consistentLogicalTimestampEnabled);
- Object values2 =
HoodieAvroUtils.getRecordColumnValues((HoodieAvroRecord)o2, sortColumnNames,
schema, consistentLogicalTimestampEnabled);
- return values1.toString().compareTo(values2.toString());
+ FlatLists.ComparableList<Comparable> cmp1 = FlatLists.ofComparableArray(
+ HoodieAvroUtils.getRecordColumnValues((HoodieAvroRecord) o1,
sortColumnNames, schema, consistentLogicalTimestampEnabled)
+ );
Review Comment:
cmp1 -> values1, cmp2 -> values2
##########
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/execution/bulkinsert/TestBulkInsertInternalPartitioner.java:
##########
@@ -238,16 +239,19 @@ private Comparator<HoodieRecord<? extends
HoodieRecordPayload>> getCustomColumnC
Comparator<HoodieRecord<? extends HoodieRecordPayload>> comparator =
Comparator.comparing(record -> {
try {
GenericRecord genericRecord = (GenericRecord)
record.getData().getInsertValue(schema).get();
- StringBuilder sb = new StringBuilder();
+ List<Object> keys = new ArrayList<>();
for (String col : sortColumns) {
- sb.append(genericRecord.get(col));
+ keys.add(genericRecord.get(col));
}
-
- return sb.toString();
+ return keys;
} catch (IOException e) {
throw new HoodieIOException("unable to read value for " + sortColumns);
}
- });
+ }, (o1, o2) -> {
+ FlatLists.ComparableList obj1 =
FlatLists.ofComparableArray(o1.toArray());
+ FlatLists.ComparableList obj2 =
FlatLists.ofComparableArray(o2.toArray());
+ return obj1.compareTo(obj2);
Review Comment:
obj1 -> values1, obj2 -> values2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
