[GitHub] [hudi] wzx140 commented on a diff in pull request #5522: [HUDI-3378][HUDI-3379][HUDI-3381] Rebasing usages of HoodieRecordPayload and raw Avro payload to rely on HoodieRecord instead

[GitBox]

wzx140 commented on code in PR #5522:
URL: https://github.com/apache/hudi/pull/5522#discussion_r873385684


##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/block/HoodieDataBlock.java:
##########
@@ -179,40 +181,45 @@ public final ClosableIterator<IndexedRecord> 
getRecordIterator(List<String> keys
     return FilteringIterator.getInstance(allRecords, keySet, fullKey, 
this::getRecordKey);
   }
 
-  protected ClosableIterator<IndexedRecord> readRecordsFromBlockPayload() 
throws IOException {
+  protected ClosableIterator<HoodieRecord> 
readRecordsFromBlockPayload(HoodieRecord.Mapper mapper) throws IOException {
     if (readBlockLazily && !getContent().isPresent()) {
       // read log block contents from disk
       inflate();
     }
 
     try {
-      return deserializeRecords(getContent().get());
+      return deserializeRecords(getContent().get(), mapper);
     } finally {
       // Free up content to be GC'd by deflating the block
       deflate();
     }
   }
 
-  protected ClosableIterator<IndexedRecord> lookupRecords(List<String> keys, 
boolean fullKey) throws IOException {
+  protected ClosableIterator<HoodieRecord> lookupRecords(List<String> keys, 
boolean fullKey, HoodieRecord.Mapper mapper) throws IOException {
     throw new UnsupportedOperationException(
         String.format("Point lookups are not supported by this Data block type 
(%s)", getBlockType())
     );
   }
 
-  protected abstract byte[] serializeRecords(List<IndexedRecord> records) 
throws IOException;
+  protected abstract byte[] serializeRecords(List<HoodieRecord> records) 
throws IOException;
 
-  protected abstract ClosableIterator<IndexedRecord> deserializeRecords(byte[] 
content) throws IOException;
+  protected abstract ClosableIterator<HoodieRecord> deserializeRecords(byte[] 
content, HoodieRecord.Mapper mapper) throws IOException;
 
   public abstract HoodieLogBlockType getBlockType();
 
   protected Option<Schema.Field> getKeyField(Schema schema) {
     return Option.ofNullable(schema.getField(keyFieldName));
   }
 
-  protected Option<String> getRecordKey(IndexedRecord record) {
-    return getKeyField(record.getSchema())
-        .map(keyField -> record.get(keyField.pos()))
-        .map(Object::toString);
+  protected Option<String> getRecordKey(HoodieRecord record) {
+    if (record.getKey() == null) {
+      IndexedRecord data = ((HoodieAvroIndexRecord) record).getData();
+      return getKeyField(data.getSchema())
+          .map(keyField -> data.get(keyField.pos()))
+          .map(Object::toString);
+    } else {
+      return Option.ofNullable(record.getRecordKey());

Review Comment:
   The hoodieRecord here may be hoodieAvroIndexRecord or hoodieAvroRecord. 
hoodieAvroIndexRecord(avoid to serialise IndexRecord to byte[]) is used to be 
compatible with the logic of the original indexedRecord and does not hold a 
recordkey.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org