cshuo commented on code in PR #19373:
URL: https://github.com/apache/hudi/pull/19373#discussion_r3654035802
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadata.java:
##########
@@ -563,10 +565,12 @@ private ClosableIterator<IndexedRecord>
readSliceWithFilter(Predicate predicate,
.rangeType(InstantRange.RangeType.EXACT_MATCH)
.explicitInstants(validInstantTimestamps).build());
+ boolean useLsmReader = shouldUseLsmReader(metadataMetaClient, fileSlice);
+
// If reuse is enabled and full scan is allowed for the partition, we can
reuse the file readers for base files and the reader context for the log files.
Map<StoragePath, HoodieAvroFileReader> baseFileReaders =
Collections.emptyMap();
ReusableFileGroupRecordBufferLoader<IndexedRecord> recordBufferLoader =
null;
- boolean shouldReuse = reuse &&
isFullScanAllowedForPartition(fileSlice.getPartitionPath());
+ boolean shouldReuse = !useLsmReader && reuse &&
isFullScanAllowedForPartition(fileSlice.getPartitionPath());
Review Comment:
Yes, agreed. Reuse is primarily intended for small FILES partitions, the
most frequent MDT lookup path. The LSM reader mainly benefits from exploiting
sorted runs to avoid buffering intermediate records, so it doesn't have a clear
advantage in this reused buffer scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
