skywalker0618 commented on code in PR #18813:
URL: https://github.com/apache/hudi/pull/18813#discussion_r3290077936
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/RecordLevelIndexBackend.java:
##########
@@ -161,15 +176,30 @@ private BucketCache bootstrapPartition(String
partitionPath) {
return cache;
}
+ // Time the remote metadata-table reads (RLI bucket listing + record index
location read) so
+ // operators can observe per-partition bootstrap latency. The local cache
fill is excluded so
+ // the histogram tracks remote cost only, matching
remoteIndexLookupLatency on the global RLI path.
+ metrics.startRemoteIndexLookup();
+ final HoodiePairData<String, HoodieRecordGlobalLocation> locations;
try {
Map<String, List<FileSlice>> partitionedFileGroups =
metadataTable.getBucketizedFileGroupsForPartitionedRLI(MetadataPartitionType.RECORD_INDEX);
List<FileSlice> fileSlices = partitionedFileGroups.get(partitionPath);
if (fileSlices == null || fileSlices.isEmpty()) {
+ metrics.endRemoteIndexLookup();
+ metrics.updateRemoteLookupKeysCount(0L);
return cache;
}
- HoodiePairData<String, HoodieRecordGlobalLocation> locations =
- metadataTable.readRecordIndexLocations(fileSlicesToFilter ->
fileSlices);
+ locations = metadataTable.readRecordIndexLocations(fileSlicesToFilter ->
fileSlices);
Review Comment:
Great point! Fixed
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/index/RecordLevelIndexBackend.java:
##########
@@ -161,15 +176,30 @@ private BucketCache bootstrapPartition(String
partitionPath) {
return cache;
}
+ // Time the remote metadata-table reads (RLI bucket listing + record index
location read) so
+ // operators can observe per-partition bootstrap latency. The local cache
fill is excluded so
+ // the histogram tracks remote cost only, matching
remoteIndexLookupLatency on the global RLI path.
+ metrics.startRemoteIndexLookup();
+ final HoodiePairData<String, HoodieRecordGlobalLocation> locations;
Review Comment:
Fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
