rahil-c commented on code in PR #19575:
URL: https://github.com/apache/hudi/pull/19575#discussion_r3858225376
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -255,6 +256,12 @@ public boolean commitStats(String instantTime,
TableWriteStats tableWriteStats,
return true;
}
extraMetadata = updateExtraMetadata(extraMetadata);
+ // The commit is the boundary for the record index lookup counters.
Snapshot them into this commit's
+ // metadata now, but only release them from the registry once the commit
has actually landed --
+ // hooked here rather than in updateExtraMetadata because that is shared
with table-service
+ // scheduling, which would otherwise consume the counters into a
compaction or clustering plan.
+ ExecutorMetrics.DrainedCounters executorCounters =
+ ExecutorMetrics.snapshotIntoCommitMetadata(extraMetadata.get(),
config);
Review Comment:
I am debating on if we should even be tracking these new metrics within the
commit metadata, or if we should fully delegate the responsibility to metrics
reporter in order to reduce the complexity here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
