github-actions[bot] commented on code in PR #65354:
URL: https://github.com/apache/doris/pull/65354#discussion_r3541325549
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonJniScanner.java:
##########
@@ -364,9 +406,157 @@ protected TableSchema parseTableSchema() throws
UnsupportedOperationException {
public Map<String, String> getStatistics() {
Map<String, String> statistics = new HashMap<>();
statistics.put("counter:PaimonJniIOManagerEnabled", ioManager != null
? "1" : "0");
+ statistics.put("counter:PaimonJniActiveScannerCount",
String.valueOf(ACTIVE_SCANNERS.get()));
+ statistics.put("counter:PaimonJniActiveScannerPeakCount",
String.valueOf(PEAK_ACTIVE_SCANNERS.get()));
Review Comment:
These process-wide snapshot metrics are being exported through the additive
JNI counter contract. `getStatistics()` now reports values like
`PaimonJniActiveScannerPeakCount`, `PaimonJniAsyncReaderThreadCount`, and the
JVM heap/non-heap `bytes:` metrics from static/JVM-global state, but both the
v2 collector and the legacy JNI collector attach same-name profile counters and
call `COUNTER_UPDATE` for every scanner close. Because
`RuntimeProfile::add_counter()` returns the existing counter for later splits,
a query with many Paimon JNI splits will sum repeated snapshots: for example a
4 GiB JVM heap snapshot collected from 50 splits is displayed around 200 GiB,
and peak active scanner/thread counts are multiplied by the number of collected
splits. Please keep `getStatistics()` to scanner-local cumulative deltas, or
add a non-additive profile path for gauges/peaks (set current values once/last
and use max-style counters for peak values) so these observability numbers
remain meaningful
for multi-split scans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
