rmcyang commented on code in PR #37638:
URL: https://github.com/apache/spark/pull/37638#discussion_r1060964263
##########
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:
##########
@@ -1904,4 +1951,52 @@ long getPos() {
return pos;
}
}
+
+ /**
+ * A class that wraps all the push-based shuffle service metrics.
+ */
+ static class PushMergeMetrics implements MetricSet {
+ // blockAppendCollisions tracks how many times a shuffle block collided
because
+ // of another block for the same reduce partition was being written
+ static final String BLOCK_APPEND_COLLISIONS_METRIC =
"blockAppendCollisions";
+ // lateBlockPushes tracks how many times a shuffle block push request is
too late
+ static final String LATE_BLOCK_PUSHES_METRIC = "lateBlockPushes";
+ // blockBytesWritten tracks the length of the pushed block data written to
file in bytes
+ static final String BLOCK_BYTES_WRITTEN_METRIC = "blockBytesWritten";
+ // deferredBlockBytes tracks the size of the current deferred block parts
buffered in memory.
+ static final String DEFERRED_BLOCK_BYTES_METRIC = "deferredBlockBytes";
+ // deferredBlocks tracks the number of deferred blocks got written to the
merged shuffle file
+ static final String DEFERRED_BLOCKS_METRIC = "deferredBlocks";
+ // staleBlockPushes tracks how many times a shuffle block push request it
stale
+ static final String STALE_BLOCK_PUSHES_METRIC = "staleBlockPushes";
Review Comment:
Sounds good, I added a few lines describing them under `shuffleService`
section, PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
