liuzqt commented on code in PR #47872:
URL: https://github.com/apache/spark/pull/47872#discussion_r1746847493
##########
core/src/main/protobuf/org/apache/spark/status/protobuf/store_types.proto:
##########
@@ -128,6 +128,8 @@ message TaskDataWrapper {
int64 shuffle_merged_local_bytes_read = 49;
int64 shuffle_remote_reqs_duration = 50;
int64 shuffle_merged_remote_req_duration = 51;
+ int64 peak_on_heap_execution_memory = 52;
+ int64 peak_off_heap_execution_memory = 53;
Review Comment:
Looks like we already have executor level stage memory metrics:
- `ExecutorStageSummary` has `val peakMemoryMetrics: Option[ExecutorMetrics]`
- `ExecutorSummary` has `val memoryMetrics: Option[MemoryMetrics]`
- `ExecutorMetricsDistributions` has `val peakMemoryMetrics:
ExecutorPeakMetricsDistributions`
which are aggregated through
[AppStatusListener.updateStageLevelPeakExecutorMetrics](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/AppStatusListener.scala#L1000)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
