Enrico Minack created SPARK-30664:
-------------------------------------
Summary: Add more metrics to the all stages page
Key: SPARK-30664
URL: https://issues.apache.org/jira/browse/SPARK-30664
Project: Spark
Issue Type: Improvement
Components: Web UI
Affects Versions: 3.0.0
Reporter: Enrico Minack
The web UI page for individual stages has many useful metrics to diagnose
poorly performing stages, e.g. spilled bytes or GC time. Identifying those
stages among hundreds or thousands of stages is cumbersome, as you have to
click through all stages on the all stages page. The all stages page should
host more metrics from the individual stages page like
- Peak Execution Memory
- Spill (Memory)
- Spill (Disk)
- GC Time
These additional metrics would make the page more complex, so showing them
should be optional. The individual stages page hides some metrics under "Show
Additional Metrics". Those new metrics on the all stages page should also be
made optional in the same way.
Existing metrics like
- Input
- Output
- Shuffle Read
- Shuffle Write
could be made optional as well and active by default. Then users can remove
them if they want but get the same view as now by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
