[ https://issues.apache.org/jira/browse/SPARK-30664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-30664: ------------------------------------ Assignee: Apache Spark > Add more metrics to the all stages page > --------------------------------------- > > Key: SPARK-30664 > URL: https://issues.apache.org/jira/browse/SPARK-30664 > Project: Spark > Issue Type: Improvement > Components: Web UI > Affects Versions: 3.1.0 > Reporter: Enrico Minack > Assignee: Apache Spark > Priority: Minor > Attachments: image-2020-01-28-16-12-49-807.png, > image-2020-01-28-16-13-36-174.png, image-2020-01-28-16-15-20-258.png > > > The web UI page for individual stages has many useful metrics to diagnose > poorly performing stages, e.g. spilled bytes or GC time. Identifying those > stages among hundreds or thousands of stages is cumbersome, as you have to > click through all stages on the all stages page. The all stages page should > host more metrics from the individual stages page like > - Peak Execution Memory > - Spill (Memory) > - Spill (Disk) > - GC Time > These additional metrics would make the page more complex, so showing them > should be optional. The individual stages page hides some metrics under > !image-2020-01-28-16-12-49-807.png! . Those new metrics on the all stages > page should also be made optional in the same way. > !image-2020-01-28-16-13-36-174.png! > Existing metrics like > - Input > - Output > - Shuffle Read > - Shuffle Write > could be made optional as well and active by default. Then users can remove > them if they want but get the same view as now by default. > The table extends as additional metrics get checked / unchecked: > !image-2020-01-28-16-15-20-258.png! > Sorting the table by metrics allows to find the stages with highest GC time > or spilled bytes. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org