[
https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kay Ousterhout updated SPARK-6418:
----------------------------------
Description:
Visualizing how tasks in a stage spend their time can be very helpful to
understanding performance. Many folks have started using the visualization
tools here: https://github.com/kayousterhout/trace-analysis (see the README at
the bottom) to analyze their jobs after they've finished running, but it would
be great if this functionality were natively integrated into Spark's UI.
I'd propose adding a relatively simple visualization to the stage detail page,
that's hidden by default but that users can view by clicking on a drop-down
menu. The plan is to implement this using D3; a mock up of how this would look
(that uses D3) is attached. One change we'll make for the initial
implementation, compared to the attached visualization, is tasks will be sorted
by start time.
This is intended to be a much simpler and more limited version of SPARK-3468
was:
Visualizing how tasks in a stage spend their time can be very helpful to
understanding performance. Many folks have started using the visualization
tools here: https://github.com/kayousterhout/trace-analysis (see the README at
the bottom) to analyze their jobs after they've finished running, but it would
be great if this functionality were natively integrated into Spark's UI.
I'd propose adding a relatively simple visualization to the stage detail page,
that's hidden by default but that users can view by clicking on a drop-down
menu. The plan is to implement this using D3; a mock up of how this would look
(that uses D3) is attached.
This is intended to be a much simpler and more limited version of SPARK-3468
> Add simple per-stage visualization to the UI
> --------------------------------------------
>
> Key: SPARK-6418
> URL: https://issues.apache.org/jira/browse/SPARK-6418
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Reporter: Kay Ousterhout
> Assignee: Pradyumn Shroff
> Attachments: Screen Shot 2015-03-18 at 6.13.04 PM.png
>
>
> Visualizing how tasks in a stage spend their time can be very helpful to
> understanding performance. Many folks have started using the visualization
> tools here: https://github.com/kayousterhout/trace-analysis (see the README
> at the bottom) to analyze their jobs after they've finished running, but it
> would be great if this functionality were natively integrated into Spark's UI.
> I'd propose adding a relatively simple visualization to the stage detail
> page, that's hidden by default but that users can view by clicking on a
> drop-down menu. The plan is to implement this using D3; a mock up of how
> this would look (that uses D3) is attached. One change we'll make for the
> initial implementation, compared to the attached visualization, is tasks will
> be sorted by start time.
> This is intended to be a much simpler and more limited version of SPARK-3468
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
