[jira] [Commented] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI
[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174576#comment-14174576 ] Apache Spark commented on SPARK-3984: - User 'kayousterhout' has created a pull request for this issue: https://github.com/apache/spark/pull/2832 Display finer grained metrics about task launch overhead in the UI -- Key: SPARK-3984 URL: https://issues.apache.org/jira/browse/SPARK-3984 Project: Spark Issue Type: Improvement Components: Web UI Reporter: Kay Ousterhout Assignee: Kay Ousterhout Fix For: 1.2.0 Right now, the UI does not display the time to deserialize the task, to serialize the task result, or to launch a new thread for the task. When running short jobs (e.g., for ML) these overheads can become significant. It would be great to show these in the summary quantiles for each stage in the UI to facilitate better performance debugging. cc [~sparks] [~shivaram] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-3984) Display finer grained metrics about task launch overhead in the UI
[ https://issues.apache.org/jira/browse/SPARK-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174575#comment-14174575 ] Kay Ousterhout commented on SPARK-3984: --- https://github.com/apache/spark/pull/2832 Display finer grained metrics about task launch overhead in the UI -- Key: SPARK-3984 URL: https://issues.apache.org/jira/browse/SPARK-3984 Project: Spark Issue Type: Improvement Components: Web UI Reporter: Kay Ousterhout Assignee: Kay Ousterhout Fix For: 1.2.0 Right now, the UI does not display the time to deserialize the task, to serialize the task result, or to launch a new thread for the task. When running short jobs (e.g., for ML) these overheads can become significant. It would be great to show these in the summary quantiles for each stage in the UI to facilitate better performance debugging. cc [~sparks] [~shivaram] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
