[ https://issues.apache.org/jira/browse/SPARK-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174577#comment-14174577 ]
Apache Spark commented on SPARK-3983: ------------------------------------- User 'kayousterhout' has created a pull request for this issue: https://github.com/apache/spark/pull/2832 > Scheduler delay (shown in the UI) is incorrect > ---------------------------------------------- > > Key: SPARK-3983 > URL: https://issues.apache.org/jira/browse/SPARK-3983 > Project: Spark > Issue Type: Bug > Components: Web UI > Reporter: Kay Ousterhout > Assignee: Kay Ousterhout > Fix For: 1.2.0 > > > The reported scheduler delay includes time to get a new thread (from a > threadpool) in order to start the task, time to deserialize the task, and > time to serialize the result. None of these things are delay caused by the > scheduler; including them as such is misleading. > This is especially problematic when debugging performance of short tasks > (that run in 10s of milliseconds), when the scheduler delay can be very large > relative to the task duration. > cc [~sparks] [~shivaram] -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org