[
https://issues.apache.org/jira/browse/SPARK-21252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068082#comment-16068082
]
igor mazor commented on SPARK-21252:
------------------------------------
I still think that the developer should be able to configure by him self what
precision he needs.
For example and issue that I have:
Consuming from kafka usually take 40-50 ms, sometimes there are network issues
and the kafka consumer unable to fetch data for a period of request.timeout.ms,
which is 20 seconds in my case, after that the kafka consumer get request time
out, it would retry the request again and usually the second attempt would
succeed. So eventually the UI would show that that stage took 20 seconds, but
in reality it took 20 seconds and 40 ms. Seeing in the UI that it took 20
seconds dont give any indication that there was afterwards seconds attend that
took 40 ms and hence its quite hard to understand what exactly happens.
> The duration times showed by spark web UI are inaccurate
> --------------------------------------------------------
>
> Key: SPARK-21252
> URL: https://issues.apache.org/jira/browse/SPARK-21252
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.1.1
> Reporter: igor mazor
>
> The duration times showed by spark UI are inaccurate and seems to be rounded.
> For example when a job had 2 stages, first stage executed in 47 ms and second
> stage in 3 seconds, the total execution time showed by the UI is 4 seconds.
> Another example, first stage was executed in 20 ms and second stage in 4
> seconds, the total execution time showed by the UI would be in that case also
> 4 seconds.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
