To me, this is a bug. In a parallel system, the duration of a stage is often
determined by the slowest task/executor. To monitor/debug a skew issue, the
maximal value (or 100% percentile value) is more useful than the 95%
percentile value. I am happy to see that we do it right with Web UI. For
R
The change makes sense of course; I think the real question for everyone
is: would you change the semantics of this info from the API in a minor
release, or would it have to wait? is it more bug fix than breaking change?
On Sun, Jan 10, 2021 at 7:38 PM angers.zhu wrote:
> Hi devs,
>
> These days
Hi devs,These days I found that about taskSummary, the Restful API(/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/taskSummary)'s default quantiles is `0.05,0.25,0.5,0.75,0.95`But in Spark Web UI’s stage page, the quantiles is `0.0,0.25,0.5,0.75,1.0`. And the
