[ https://issues.apache.org/jira/browse/SPARK-34488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ron Hu updated SPARK-34488: --------------------------- Summary: Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specified stage (was: Support task Metrics Distributions and executor Metrics Distributions in the REST API entry for a specified stage) > Support task Metrics Distributions and executor Metrics Distributions in the > REST API call for a specified stage > ---------------------------------------------------------------------------------------------------------------- > > Key: SPARK-34488 > URL: https://issues.apache.org/jira/browse/SPARK-34488 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 3.0.2 > Reporter: Ron Hu > Priority: Major > Attachments: executorMetricsDistributions.json, > taskMetricsDistributions.json > > > For a specific stage, it is useful to show the task metrics in percentile > distribution. This information can help users know whether or not there is a > skew/bottleneck among tasks in a given stage. We list an example in > [^taskMetricsDistributions.json] > Similarly, it is useful to show the executor metrics in percentile > distribution for a specific stage. This information can show whether or not > there is a skewed load on some executors. We list an example in > [^executorMetricsDistributions.json] > > We define withSummaries query parameter in the REST API for a specific stage > as: > applications/<application_id>/<application_attempt/stages/<stage_id>/<stage_attempt>?withSummaries=[true|false] > When withSummaries=true, both task metrics in percentile distribution and > executor metrics in percentile distribution are included in the REST API > output. The default value of withSummaries is false, i.e. no metrics > percentile distribution will be included in the REST API output. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org