[ https://issues.apache.org/jira/browse/SPARK-26399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17257639#comment-17257639 ]
Ron Hu commented on SPARK-26399: -------------------------------- Hi [~Baohe Zhang] , This ticket proposes a new REST API: http://<spark history server>:18080/api/v1/applications/<application_id>/<application_attempt/stages/<stage_id>/<stage_attempt>/executorSummary. It means to display the percentile distribution of peak memory metrics among the executors used in a given stage. It can help Spark users debug/monitor a bottleneck of a stage. In the ticket https://issues.apache.org/jira/browse/SPARK-32446, it proposed to add a REST API, which can display the percentile distribution of peak memory metrics for all executors used in an application. The REST API is: http://<spark history server>:18080/api/v1/applications/<application_id>/<application_attempt>/executorSummary Hence this ticket displays executorSummary for a given stage inside an application. SPARK-32446 wants to display executorSummary for the entire application. They are different. > Add new stage-level REST APIs and parameters > -------------------------------------------- > > Key: SPARK-26399 > URL: https://issues.apache.org/jira/browse/SPARK-26399 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 3.1.0 > Reporter: Edward Lu > Priority: Major > > Add the peak values for the metrics to the stages REST API. Also add a new > executorSummary REST API, which will return executor summary metrics for a > specified stage: > {code:java} > curl http://<spark history server>:18080/api/v1/applications/<application > id>/<application attempt/stages/<stage id>/<stage > attempt>/executorSummary{code} > Add parameters to the stages REST API to specify: > * filtering for task status, and returning tasks that match (for example, > FAILED tasks). > * task metric quantiles, add adding the task summary if specified > * executor metric quantiles, and adding the executor summary if specified -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org