[
https://issues.apache.org/jira/browse/SPARK-27565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
peng bo updated SPARK-27565:
----------------------------
Description:
Currently it's really hard to link SQL plan to Spark jobs on SparkSQL UI page.
When one job failed, which {{PhysicalPlan}} nodes are involved are difficult to
be located. This PR is to show the job info of {{WholeStageCodegen}} node on
SparkSQL UI page to make life easier.
Each {{WholeStageCodegen}} node has a task metric {{PIPELINE_DURATION_METRIC}}
which records the total duration time shown in the top right. The idea is to
put the job info into its metric value during metric aggregation.
was:
Currently it's really hard to link SQL plan to Spark jobs on SparkSQL UI page.
When one job failed, which PhysicalPlan nodes are involved are difficult to be
located. This PR is to show the job info of WholeStageCodegen node on SparkSQL
UI page to make life easier.
Each WholeStageCodegen node has a task metric PIPELINE_DURATION_METRIC which
records the total duration time shown in the top right. The idea is to put the
job info into its metric value during metric aggregation.
> Show job info of WholeStageCodegen node on SparkSQL UI page
> -----------------------------------------------------------
>
> Key: SPARK-27565
> URL: https://issues.apache.org/jira/browse/SPARK-27565
> Project: Spark
> Issue Type: Improvement
> Components: SQL, Web UI
> Affects Versions: 3.0.0
> Reporter: peng bo
> Priority: Major
> Attachments: SPARK-27565.jpg
>
>
> Currently it's really hard to link SQL plan to Spark jobs on SparkSQL UI
> page. When one job failed, which {{PhysicalPlan}} nodes are involved are
> difficult to be located. This PR is to show the job info of
> {{WholeStageCodegen}} node on SparkSQL UI page to make life easier.
> Each {{WholeStageCodegen}} node has a task metric
> {{PIPELINE_DURATION_METRIC}} which records the total duration time shown in
> the top right. The idea is to put the job info into its metric value during
> metric aggregation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
