[
https://issues.apache.org/jira/browse/SPARK-38322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
XiDuo You updated SPARK-38322:
------------------------------
Description:
The formatted explalin mode is the powerful explain mode to show the details of
query plan. In AQE, the query stage know its statistics if has already
materialized. So it can help to quick check the conversion of plan, e.g. join
selection.
A simple example:
{code:java}
SELECT * FROM t JOIN t2 ON t.c = t2.c;{code}
{code:java}
== Physical Plan ==
AdaptiveSparkPlan (21)
+- == Final Plan ==
* SortMergeJoin Inner (13)
:- * Sort (6)
: +- AQEShuffleRead (5)
: +- ShuffleQueryStage (4), Statistics(sizeInBytes=16.0 B, rowCount=1)
: +- Exchange (3)
: +- * Filter (2)
: +- Scan hive default.t (1)
+- * Sort (12)
+- AQEShuffleRead (11)
+- ShuffleQueryStage (10), Statistics(sizeInBytes=16.0 B, rowCount=1)
+- Exchange (9)
+- * Filter (8)
+- Scan hive default.t2 (7)
+- == Initial Plan ==
SortMergeJoin Inner (20)
:- Sort (16)
: +- Exchange (15)
: +- Filter (14)
: +- Scan hive default.t (1)
+- Sort (19)
+- Exchange (18)
+- Filter (17)
+- Scan hive default.t2 (7){code}
was:
The formatted explalin mode is the powerful explain mode to show the details of
query plan. In AQE, the query stage know its statistics if has already
materialized. So it can help to quick check the conversion of plan, e.g. join
selection.
A simple example:
{code:java}
SELECT * FROM t JOIN t2 ON t.c = t2.c;{code}
{code:java}
== Physical Plan ==
AdaptiveSparkPlan (21)
+- == Final Plan ==
* SortMergeJoin Inner (13)
:- * Sort (6)
: +- AQEShuffleRead (5)
: +- ShuffleQueryStage (4), Statistics(sizeInBytes=16.0 B, rowCount=1)
: +- Exchange (3)
: +- * Filter (2)
: +- Scan hive default.t (1)
+- * Sort (12)
+- AQEShuffleRead (11)
+- ShuffleQueryStage (10), Statistics(sizeInBytes=16.0 B, rowCount=1)
+- Exchange (9)
+- * Filter (8)
+- Scan hive default.t2 (7)
+- == Initial Plan ==
SortMergeJoin Inner (20)
:- Sort (16)
: +- Exchange (15)
: +- Filter (14)
: +- Scan hive default.t (1)
+- Sort (19)
+- Exchange (18)
+- Filter (17)
+- Scan hive default.t2 (7){code}
> Support query stage show runtime statistics in formatted explain mode
> ---------------------------------------------------------------------
>
> Key: SPARK-38322
> URL: https://issues.apache.org/jira/browse/SPARK-38322
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: XiDuo You
> Priority: Major
>
> The formatted explalin mode is the powerful explain mode to show the details
> of query plan. In AQE, the query stage know its statistics if has already
> materialized. So it can help to quick check the conversion of plan, e.g. join
> selection.
> A simple example:
> {code:java}
> SELECT * FROM t JOIN t2 ON t.c = t2.c;{code}
>
> {code:java}
> == Physical Plan ==
> AdaptiveSparkPlan (21)
> +- == Final Plan ==
> * SortMergeJoin Inner (13)
> :- * Sort (6)
> : +- AQEShuffleRead (5)
> : +- ShuffleQueryStage (4), Statistics(sizeInBytes=16.0 B, rowCount=1)
> : +- Exchange (3)
> : +- * Filter (2)
> : +- Scan hive default.t (1)
> +- * Sort (12)
> +- AQEShuffleRead (11)
> +- ShuffleQueryStage (10), Statistics(sizeInBytes=16.0 B, rowCount=1)
> +- Exchange (9)
> +- * Filter (8)
> +- Scan hive default.t2 (7)
> +- == Initial Plan ==
> SortMergeJoin Inner (20)
> :- Sort (16)
> : +- Exchange (15)
> : +- Filter (14)
> : +- Scan hive default.t (1)
> +- Sort (19)
> +- Exchange (18)
> +- Filter (17)
> +- Scan hive default.t2 (7){code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
