[
https://issues.apache.org/jira/browse/SPARK-38222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17492927#comment-17492927
]
Apache Spark commented on SPARK-38222:
--------------------------------------
User 'erenavsarogullari' has created a pull request for this issue:
https://github.com/apache/spark/pull/35536
> Expose Node Description attribute in SQL Rest API
> -------------------------------------------------
>
> Key: SPARK-38222
> URL: https://issues.apache.org/jira/browse/SPARK-38222
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Eren Avsarogullari
> Priority: Major
>
> Currently, SQL public Rest API does not expose node description and it is
> useful to have nodeDesc attribute at query level to have more details such as:
> {code:java}
> - Join Operators(BHJ, SMJ, SHJ) => when correlating join operator with join
> type and which leg is built for BHJ.
> - HashAggregate => aggregated keys and agg functions
> - List can be extended for other physical operators.{code}
> *Current Sample Json Result:*
> {code:java}
> {
> "nodeId" : 14,
> "nodeName" : "BroadcastHashJoin",
> "wholeStageCodegenId" : 3,
> "stageIds" : [ 5 ],
> "metrics" : [ {
> "name" : "number of output rows",
> "value" : {
> "amount" : "2"
> }
> }
> },
> ...
> {
> "nodeId" : 8,
> "nodeName" : "HashAggregate",
> "wholeStageCodegenId" : 4,
> "stageIds" : [ 8 ],
> "metrics" : [ {
> "name" : "spill size",
> "value" : {
> "amount" : "0.0"
> }
> }
> } {code}
> *New* {*}Sample Json Result{*}{*}:{*}
> {code:java}
> {
> "nodeId" : 14,
> "nodeName" : "BroadcastHashJoin",
> "nodeDesc" : "BroadcastHashJoin [id#4], [id#24], Inner, BuildLeft, false",
> "wholeStageCodegenId" : 3,
> "stageIds" : [ 5 ],
> "metrics" : [ {
> "name" : "number of output rows",
> "value" : {
> "amount" : "2"
> }
> }
> },
> ...
> {
> "nodeId" : 8,
> "nodeName" : "HashAggregate",
> "nodeDesc" : "HashAggregate(keys=[name#5, age#6, salary#18],
> functions=[avg(cast(age#6 as bigint)), avg(salary#18)])",
> "wholeStageCodegenId" : 4,
> "stageIds" : [ 8 ],
> "metrics" : [ {
> "name" : "spill size",
> "value" : {
> "amount" : "0.0"
> }
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
