[ https://issues.apache.org/jira/browse/SPARK-38222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-38222: ------------------------------------ Assignee: (was: Apache Spark) > Expose Node Description attribute in SQL Rest API > ------------------------------------------------- > > Key: SPARK-38222 > URL: https://issues.apache.org/jira/browse/SPARK-38222 > Project: Spark > Issue Type: Task > Components: SQL > Affects Versions: 3.2.0 > Reporter: Eren Avsarogullari > Priority: Major > > Currently, SQL public Rest API does not expose node description and it is > useful to have nodeDesc attribute at query level to have more details such as: > {code:java} > - Join Operators(BHJ, SMJ, SHJ) => when correlating join operator with join > type and which leg is built for BHJ. > - HashAggregate => aggregated keys and agg functions > - List can be extended for other physical operators.{code} > *Current Sample Json Result:* > {code:java} > { > "nodeId" : 14, > "nodeName" : "BroadcastHashJoin", > "wholeStageCodegenId" : 3, > "stageIds" : [ 5 ], > "metrics" : [ { > "name" : "number of output rows", > "value" : { > "amount" : "2" > } > } > }, > ... > { > "nodeId" : 8, > "nodeName" : "HashAggregate", > "wholeStageCodegenId" : 4, > "stageIds" : [ 8 ], > "metrics" : [ { > "name" : "spill size", > "value" : { > "amount" : "0.0" > } > } > } {code} > *New* {*}Sample Json Result{*}{*}:{*} > {code:java} > { > "nodeId" : 14, > "nodeName" : "BroadcastHashJoin", > "nodeDesc" : "BroadcastHashJoin [id#4], [id#24], Inner, BuildLeft, false", > "wholeStageCodegenId" : 3, > "stageIds" : [ 5 ], > "metrics" : [ { > "name" : "number of output rows", > "value" : { > "amount" : "2" > } > } > }, > ... > { > "nodeId" : 8, > "nodeName" : "HashAggregate", > "nodeDesc" : "HashAggregate(keys=[name#5, age#6, salary#18], > functions=[avg(cast(age#6 as bigint)), avg(salary#18)])", > "wholeStageCodegenId" : 4, > "stageIds" : [ 8 ], > "metrics" : [ { > "name" : "spill size", > "value" : { > "amount" : "0.0" > } > } > } {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org