mohitgurav20 commented on issue #6463:
URL:
https://github.com/apache/datafusion-comet/issues/6463#issuecomment-5915088172
Hi @andygrove,
Thanks for filing this issue! I investigated the mechanics behind how Spark
constructs the UI plan graph and event logs.
### Root Cause Analysis
Spark builds `SparkPlanInfo` objects for the SQL UI tab, physical plan
trees, and event logs via `SparkPlanInfo.fromSparkPlan(plan: SparkPlan)`.
Inside Spark's `SparkPlanInfo.fromSparkPlan`, there is an explicit pattern
match for Spark's built-in `InMemoryTableScanExec`:
```scala
val children = plan match {
case inMemTab: InMemoryTableScanExec =>
Seq(fromSparkPlan(inMemTab.relation.cachedPlan))
case _ =>
plan.children.map(fromSparkPlan)
}
val innerChildren = plan.innerChildren.map(fromSparkPlan)
// returns SparkPlanInfo(..., children ++ innerChildren, ...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
