Hi all
i want to ask a question is there any tool to visualize the spark physical
plan or spark plan? sometimes the physical plan is very long so it is difficult
to view it.
Best Regards
KellyZhang
Hi Kelly,
If you can parse event log, then try listening on
`SparkListenerSQLExecutionStart` event and build a `SparkPlanGraph` like
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala#L306
.
`SparkPlanGraph` has a `make
I feel a little pushed... :-) I still don't get the point of why it's
urgent to make the decision now. AFAIK, it's a common practice to handle
Scala types conversions by self when Java programmers prepare to
invoke Scala libraries. I'm not sure which one is the Java programmers'
root complaint, Sca
Does the Spark SQL web UI work for you?
https://spark.apache.org/docs/3.0.0-preview/web-ui.html#sql-tab
On Thu, Apr 30, 2020 at 5:30 PM Manu Zhang wrote:
> Hi Kelly,
>
> If you can parse event log, then try listening on
> `SparkListenerSQLExecutionStart` event and build a `SparkPlanGraph` like
>
Nothing is urgent. I just don't want to leave it undecided and just keep
adding Java APIs inconsistently as it's currently happening.
We should have a set of coherent APIs. It's very difficult to change APIs
once they are out in releases. I guess I have seen people here agree with
having a general
really thanks for your suggestion
At 2020-04-30 17:30:13, "Manu Zhang" wrote:
Hi Kelly,
If you can parse event log, then try listening on
`SparkListenerSQLExecutionStart` event and build a `SparkPlanGraph` like
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/
