The toString method of Dataset.queryExecution includes the various plans.
I usually just log that directly.
On Thu, Feb 16, 2017 at 8:26 AM, Muthu Jayakumar <bablo...@gmail.com> wrote:
> Hello there,
>
> I am trying to write to log-line a dataframe/dataset queryExecution and/or
> its logical plan. The current code...
>
> def explain(extended: Boolean): Unit = {
> val explain = ExplainCommand(queryExecution.logical, extended = extended)
>
> sparkSession.sessionState.executePlan(explain).executedPlan.executeCollect().foreach
> {
> // scalastyle:off println
> r => println(r.getString(0))
> // scalastyle:on println
> }
> }
>
> sessionState is not accessible if I were to write my own explain(log:
> LoggingAdapter).
>
> Please advice,
> Muthu
>
