[
https://issues.apache.org/jira/browse/SPARK-36658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huangtengfei updated SPARK-36658:
---------------------------------
Description:
Now in
[QueryExecutionListener|https://github.com/apache/spark/blob/v3.2.0-rc2/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L38]
we have exposed API to get the query execution information:
def onSuccess(funcName: String, qe: QueryExecution, durationNs: Long): Unit
def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit
But we can not get a clear information that which query is this. In Spark SQL,
I think that executionId is the direct identifier of a query execution. So I
think it make sense to expose executionId to the QueryExecutionListener, so
that people can easily find the exact query in UI or history server to track
more information of the query execution. And there is no easy way we can find
the relevant executionId from a QueryExecution object.
was:
Now in
[QueryExecutionListener|https://github.com/apache/spark/blob/v3.2.0-rc2/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L38]
we have exposed API to get the query execution information:
def onSuccess(funcName: String, qe: QueryExecution, durationNs: Long): Unit
def onFailure(funcName: String, qe: QueryExecution, exception: Exception): Unit
But we can not get a clear information that which query is this. In Spark SQL,
I think that executionId is the direct identifier of a query execution. So I
think it make sense to expose executionId to the QueryExecutionListener, so
that people can easily find the exact query in UI or history server to track
more information of the query execution.
> Expose executionId to QueryExecutionListener
> --------------------------------------------
>
> Key: SPARK-36658
> URL: https://issues.apache.org/jira/browse/SPARK-36658
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.2
> Reporter: huangtengfei
> Priority: Minor
>
> Now in
> [QueryExecutionListener|https://github.com/apache/spark/blob/v3.2.0-rc2/sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala#L38]
> we have exposed API to get the query execution information:
> def onSuccess(funcName: String, qe: QueryExecution, durationNs: Long): Unit
> def onFailure(funcName: String, qe: QueryExecution, exception: Exception):
> Unit
>
> But we can not get a clear information that which query is this. In Spark
> SQL, I think that executionId is the direct identifier of a query execution.
> So I think it make sense to expose executionId to the QueryExecutionListener,
> so that people can easily find the exact query in UI or history server to
> track more information of the query execution. And there is no easy way we
> can find the relevant executionId from a QueryExecution object.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
