Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19981#discussion_r157135091
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala ---
@@ -82,6 +82,19 @@ private[sql] class SharedState(val sparkContext:
SparkContext) extends Logging {
*/
val cacheManager: CacheManager = new CacheManager
+ /**
+ * A status store to query SQL status/metrics of this Spark application,
based on SQL-specific
+ * [[org.apache.spark.scheduler.SparkListenerEvent]]s.
+ */
+ val statusStore: SQLAppStatusStore = {
--- End diff --
I just realized this is important. Previously we can use
`SharedState.listener` to query the SQL status. `SharedState` is actually a
semi-public interface, as it's marked as `Unstable` in
`SparkSession.sharedState`.
Sometimes for debugging I just type `spark.sharedState.listener.xxx` in
Spark Shell and check some status, but it's impossible now after #19681
There might be some other people like me that love this ability, we should
not just remove it for future UI discoverbility(I think it's just SQL and
streaming, really not a big gain)
cc @hvanhovell @viirya @gatorsmile
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
