This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e86c499b008c [SPARK-47674][CORE] Enable
`spark.metrics.appStatusSource.enabled` by default
e86c499b008c is described below
commit e86c499b008ccc96b49f3fb9343ce67ff642c204
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Mon Apr 1 13:16:35 2024 -0700
[SPARK-47674][CORE] Enable `spark.metrics.appStatusSource.enabled` by
default
### What changes were proposed in this pull request?
This PR aims to enable `spark.metrics.appStatusSource.enabled` by default.
### Why are the changes needed?
`spark.metrics.appStatusSource.enabled` was introduced at `Apache Spark
3.0.0` and has been used usefully in the production in order to expose app
status. We had better enable it by default in `Apache Spark 4.0.0`.
### Does this PR introduce _any_ user-facing change?
This will expose additional metrics.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45798 from dongjoon-hyun/SPARK-47674.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
core/src/main/scala/org/apache/spark/internal/config/Status.scala | 2 +-
docs/monitoring.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/core/src/main/scala/org/apache/spark/internal/config/Status.scala
b/core/src/main/scala/org/apache/spark/internal/config/Status.scala
index 7f03f134d187..50dda81fc2e0 100644
--- a/core/src/main/scala/org/apache/spark/internal/config/Status.scala
+++ b/core/src/main/scala/org/apache/spark/internal/config/Status.scala
@@ -69,7 +69,7 @@ private[spark] object Status {
"will be reported for the status of the running spark app.")
.version("3.0.0")
.booleanConf
- .createWithDefault(false)
+ .createWithDefault(true)
val LIVE_UI_LOCAL_STORE_DIR = ConfigBuilder("spark.ui.store.path")
.doc("Local directory where to cache application information for live UI.
By default this is " +
diff --git a/docs/monitoring.md b/docs/monitoring.md
index 79bbb93e50d1..5dc470f1f7e0 100644
--- a/docs/monitoring.md
+++ b/docs/monitoring.md
@@ -1187,7 +1187,7 @@ This is the component with the largest amount of
instrumented metrics
- namespace=appStatus (all metrics of type=counter)
- **note:** Introduced in Spark 3.0. Conditional to a configuration
parameter:
- `spark.metrics.appStatusSource.enabled` (default is false)
+ `spark.metrics.appStatusSource.enabled` (default is true)
- stages.failedStages.count
- stages.skippedStages.count
- stages.completedStages.count
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
