Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9571#discussion_r45874984
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -678,6 +709,69 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
method.invoke(dfs, action).asInstanceOf[Boolean]
}
+ /**
+ * Metrics integration: the various counters of activity
+ */
+ private[history] class HistoryMetrics extends Source {
+
+ /** Name for metrics: `history.fs` */
+ override val sourceName = "history.fs"
+
+ private val name = MetricRegistry.name(sourceName)
+
+ /** Metrics registry */
+ override val metricRegistry = new MetricRegistry()
+
+ /** Number of updates */
+ val updateCount = new Counter()
+
+ /** Number of update failures */
+ val updateFailureCount = new Counter()
+
+ /** Number of App UI load operations */
+ val appUILoadCount = new Counter()
+
+ /** Number of App UI load operations that failed */
+ val appUILoadFailureCount = new Counter()
+
+ /** Statistics on update duration */
+ val updateTimer = new Timer()
+
+ /** Statistics on time to load app UIs */
+ val appUiLoadTimer = new Timer()
+
+ private var metrics: Option[MetricRegistry] = None
+ private var health: Option[HealthCheckRegistry] = None
+
+ /**
+ * Register metrics.
+ */
+ def registerMetrics(metrics: MetricRegistry, health:
HealthCheckRegistry): Unit = {
+
+ this.metrics = Some(metrics)
+ this.health = Some(health)
+ def register(s: String, m: Metric) = {
+ metricRegistry.register(s, m)
+ }
+ register("update.count", updateCount)
--- End diff --
copy design in #6935 and use a list of (name, metric) both for registration
& generating toString value
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
