Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/9571#discussion_r73624122
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -667,6 +700,90 @@ private[history] class FsHistoryProvider(conf:
SparkConf, clock: Clock)
prevFileSize < latest.fileSize
}
}
+
+ /**
+ * Metrics integration: the various counters of activity
+ * Time a closure, returning its output.
+ * @param timer timer
+ * @param fn function
+ * @tparam T type of return value of time
+ * @return the result of the function.
+ */
+ private def time[T](timer: Timer)(fn: => T): T = {
+ val timeCtx = timer.time()
+ try {
+ fn
+ } finally {
+ timeCtx.close()
+ }
+ }
+}
+
+/**
+ * Metrics integration: the various counters of activity.
+ */
+private[history] class FsHistoryProviderMetrics(owner: FsHistoryProvider)
extends Source {
+
+ override val sourceName = "history.fs"
+
+ private val name = MetricRegistry.name(sourceName)
+
+ /** Metrics registry. */
+ override val metricRegistry = new MetricRegistry()
+
+ /** Number of updates. */
+ val updateCount = new Counter()
+
+ /** Number of update failures. */
+ val updateFailureCount = new Counter()
+
+ /** Update duration timer. */
+ val updateTimer = new Timer()
+
+ /** Time the last update was attempted. */
+ val updateLastAttempted = new Timestamp()
+
+ /** Time the last update succeded. */
+ val updateLastSucceeded = new Timestamp()
+
+ /** Number of App UI load operations. */
+ val appUILoadCount = new Counter()
+
+ /** Number of App UI load operations that failed due to a
load/parse/replay problem. */
+ val appUILoadFailureCount = new Counter()
+
+ /** Number of App UI load operations that failed due to an unknown file.
*/
+ val appUILoadNotFoundCount = new Counter()
+
+ /** Statistics on time to load app UIs. */
+ val appUiLoadTimer = new Timer()
--- End diff --
I see you added the more useful `appUIEventReplayTime`, so is this metric
needed? I don't think a lot of useful information, if any, can be derived from
it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
