Steve Loughran created SPARK-11373:
--------------------------------------
Summary: Add metrics to the History Server and providers
Key: SPARK-11373
URL: https://issues.apache.org/jira/browse/SPARK-11373
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 1.5.1
Reporter: Steve Loughran
The History server doesn't publish metrics about JVM load or anything from the
history provider plugins. This means that performance problems from massive job
histories aren't visible to management tools, and nor are any
provider-generated metrics such as time to load histories, failed history
loads, the number of connectivity failures talking to remote services, etc.
If the history server set up a metrics registry and offered the option to
publish its metrics, then management tools could view this data.
# the metrics registry would need to be passed down to the instantiated
{{ApplicationHistoryProvider}}, in order for it to register its metrics.
# if the codahale metrics servlet were registered under a path such as
{{/metrics}}, the values would be visible as HTML and JSON, without the need
for management tools.
# Integration tests could also retrieve the JSON-formatted data and use it as
part of the test suites.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
