[
https://issues.apache.org/jira/browse/FLUME-748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263154#comment-13263154
]
E. Sammer commented on FLUME-748:
---------------------------------
I'd prefer not to use code from Hadoop directly, but something like that, yes.
Generally, I was thinking of:
* Create an interface called MetricAware (or Reportable, or something) with a
method collectMetrics(MetricContext).
* Sinks, sources, channels, formatters, etc. could implement this and populate
MetricContext when called.
* A MetricCollector or something like that would hold a list of MetricAware
instances and poll them regularly.
* Plugins (e.g. JMX mbean, Ganglia, REST) would get a MetricCollector reference
injected when they start up and would be able to poll it or interrogate it as
needed.
Actors:
* MetricContext - A map-like data object that holds named metrics. Maybe it has
nice features to track counters, as well as simple values (e.g. Float, Long).
We probably need MetricContext to support the notion of groups of related
metrics. Maybe it's just a naming convention.
* MetricAware - Interface implemented by components that support exposing
metrics.
* MetricCollector - Internal service that polls a list of MetricAware at some
interval and creates an aggregate MetricContext with all metrics.
Gluing it together:
FlumeNode creates a MetricCollector on startup and injects it into each of the
factories (ChannelFactory, SourceFactory, ...). When an instance is created, if
it implements MetricAware, start tracking it. No singletons, no need to create
instances.
Important: The MetricCollector isn't aware of any particular protocol. It's up
to a plugin to expose metrics in some meaningful way. We could ship a JMX,
SNMP, REST, Ganglia, or logger-only plugin that writes metrics to a separate
log4j category. The critical part is to decouple metric collection from how
they're exposed. I expect advanced users to write their own plugins to expose
metrics how they want, but they shouldn't need to worry about threading /
collecting them internally.
There's already a CounterGroup in the codebase that looks a lot like
MetricContext. You may be able to refactor it and use it for this purpose.
> Create metric collection infrastructure
> ---------------------------------------
>
> Key: FLUME-748
> URL: https://issues.apache.org/jira/browse/FLUME-748
> Project: Flume
> Issue Type: Sub-task
> Components: Build, Docs, Master, Node, Shell, Sinks+Sources,
> Technical Debt, Test, Web
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Fix For: v1.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
