[
https://issues.apache.org/jira/browse/HADOOP-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Lu updated HADOOP-6728:
----------------------------
Attachment: hadoop-6728-y20.104.patch
The patch is a combined (commons/mapreduce/hdfs) patch against
yahoo-hadoop-0.20.104 (published at http://github.com/yahoo/hadoop-common/).
The code should be quite stable (based on ongoing scale testing.)
The trunk patch will include significant API improvement with annotations (not
included in the above patch). The design notes have been updated to reflect the
current code:
[rev2|http://wiki.apache.org/hadoop/HADOOP-6728-MetricsV2?action=recall&rev=2]
Note, the patch completely removes the old metrics framework, which is
appropriate for our use cases (and forced porting of everything). The main
question for the community is: do we want to provide a mechanism to have the
two framework coexists in 0.22, where the metrics framework can be switched at
runtime or let's move everything to v2 in 0.22?
The answer to the question would alter the trunk patch significantly.
I prefer the latter as the former (dual framework switchable at runtime per
process) would cause more confusion to users (and more throwaway work for me.).
As an incentive for the latter, I offer to port ganglia 3.1+ plugin for the new
framework, even though we don't use it :)
> Overhaul metrics framework
> --------------------------
>
> Key: HADOOP-6728
> URL: https://issues.apache.org/jira/browse/HADOOP-6728
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.20.2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.22.0
>
> Attachments: hadoop-6728-y20.104.patch, metrics1-flow.png,
> metrics2-builder-r2.png, metrics2-builder.png, metrics2-flow.png,
> metrics2-mutable-r2.png, metrics2-mutable.png, metrics2-record-r2.png,
> metrics2-record.png, metrics2-uml-r2.png, metrics2-uml.png
>
>
> Per discussions with Arun, Chris, Hong and Rajiv, et al, we concluded that
> the current metrics framework needs an overhaul to:
> * Allow multiple plugins for different monitoring systems simultaneously (see
> also: HADOOP-6508).
> * Refresh metrics plugin config without server restart.
> ** Including filtering of metrics per plugin.
> * Support metrics schema for plugins.
> The jira will be resolved when core hadoop components (hdfs, mapreduce) are
> updated to use the new framework . Updates to external components that use
> the existing metrics framework will be tracked by different issues.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
