[
https://issues.apache.org/jira/browse/EAGLE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JiJun Tang resolved EAGLE-928.
------------------------------
Resolution: Resolved
> Refine system metric schema design and fix system metric collector
> ------------------------------------------------------------------
>
> Key: EAGLE-928
> URL: https://issues.apache.org/jira/browse/EAGLE-928
> Project: Eagle
> Issue Type: Bug
> Affects Versions: v0.5.0
> Reporter: Hao Chen
> Assignee: Hao Chen
> Fix For: v0.5.0
>
>
> Refine system metric schema design and fix system metric collector
> h1. Principle
> Any metric (no mater sys metric) should at least have three fields: "metric",
> "group", "timestamp", "value",
> "group" is the metric category path separated with slash, for example "FIRST
> CATEGORY.SECOND CATEGORY", which will be commonly used for well organizing
> metric groups in customized metric dashboard.
> h1. Stream Name for System Metric
> {code}
> <stream>
> <streamId>SYSTEM_METRIC_STREAM</streamId>
> <description>System Metrics Stream including CPU, Network, Disk,
> etc.</description>
> <columns>
> <column>
> <name>host</name>
> <type>string</type>
> </column>
> <column>
> <name>timestamp</name>
> <type>long</type>
> </column>
> <column>
> <name>metric</name>
> <type>string</type>
> </column>
> <column>
> <name>group</name>
> <type>string</type>
> </column>
> <column>
> <name>site</name>
> <type>string</type>
> </column>
> <column>
> <name>device</name>
> <type>string</type>
> </column>
> <column>
> <name>value</name>
> <type>double</type>
> <defaultValue>0.0</defaultValue>
> </column>
> </columns>
> </stream>
> {code}
> h1. Stream Schema for System metric
>
> * *metric*: [STRING] metric name string
> * *group*: [STRING] metric group/type
> * *timestamp*: [LONG] metric generation time
> * *site*: [STRING] siteId
> * *host*: [STRING] source host name
> * *device*: [STRING] device name, like cpu
> h1. Sample CPU Metric
> {code}
> {
> "timestamp": 1487918913569,
> "metric": "system.cpu.usage",
> "group": "system.cpu",
> "site": "sandbox",
> "value": 0.058,
> "host": "sandbox.hortonworks.com",
> "device": "cpu7"
> }
> {code}
> h1. Sample Network Metrics
> {code}
> {
> "timestamp": 1487918913569,
> "metric": "system.nic.transmitdrop",
> "group": "system.network",
> "site": "sandbox",
> "value": 7724.0,
> "host": "sandbox.hortonworks.com",
> "device": "eth0"
> }
> {code}
> h1. Sample Metric Schema
> {code}
> {
> "tags": {
> "site": "sandbox",
> "name": "system.memory.nfs_unstable.kb",
> "group": "system.memory"
> },
> "dimensionFields": [
> "host",
> "group",
> "site",
> "device"
> ],
> "metricFields": [
> "value"
> ],
> "granularity": "MINUTE",
> "modifiedTimestamp": 1488190388479
> }
> {code}
> h1. Stream Persist Topology
> {code}
> environment.newApp(config)
> .fromStream("HADOOP_JMX_METRIC_STREAM")
> .saveAsMetric(
> MetricDescriptor.metricGroupAs((MetricGroupSelector)
> event -> {
> if (event.containsKey("component")) {
> return String.format("hadoop.%s", ((String)
> event.get("component")).toLowerCase());
> } else {
> return "hadoop.metrics";
> }
> })
> .siteByField("site")
> .namedByField("metric")
> .eventTimeByField("timestamp")
> .dimensionFields("host", "component", "site")
> .granularity(Calendar.MINUTE)
> .valueField("value"))
> .fromStream("SYSTEM_METRIC_STREAM")
> .saveAsMetric(MetricDescriptor.metricGroupByField("group")
> .siteByField("site")
> .namedByField("metric")
> .eventTimeByField("timestamp")
> .dimensionFields("host", "group", "site", "device")
> .granularity(Calendar.MINUTE)
> .valueField("value")
> )
> .toTopology();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
