[ https://issues.apache.org/jira/browse/EAGLE-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887398#comment-15887398 ]
ASF GitHub Bot commented on EAGLE-928: -------------------------------------- Github user asfgit closed the pull request at: https://github.com/apache/eagle/pull/842 > Refine system metric schema design and fix system metric collector > ------------------------------------------------------------------ > > Key: EAGLE-928 > URL: https://issues.apache.org/jira/browse/EAGLE-928 > Project: Eagle > Issue Type: Bug > Affects Versions: v0.5.0 > Reporter: Hao Chen > Assignee: Hao Chen > Fix For: v0.5.0 > > > Refine system metric schema design and fix system metric collector > h1. Principle > Any metric (no mater sys metric) should at least have three fields: "metric", > "group", "timestamp", "value", > "group" is the metric category path separated with slash, for example "FIRST > CATEGORY.SECOND CATEGORY", which will be commonly used for well organizing > metric groups in customized metric dashboard. > h1. Stream Name for System Metric > {code} > <stream> > <streamId>SYSTEM_METRIC_STREAM</streamId> > <description>System Metrics Stream including CPU, Network, Disk, > etc.</description> > <columns> > <column> > <name>host</name> > <type>string</type> > </column> > <column> > <name>timestamp</name> > <type>long</type> > </column> > <column> > <name>metric</name> > <type>string</type> > </column> > <column> > <name>group</name> > <type>string</type> > </column> > <column> > <name>site</name> > <type>string</type> > </column> > <column> > <name>device</name> > <type>string</type> > </column> > <column> > <name>value</name> > <type>double</type> > <defaultValue>0.0</defaultValue> > </column> > </columns> > </stream> > {code} > h1. Stream Schema for System metric > > * *metric*: [STRING] metric name string > * *group*: [STRING] metric group/type > * *timestamp*: [LONG] metric generation time > * *site*: [STRING] siteId > * *host*: [STRING] source host name > * *device*: [STRING] device name, like cpu > h1. Sample CPU Metric > {code} > { > "timestamp": 1487918913569, > "metric": "system.cpu.usage", > "group": "system.cpu", > "site": "sandbox", > "value": 0.058, > "host": "sandbox.hortonworks.com", > "device": "cpu7" > } > {code} > h1. Sample Network Metrics > {code} > { > "timestamp": 1487918913569, > "metric": "system.nic.transmitdrop", > "group": "system.network", > "site": "sandbox", > "value": 7724.0, > "host": "sandbox.hortonworks.com", > "device": "eth0" > } > {code} > h1. Sample Metric Schema > {code} > { > "tags": { > "site": "sandbox", > "name": "system.memory.nfs_unstable.kb", > "group": "system.memory" > }, > "dimensionFields": [ > "host", > "group", > "site", > "device" > ], > "metricFields": [ > "value" > ], > "granularity": "MINUTE", > "modifiedTimestamp": 1488190388479 > } > {code} > h1. Stream Persist Topology > {code} > environment.newApp(config) > .fromStream("HADOOP_JMX_METRIC_STREAM") > .saveAsMetric( > MetricDescriptor.metricGroupAs((MetricGroupSelector) > event -> { > if (event.containsKey("component")) { > return String.format("hadoop.%s", ((String) > event.get("component")).toLowerCase()); > } else { > return "hadoop.metrics"; > } > }) > .siteByField("site") > .namedByField("metric") > .eventTimeByField("timestamp") > .dimensionFields("host", "component", "site") > .granularity(Calendar.MINUTE) > .valueField("value")) > .fromStream("SYSTEM_METRIC_STREAM") > .saveAsMetric(MetricDescriptor.metricGroupByField("group") > .siteByField("site") > .namedByField("metric") > .eventTimeByField("timestamp") > .dimensionFields("host", "group", "site", "device") > .granularity(Calendar.MINUTE) > .valueField("value") > ) > .toTopology(); > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)