On Tue, Jan 19, 2016 at 5:18 PM Pawel Akonom <pawel.ako...@vnomic.com> wrote:
> Hi, > > I am trying to visualize some system metrics on grafana. Metrics are > sent from CollectD to Logstash, then from Logstash to Elasticsearch and > from Elasticsearch to Grafana. The problem is rx tx if_octets and > if_packets metric is growing all the time. I would like to see > differential metric on the graph - how many octets or packets was send > and received in last 10 seconds. > > I am using collectd 5.5 on CentOS 6.7 and I have compiled it from source > code. In /usr/share/collectd/types.db if_packets and if_octets are set > to DERIVE. As I understand from wiki page > https://collectd.org/wiki/index.php/Data_source this data should be sent > as a change from the last time: > > [root@hdptest-mster2 collectd-5.5.0]# grep if_packets > /usr/share/collectd/types.db > if_packets rx:DERIVE:0:U, tx:DERIVE:0:U > [root@hdptest-mster2 collectd-5.5.0]# grep if_octets > /usr/share/collectd/types.db > if_octets rx:DERIVE:0:U, tx:DERIVE:0:U > > Here is my configuration file: > > [root@hdptest-mster2 collectd-5.5.0]# egrep -v ^#.* /etc/collectd.conf | > sed '/^$/d' > Hostname "ACI_184_vora2_Master_MT2" > TypesDB "/usr/share/collectd/types.db" > LoadPlugin cpu > LoadPlugin df > LoadPlugin disk > LoadPlugin exec > LoadPlugin interface > LoadPlugin load > LoadPlugin memory > LoadPlugin network > <Plugin cpu> > ReportByCpu true > ReportByState true > ValuesPercentage true > </Plugin> > <Plugin df> > Device "/dev/sda1" > Device "/dev/sdb1" > FSType "ext4" > ValuesPercentage true > </Plugin> > <Plugin disk> > Disk "/^sd[a-b]?$/" > IgnoreSelected false > </Plugin> > <Plugin interface> > Interface "eth0" > IgnoreSelected false > </Plugin> > <Plugin memory> > ValuesAbsolute true > ValuesPercentage false > </Plugin> > <Plugin network> > <Server "192.168.112.228" "25826"> > </Server> > <Server "192.168.112.227" "25826"> > </Server> > </Plugin> > > This is a sample of raw data sent to logstash from this host: > > [root@hdptest-elk logstash]# tail -f logstash-rawdata.log | grep MT2 | > grep if_octets > > {"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:37:41.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4202999326,"tx":1272176172,"@version":"1","type":"collectd"} > > {"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:37:51.485Z","plugin":"interface","plugin_instance":"eth0","collectd_type":"if_octets","rx":4203048744,"tx":1272200296,"@version":"1","type":"collectd"} > > {"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:38:01.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4203131562,"tx":1272215389,"@version":"1","type":"collectd"} > > {"host":"ACI_184_vora2_Master_MT2","@timestamp":"2016-01-19T21:38:11.485Z","plugin":"interface","collectd_type":"if_octets","plugin_instance":"eth0","rx":4203152162,"tx":1272241466,"@version":"1","type":"collectd"} > > It looks like data are sent as a GAUGE value instead of DERIVE. What I > am doing wrong? How can I configure CollectD to send differential > statistics for the interfaces? > > Best Regards, > Pawel > > > What is your logstash configuration? I'm assuming that logstash is configured to listen on udp/25826 and you are using the network plugin as a write plugin to this port? I don't have the answer to your question, but I have a similar setup and I see the same issue, where grafana renders the datapoints as a counter, rather than deltas between successive datapoints. >From what I understand about collectd, it will cache values in the global cache (https://collectd.org/wiki/index.php/Global_cache) where DERIVE values are converted to a rate. The interface plugins collects values from /proc/net/dev and sends them to the network plugin on the client as is. It is here, I believe, that the network plugin sends the values to the logstash listener, if that is how you have it setup, again with the values intact and not as a rate. This is why grafana renders them as such. In a typical setup, many clients will use the network plugin to send plugin datapoints to a server instance running the network plugin in Listening mode. From there, an output plugin is configured to send to somewhere else, like write_graphite, for example. If you are familiar with graphite, the datapoints first go to carbon. If you run ngrep on the carbon port, you would see that the values for if_octets and if_packets are the rate values (I assume from the global cache) and not the raw DERIVE values that continue to grow. It's almost like the network plugin, when configured to Listen, should have an option like StoreRates=True, like some of the other plugins, but I don't think the network plugin was intended for this. Giovanni
_______________________________________________ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
