Re: [collectd] Curl Header
Hello, Poil writes: > It does not seem possible to pass a custom header in the curl plugins > documentation (Like a header: host) . > > But when I check the source code, what I understand is it seems > possible, anyone can confirm or not ? (and have an example ?) I just double-checked on collectd 5.6, but as far as I can tell, this feature works, for all 3 curl, curl_json and curl_xml plugins. Also, the documentation/examples are correct. This feature has been introduced in version 5.3. Are you running a recent enough version ? Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Version 5.6.2 available.
Florian Forster writes: > Download > > > The new version is available in source-code form from collectd's > download page. The direct download links are: > > * http://collectd.org/files/collectd-5.5.2.tar.bz2 > SHA-256: cc0b4118a91e5369409ced22d1d8a85c1a400098419414160c1839268ecad0c6 NB: the link should be http://collectd.org/files/collectd-5.6.2.tar.bz2 The sha256sum is valid for collectd-5.6.2.tar.bz2 and obviously wrong for collectd-5.5.2.tar.bz2 Thanks ! Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Multiple prefixes for graphite at different resolutions.
Jerry Lombardo writes: > Hi, > > > I'm using collectD to poll servers at a 5 minute interval and pushing it > out using write graphite. > > > I'd like to keep that 5 minute polling going but then run a second > temporary poll at say a 30 sec interval and push that to graphite under a > different prefix. The purpose of this is higher resolution metrics for > troubleshooting or load testing events. > > > What is the best way to accomplish this? You'll need to run 2 concurrent collectd instances, with different Interval and Prefix (write_graphite) settings. Another option is to set different aggregation policies in carbon's storage-schemas.conf and define 2 write_graphite blocks, pointing to the same server but with a different Prefix. HTH, Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd inside a container monitoring the host
Maurizio Vitale writes: > There's another way that consists in LD_PRELOADing a shared object that > intercepts all calls to fopen and prepend a prefix to filenames. I like this approach, and actually have implemented such a wrapper and have been using it for a couple of weeks now. So far so good. See https://github.com/collectd/collectd/pull/2041 This way, no need to modify or rebuild collectd itself. It should work with any existing or future version, with out-of-tree plugins, and even with other tools than collectd ;-) Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] write_http plugin - dynamic tag
Not sure if this answers your question, but starting with 5.6, the
following config should allow you to add arbitry key-values to outgoing
metrics:
URL "http://localhost:4567/collectd-post";
Format "json"
PostCacheChain "PostCache"
Plugin "^cpu$"
MetaData "foo" "bar"
Target "write"
Running with the config above, values such as this get POSTed by
collectd's write_http plugin:
{"values":[136247],"dstypes":["derive"],"dsnames":["value"],"time":1478019107.077,"interval":1.000,"host":"localhost","plugin":"cpu","plugin_instance":"2","type":"cpu","type_instance":"idle","meta":{"foo":"bar"}},
HTH,
Marc
Shirly Radco writes:
> Would really appreciate if you can elaborate how...
>
> Thank you in advance,
>
> Shirly Radco
>
> BI Software Engineer
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109
>
>
> On Thu, Oct 13, 2016 at 1:36 PM, Aurelien ROUGEMONT
> wrote:
>
>> On 10/13/2016 12:29 PM, Shirly Radco wrote:
>> > Hi,
>> >
>> > Is there an option to dynamically generate the tag in the collectd
>> > write_http plugin?
>> > (Use the record values to add as tag)
>> >
>> > Thank you,
>> >
>> > Shirly Radco
>> >
>> > BI Software Engineer Red Hat Israel Ltd. 34 Jerusalem Road Building A,
>> > 4th floor Ra'anana, Israel 4350109
>> >
>> >
>> >
>> > ___
>> > collectd mailing list
>> > collectd@verplant.org
>> > https://mailman.verplant.org/listinfo/collectd
>> >
>> Hello Shirly,
>>
>> I've done just that in the kairosdb format for write_http
>>
>> Aurelien
>>
>> --
>> Aurelien ROUGEMONT
>> be...@binaries.fr
>>
>>
>> ___
>> collectd mailing list
>> collectd@verplant.org
>> https://mailman.verplant.org/listinfo/collectd
>>
> ___
> collectd mailing list
> collectd@verplant.org
> https://mailman.verplant.org/listinfo/collectd
___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Broken Ubuntu repositories
Hello, Yevgeny Zegzhda writes: > Hello, > we use pkg.ci.collectd.org repositories on Ubuntu 12.04 and 14.04. > Last two week repositories stopped working. > apt can't get packages list. Yes, the nightly builds have been broken these past days. I've just added the missing bits and the CI system is currently rebuilding new packages. The repository should be fit for use again within a couple of hours. Thanks for reporting the problem ! Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Documentation to write Plugins
On Tue, Sep 13, 2016 at 11:25:35AM +0200, Thijs Cramer wrote: > Nice! That helps a lot! > Is there any guide on how to compile/test properly? I couldn't find > any guide on how the .c code is included. Something along these lines *should* work: gcc -DHAVE_CONFIG_H -Wall -Werror -g -O2 -shared -fPIC -Isrc/ -Isrc/daemon/ -ldl -o myplugin.so contrib/examples/myplugin.c Although it seems there are a couple of issues with this myplugin.c example (include paths are incorrect to start with). > Can i presume that the make script will automatically compile all .c > files in the /src directory and include them in the CollectD builtin > plugins? > (as in, there is no hard linking of plugin source code files) For the "automatic" part to happen, the plugin needs to get declared in configure.ac and src/Makefile.am, then the build.sh script ran. This will generate the configure script, which itself will generate the Makefiles. Take a look at a recent pull-request adding a new plugin on github. https://github.com/collectd/collectd/pull/1805/files is a good example. HTH, Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Collectd plugin zfs_arc
Frederic Daguenet writes: > Hi Emma, > > i saw your answer very late, because i received no verplant-mails for a long > time by an mail error. :( > > I named the wrong os. The right is Debian Jessie 8.0. I installed the > following zfs and > collectd packets: > > debian-zfs libzfs2 zfs-dkms zfsutils > collectd collectd-core > > But no zfs_arc plugin exists. Must i install an additional package? > Or what is wrong? ZFS-on-Linux support was added in collectd 5.5. Debian jessie comes with 5.4.1. But the jessie-backports repo has the collectd version you need. HTH, Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] Documentation to write Plugins
Thijs Cramer writes: > Hi all, > > Is there any documentation / quickstart guide available on how to > develop Plugins? > I've been unable to find any. You might also want to take a look at the plugin skeleton: https://github.com/collectd/collectd/blob/master/contrib/examples/myplugin.c Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] parse errors on predefined postgresql plugin queries
Hello, Aaron Vinson writes: > I'm attempting to use collectd postgresql plugin to monitor a Postgres > 9.5.2 RDS instance. I've created a user with the correct permissions and > tested the queries manually. The predefined "transactions" query works as > expected but many of the others such as "queries" and "query_plans" result > in the following error: This same issue got mentioned on the IRC channel a couple of days ago, so I opened https://github.com/collectd/collectd/issues/1905 to keep track of it. Please feel free to chime in if you have any updates. Thanks for reporting this issue ! Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd inside a container monitoring the host
Maurizio Vitale writes: > Hi all, > is there an easy way for running collectd inside a docker container while > still monitoring the host? > For a number of plugin of interest to me it is only a matter of mounting > /proc to /host_proc inside the container and then use that. Other plugins > might be trickier. > But even for the simple ones, one might have to change the source code. > > Has anybody done this successfully? I started working on (but didn't really finish yet): https://github.com/collectd/collectd/pull/1867 > [the reason for this, rather than running collectd as a normal process on > every host, is that I need to have it running in kubernetes and having > non-contenairazed processes is not nice] Yes, this was also my use-case. Cheers, Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd 5.6 release schedule
Florian Forster writes: Hello, > * August 15: 5.6 will enter a "feature freeze": we're going to create > the "collectd-5.6" branch based on master. From that point onwards, > only bug fixes will to into this branch. > * after August 15: one or more "release candidates" will be available as > release tarballs. > * September 11: 5.6.0 will be tagged and the final release tarballs will > be published. Since the "collectd-5.6" branch has now materialized, the nightly builds have been updated to also generate RPM and DEB packages from this branch. So testing upcoming 5.6 version should be pretty easy for CentOS, Debian and Ubuntu users (instructions at: https://github.com/collectd/collectd-ci/blob/master/README.md) Marc ___ collectd mailing list collectd@verplant.org https://mailman.verplant.org/listinfo/collectd
Re: [collectd] how to set a threshold using postgresql plugin
Gerardo Herzig writes: > Hi all. Im sending collectd values from a postgres DB to our collectd central > server. > In DB, there is using postgres plugin like: > > > Statement "select sum(numbackends) as backends, sum(tup_inserted) as > inserts, sum(tup_updated) as updates, sum(tup_deleted) as deletes > from pg_stat_database;" > > Type gauge > InstancePrefix total_backends > ValuesFrom "backends" > > > > Interval 10 > Host localhost > User postgres > Query db_movements > > > And, in the "central" collectd server i want to set a threshold for that > server, so > " If that db server is over 100 connections in use, ALERT " > > At that point, i got confused about how to get it work. > I have something like that: > > > > > FailureMax 100.0 > > > > > > It throws no error, but even if i put a FailureMax of 2 or so, alert is not > raising. > So, anyone can help me to know to set that alert? I don't think the above specification will match the value you're trying to track. Try something like this: Instance "template1" Instance "total_backends" FailureMax 100.0 You might want to enable the unixsock plugin and run "collectdctl listval" to make sure your specification actually matches an existing element name. Also, take a look at https://collectd.org/wiki/index.php/Naming_schema Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Collecting stats from a tab delimited file
CJ Ess writes: > I'm looking to start collecting data from a log file with tab > delimited fields. Sometimes I just need to capture the nth field, in > some cases I need to sum several fields before passing them to > collectd. It doesn't appear that I can do either thing with the Tail > plug-in. Before I go and write something to do this, I'm wondering if > anyone knows of something similar? The tail_csv plugin does that, but unfortunately it has the comma character as the separator hardcoded. Modifying the plugin to use tab should be pretty straightforward though. Bonus points for making this character configurable and sharing the patch ;-) For summing fields, assuming you base on tail_csv for metrics collection, you can imagine using the aggregation plugin. HTH, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Exec script
Collectd User writes: >> On 24. feb. 2016, at 21.51, Marc Fournier >> wrote: >> >> Collectd User mailto:collectd@siimnet.dk>> >> writes: >> >>>> On 15. feb. 2016, at 10.55, Collectd User wrote: >>>> >>>> Fellow Collectors, >>>> >>>> Trying my first exec plugin script, but it’s not working 100%. >>>> >>>> I’ve added this type in /usr/share/collectd/types.db on the end point >>>> collecting the data: >>>> >>>> ssdhealth health:GAUGE:0:100.1, hours:GAUGE:0:4294967295 >>>> >>>> and submits data on stdout like this: >>>> >>>> PUTVAL d1r1n1/ssdhealth-1I:1:3/ssdhealth 1452173939:100.00:2227 >>>> >>>> but after coming through my logstash and into elasticsearch, I’m missing >>>> health and hours fields in stored ES documents and wondering why only >>>> these two rather important fileds are missing… >>>> >>>> any hints on howto debug this appreciated, TIA! >>> Any one? >> >> Mmmh, not sure exactly how the collectd protocol decoder works on the >> logstash side, but I could imagine it needs the matching types in >> types.db. Did you try adding your "ssdhealth" type to logstash's >> types.db ? >> >> Other than that, maybe try enabling the cvs plugin to make sure >> everything works fine on the collectd side. > Thanks for hinted, started a new collectd with a different config file with > just my exec collector and output directly to csv an dit shows values fine :) > Will try to dig further into logstash… This recent post on the logstash forum seems to be the problem you're having, doesn't it ? https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727 Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] modbus plugin spamming console with debug info
Hello Eric ! Eric Sandeen writes: > I hadn't noticed this before, but after running on a systemd system it > comes to the forefront: > >> Message from syslogd@sandeen at Feb 24 22:39:48 ... >> collectd:<21><00><00><00><05><01><04><02><00><02> > >> Broadcast message from systemd-journ...@sandeen.net (Wed 2016-02-24 22:39:48 >> CST): > >> collectd[22528]: <22><00><00><00><05><01><04><02><00><19> > > on every modbus communication. ;) > > The plugin sets modbus_set_debug(ctx, 1); - that seems a bit odd to do > unconditionally. > > Has anyone else run into this? Setting to debug should almost certainly be > optional, > AFAICT. Thanks for pointing this out ! Here's a patch which should prevent this: https://github.com/collectd/collectd/pull/1561 Does it sound an acceptable solution ? Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Exec script
Collectd User writes: >> On 15. feb. 2016, at 10.55, Collectd User wrote: >> >> Fellow Collectors, >> >> Trying my first exec plugin script, but it’s not working 100%. >> >> I’ve added this type in /usr/share/collectd/types.db on the end point >> collecting the data: >> >> ssdhealth health:GAUGE:0:100.1, hours:GAUGE:0:4294967295 >> >> and submits data on stdout like this: >> >> PUTVAL d1r1n1/ssdhealth-1I:1:3/ssdhealth 1452173939:100.00:2227 >> >> but after coming through my logstash and into elasticsearch, I’m missing >> health and hours fields in stored ES documents and wondering why only these >> two rather important fileds are missing… >> >> any hints on howto debug this appreciated, TIA! > Any one? Mmmh, not sure exactly how the collectd protocol decoder works on the logstash side, but I could imagine it needs the matching types in types.db. Did you try adding your "ssdhealth" type to logstash's types.db ? Other than that, maybe try enabling the cvs plugin to make sure everything works fine on the collectd side. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] AMQP Subscription
Theo Bot writes: Hello, > I have a collectd server setup on a linux platform having an influxdb as a > beackend. Is it possible to receive values via AMQP (RabbitMQ). And if so > what would be the message format to process these values and what are the > other requirements? Yes, the amqp plugin supports both subscribing and publishing to an AMQP broker. I never used this feature myself, but my understanding is that you need to publish messages using the PUTVAL format (see the collectd-unixsock(5) manpage) and set "text/collectd" as "Content-Type". HTH, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Configure reporting interval on network plugin
Excerpts from Joel Serrano's message of 2016-02-15 20:41:58 -0800: > > Has anyone successfully configured the network plugin in collectd to send > metrics every X seconds? (like 10 seconds). You can't control how frequently collected data is flushed to the write target(s). Only how frequently read targets are probed. In the case of the network plugin, the data gets flushed once its buffer is full, which size defaults to 1.5kB. Are you collecting very little data with a relatively high read interval ? If not you might want to fire tcpdump to check you aren't loosing packets (remember the protocol is UDP-based). Note that the values dispatched include the collection time timestamp. So even if there is some delay before they reach their final destination, they should be stored with the correct time reference. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Problem using the write_sensu plugin
Excerpts from Joshua J. Kugler's message of 2016-02-03 15:27:33 -0900:
> [...]
> Metrics false
> Notifications true
> NotificationHandler "pagerduty"
> NotificationHandler "irc"
collectd dispatches 2 type of events:
- metrics ("value of %free memory is N" every few seconds)
- notifications ("warning: %free memory is below N")
Some write plugins only handle metrics (rrdtool), others only handle
notifications (notify_email). A 3rd category handle both (write_riemann,
write_sensu). Hence the Metrics, Notifications, MetricHandler and
NotificationHandler options to the write_sensu plugin.
According to your config, you've disabled metrics dispatching. The
easiest way to make sure the collectd->sensu link is working would be to
flip the Metrics boolean and add a MetricHandler. Or alternatively,
change thresholds so that notifications get raised. See
collectd-threshold(5).
If you don't want to store the metrics collected at all, you can use a
PostCacheChain to send them all to /dev/null. This should stop the "No
write callback has been registered" log message:
PostCacheChain "PostCache"
Target "stop"
But I agree with you, the plugin could be slightly improved to make it
emit a couple of log messages during initalisation. It isn't obvious to
figure that out currently.
HTH,
Marc
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd on Windows
Excerpts from François-Xavier Choinière's message of 2016-01-19 00:08:05 +: > Hi, > I manage a network with hybrid OS, Linux and ... Windows :-(. > I tried sscserv but the support is not good and it's not open source. > I've tried to compile collectd on Windows with cygwin and the core seems to > compile. > Any of you think it's a good idea to compile the core on Windows and add a > specific plugin for Windows. > I can put some time on it. Like that you could have an official Windows > port. I'm aware the Stackdriver folks have done some work in this direction. Unfortunately it didn't get to the point of a formal review/merge request. Maybe would you like to get in touch with them and take over this project ? Take a look at https://github.com/kele/collectd/branches/all and https://github.com/kele/collectd/pulls Have fun :-) Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd-5.5.0 symbol lookup error: interface.so: undefined symbol: ignorelist_create
Excerpts from Matt.Garrett's message of 2015-11-13 14:02:42 +: > Folks > > OS is RedHat 5.10 been using collectd 4.10.9 without any issues > Looking at collectd-5.5.0 , seems to compile fine but on running I get > error's around the Network plugins > > i.e > collectd: symbol lookup error: > /opt/collectd/lib/collectd/interface.so: undefined symbol: > ignorelist_create > > If I remove Plugin Interface and Plugin Network then all is fine (of > course no data is going anywhere so not very usfull) Actually now that Giovanni mentions it, this reminds me of https://github.com/collectd/collectd/issues/1323 (which is fixed and will be part of 5.5.1). That said, it's surprising you are getting this error. Do you have 4.9.10 installed alongside 5.5.0 ? Could you trying to load a 5.5.0 plugin into the 4.9.10 daemon or the other way round ? Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd calculations on multiple metrics?
Excerpts from Matt Gravlin's message of 2015-08-03 16:20:25 -0700: > Is there a way to pre-calculate metrics using basic math before > sending them to an output plugin such as Graphite? For example, I > would like to calculate memory as a percent when the metrics gathered > only provide free and used. I can do this using Graphite functions, > but I would prefer to just send a calculated metric from collectd > rather than using Graphite formulas. Not much, apart from what you can do using the "aggregation" plugin. That said, are you aware that since version 5.5 a bunch of plugins come with a ValuesPercentage config option doing exactly what you want ? Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Problem configuring multiple KVM hosts with plugin libvirt
Excerpts from Martin Chamambo's message of 2015-07-03 06:16:58 +: > I am having issues with configuring multiple KVM hosts to be monitored > by the virt plugin on collectd. I have tried the following configs > below > > NB: if I configure just one connection URL of just one server ,im able > to see traffic for the different VMs but when I add more servers ,then > the last connection URL will work Yes, I don't think the virt plugin is able to gather data from more than one libvirtd instance. A typical usage pattern which would get you out of this situation, is to run the collectd agent on each hypervisor, and have them all ship their metrics to a central server for archiving & graphing. Hope that helps ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] how to build a debian package of a daily snapshot?
Excerpts from Svante Karlsson's message of 2015-05-11 23:55:46 +0200: > I finally managed to build both ubuntu 12 & 14 packages based on pkg-debian > - if I skipped the patches and relaxed the dependency check on my home > built packages. > > I also had to add packages libvirt-bin dpatch to be able to build. > > Finally I had to build protobuf and protobuf-c for both platforms (and > librdkafka for ubuntu 12) Great :-) > If/when you start building the packages do you plan to provide them for > download somewhere? That would be awesome! I'm still messing around with this, but yes, packages are already built (hopefully) each time new patches are commited. See the README of https://github.com/collectd/collectd-ci for the "where". Once the last details are sorted out, I'll send a more formal email to the mainling-list with all the details. In the meantime, fell free to give these packages a try. Feedback is of course welcome :) Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] how to build a debian package of a daily snapshot?
Hello Svante, I still have a couple of details to sort out so nothing "official" yet, but I've been working on making nightly builds of collectd for debian, ubuntu LTS and centos. So if you feel like giving this a try, recent packages for your target distros are ready for consumption. Details here: https://github.com/collectd/collectd-ci By the way, an important fix to write_kafka has been merged this WE, so I recommend using the current tip of the master branch if you need this plugin. Anwsering your questions regarding building your own packages: - libprotobuf-c-dev has been renamed to libprotobuf-c0-dev in ubuntu 12.04 and 14.04. You should edit debian/control to reflect this change. - https://github.com/collectd/pkg-debian.git doesn't know yet about current collectd master (which has write_kafka). Have a look at https://github.com/mfournier/pkg-debian/branches/all for the differences which allow the nightly builds to work. - ubuntu 12.04 doesn't provide a package for librdkafka, so you'll need to backport it or install the library manually. The nightly builds I mentioned above don't provide write_kafka on ubuntu 12.04 because of this. Hope that helps ! Marc Excerpts from Svante Karlsson's message of 2015-05-10 23:00:06 +0200: > I'm struggling to build a deb package for internal distribution on our > servers. I need to have kafka-writer plugin so I think I need to build from > a snapshot. > > this is what I'm doing > > sudo apt-get update > sudo apt-get upgrade > sudo apt-get install liboping-dev libmysqlclient-dev libpq-dev > librabbitmq-dev libvirt-bin librdkafka-dev > sudo apt-get install po-debconf dpatch iptables-dev javahelper > libcurl4-gnutls-dev libcurl4-gnutls-dev libcurl3-gnutls-dev libdbi0-dev > libesmtp-dev > sudo apt-get install libganglia1-dev libglib2.0-dev libhiredis-dev > liblvm2-dev libmemcached-dev libmodbus-dev libmnl-dev > sudo apt-get install libmysqlclient-dev libnotify-dev libopenipmi-dev > libpcap0.8-dev > sudo apt-get install libperl-dev libprotobuf-c0-dev librabbitmq-dev > librrd-dev libsensors4-dev libsnmp-dev perl > sudo apt-get install libtokyocabinet-dev libtokyotyrant-dev > libupsclient-dev libvarnish-dev libvirt-dev libyajl-dev > sudo apt-get install default-jdk protobuf-c-compiler libow-dev libprotobuf > > > git clone https://github.com/collectd/pkg-debian.git > wget > http://snapshots.tokkee.org/collectd/2015-03-17/collectd-5.4.2.694.g9817e72.tar.gz > -Ocollectd-5.4.2.694.g9817e72.tar.gz > tar xvzf collectd-5.4.2.694.g9817e72.tar.gz -C pkg-debian > --strip-components=1 > cd pkg-debian > dpkg-buildpackage -us -uc > > > I've tried to build and install > git clone https://github.com/google/protobuf.git > git clone https://github.com/protobuf-c/protobuf-c.git > > but can't find dependency libprotobuf-c-dev > > right now I'm running ubuntu 14.04 but I also need to get this working on > 12.04. I realize that I'm doing something wrong - could you please give me > a hint on how to produce the deb package? > > thanks in advance > svante ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] python plugin.
Excerpts from Dan Dong's message of 2015-03-10 16:19:06 +0100: > Hi, All, > Could anyone hint me how to install the python plugin of collectd? I > downloaded the collectd-5.4.2.tar.gz but after: > $./configure --prefix=/opt/collectd --enable-python > $sudo make all install > > Where can I find the python plugin? Is it supposed to be under: > /opt/collectd/lib/collectd/plugins/python/ ? > > But nothing there under /opt/collectd/lib/collectd/plugins/ You should be looking for /opt/collectd/lib/collectd/python.so. After running ./configure, a summary is printed out which should tell you which plugins won't be built and why not. Maybe the configure script can't figure out where python is located on your system ? Try adding --with-python=/path/to/python to ./configure in this case. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] observe idle state of home server with collectd?
Excerpts from Thomas Koch's message of 2015-02-10 20:48:55 +0100: > first asked here: http://stackoverflow.com/questions/28399754 > > I've setup my home server to wake-on-lan and my router sends the wake-on-lan > package when I try to access the server from anywhere. > > Now the server should also suspend again after like 5 minutes after > > - last disk activity > - last network activity > - and last considerable CPU load > > I thought I could write a collectd configuration with lower bound thresholds > for cpu load, disk io and network traffic. If this config detects an idle > state > for 5 consecutive minutes than it notifies "pm-suspend". > > Do you think this might be possible? Do you have some hints for the config? > I've not used collectd before. This should be doable. Basically, you'll want to use the threshold plugin and configure it to watch the metrics which indicate low usage. This is described in collectd-threshold(5). When a threshold is met, collectd will emit a notification event, which you can intercept in a custom script using the exec plugin's NotificationExec parameter. collectd-exec(5) has all the details. Have fun ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] [PATCH] modbus: rename register types once more
Excerpts from Eric Sandeen's message of 2015-01-28 05:36:05 +0100: > As I was looking over my config files again, ModbusRegisterType > kind of stuck out like a sore thumb. > > Modbus does talk about "read holding" and "read input" as "commands"; > perhaps renaming it as this patch does is more intuitive, so that all > of the Modbus register configurations start with Register. > > I'm not wedded to it, but if you like it & agree, here you go. Thanks a lot Eric! As no one seems to have stepped up against this small naming change, I've merged your patch to master as e6553128. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] [Patch] Netlink plugin: query 64bit counters where possible
Hello Bruno, Thanks a lot for your patch ! I wanted to give it a try, but then noticed someone else implemented the same feature in a different way. His patch was merged to the master branch (which means it will be in version 5.5). Here is his patchset: https://github.com/collectd/collectd/pull/435 Cheers, Marc Excerpts from Bruno Prémont's message of 2014-11-15 22:27:19 +0100: > When 64bit counters can be queried via netlink, query those instead > of short 32bit counters. > [...] ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] [PATCH V2] add Modbus/RTU support to modbus plugin
Excerpts from Eric Sandeen's message of 2014-12-10 22:38:09 +0100: > This allows access to a local RS-485 serial port > via the modbus plugin > [...] Fabien Wernli was kind enough to test your patch on his modbus equipment to make sure there was no side effect on the existing modbus-over-TCP functionnality. So I'm pleased to inform you your patch has been included and will be part of upcoming 5.5 release. Thanks a lot Eric ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] ampq output plugin SSL support
Excerpts from Pallavi Kalapatapu (pkalapat)'s message of 2015-01-13 08:47:28 +0100: > Hi, > > The ampq output plugin seems to list only port 5672 as referred by > this link https://collectd.org/wiki/index.php/Plugin:AMQP. How can I > get it to support SSL over 5671 port ? SSL support doesn't seem to have been implemented yet unfortunately... https://github.com/collectd/collectd/blob/master/src/amqp.c#L428-L429 Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] collectd not logging
Excerpts from Tom Throckmorton's message of 2014-10-28 02:34:32 +0100: > On Mon, Oct 27, 2014 at 6:41 PM, Alison Chan wrote: > > > Hello, > > I'm developing a few new collectd plugins so I am interested in > > debug-level logs. However, when I configure logging no file is > > created. What should I do to solve this problem? > > > > From > http://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_logfile > : > > "Please note that *debug* is only available if collectd has been compiled > with debugging support." > > It's probably not enabled in the build you're using. You can verify that > logging generally works by switching to LogLevel info. > > The solution would be to rebuild and --enable-debug. FYI, upcoming collectd 5.5 will fallback to "info" log level in this case. Avoiding this confusing situation where nothing gets logged at all. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] flapjack integration
Excerpts from Oğuz Yarımtepe's message of 2014-10-15 10:12:52 +0200: > > I am also planning to combine health check mechanism with collectd and > write custom > plugins for checking whether a required service is working or not. Any one > of you did this with collectd or any suggestions to how to do it? Collectd's threshold plugin has various ways to send out notifications when collected values go out of range or stop receiving updates. See: https://collectd.org/documentation/manpages/collectd-threshold.5.shtml Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] relocation error at startup of collectd [Varnish V4.0.2]
Excerpts from John Platter's message of 2014-10-19 11:59:03 +0200: > I have a working collectd environment but as soon as I enable & > configure this plugin then the collectd daemon stops and generates an > error as follows. Please help. If you require more info then please > let me know. Yes, it's a known problem with varnish 4. You can keep track of it and add your comments, if you have some, to this issue: https://github.com/collectd/collectd/issues/618 Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] bind plugin failing
Excerpts from Chip Marshall's message of 2014-09-26 20:26:00 +0200: > I'm having some trouble with the bind plugin, and I could use > some guidance in tracking down what the problem is. At the > moment, I'm just getting "read-function of plugin `bind' failed." > in the log, followed by various "Will suspend it for" times. > [...] > I'm using collectd 5.4.1 on FreeBSD 10.0-RELEASE with BIND 9.10.1. I believe your issue is adressed by this patch: https://github.com/collectd/collectd/pull/742 It will be part of collectd 5.5, but can probably easily be applied to the 5.4 branch if you'd like to benefit from it immediatly. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] ldap plugin for collectd ?
Hello Kimo, Excerpts from Kimo Rosenbaum's message of 2014-06-04 17:56:13 +0200: > > A few years ago I started on an openldap plugin: > > http://mailman.verplant.org/pipermail/collectd/2011-January/004336.html > (although this seems to have truncated the email) > > Here is the original email and patch I sent: > > https://gist.github.com/kimor79/684e2ef4f48037e9 I just wanted to inform you I made a bunch of improvements to your openldap plugin recently: https://github.com/mfournier/collectd/compare/kimor79:kimor79/openldap...openldap-improvements If by any chance you still use it, I'd be interested by your feedback ! Andrew: if you're still planning to use this plugin, I was able to build it against an existing installation of collectd using: gcc -DHAVE_CONFIG_H -Wall -Werror -g -O2 -shared -fPIC \ -I/usr/include/collectd/core/ -ldl -lldap -o openldap.so openldap.c (after installing the openldap-devel/libldap2-dev package) My plan is to test this on a bunch of servers for a bit, then once I'm happy with it, rebase the branch against current master and (re)submit it for inclusion. Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] generating composite metrics
Hello, Excerpts from Gerardo Herzig's message of 2014-06-25 18:14:10 +0200: > Hi all. Im wondering if it is possible to generate a metric as a > calculation between others. > > Example I: Know the rate between the "used memory" and the "total > memory". Example II: Know the rate between "In use" apache threads > against "total" apache threads. As a side note, the master branch has a patch which allows the memory plugin to emit percentages instead of absolute values. Maybe this could solve example 1 in a way ? > I know how to make a graph as this sort of calculations, but i also > need to generate an alert when, say, 75% of the apache's threads are > in use, so the graphical solutions does not cover all my needs. > > So i guess the correct solution is to generate my own .rrd files > according to existing .rrd files. I just dont have any idea how. I like using riemann for this sort of use-cases (http://riemann.io). Basically, you just use collectd to collect metrics from your systems and send them to a riemann server using the write_riemann plugin. Then do the calculations/aggregations/correlations (and optionnally, the alerting based on the result) in riemann. If you're tied to having the resulting values in RRD files (which riemann doesn't know about), you can run riemann embedded in collectd using the java language binding plugin. This way you can do the filering and calculations you want using riemann, and have the results passed down to collectd's rrdtool/rrdcached plugin. Search for "collmann" on github for the details. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] varnish plugin, not found
Excerpts from L Cruzero's message of 2014-05-23 15:38:06 +0200: > Hi, i've been using collectd for a few months to gather based system stats, > recently i've tried unsuccessfully to use the varnish plugin. > > on start: > > stdout: > > "Starting collectd: Could not find plugin varnish" > [...] > collectd-5.1.0-1.el6.rft.x86_64 > > > varnish-release-3.0-1.el6.noarch As neither of these packages are part of rhel6, I guess you have enabled additional repositories from where you installed them. It's important that you install the same libvarnishapi version that was used to build the collectd plugin (it will typically be found in the same repo). If you don't find the exact match of varnish + collectd versions you wish in a repository out there, you'll have to build collectd yourself. There's a rpm specfile for el5 & el6 in contrib/redhat/, in collectd's source tree. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Which plug-ins support collection interval on a per-plugin basis feature introduced from 5.2
Hello, Excerpts from andrewbooty .'s message of 2014-05-09 23:22:22 +0200: > I've been testing the 5.4.1 release of collectd and assumed that most core > plug-ins now supported the Interval feature to override the global setting. > For ethstat, interface and table this hasn't been implemented yet. So > wondering if there is a list of which plug-ins do currently support their > own interval settings. I could check the source code to see if Interval is > included in the callback function. But if Interval support was included in > say the wiki plugin table that would be helpful. *All* the plugins support custom intervals since 5.2.0. This is how you have to load a plugin with a non-default interval: Interval 3600 What is a bit confusing, is that some plugins implement this or some similar behaviour themselves for various reasons. https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_netapp is an example of such a plugin. So writing something like this *could* work for some plugins, but is not the correct way of using this feature: LoadPlugin df Interval 3600 Could there be some misunderstanding on your side ? Or do you have reasons to believe there's something wrong with the ethstat, interface and table plugins and not the others ? Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin : Objects LRU nuked are not reported
Hello, Excerpts from smainklh's message of 2014-04-04 15:49:16 +0200: > Hello, > > I'm currently benching a varnish server and there's an essential value i need > to monitor : LRU Nuked. > Actually all other metrics are well gathered by collectd but for this one i > always get a 0. > However when i look to the varnishstat the value increases. > > varnishstat -1 | grep n_lru_nuked > n_lru_nuked 680 . N LRU nuked objects > > Do you know how to solve this issue please ? What version of collectd & varnish are you running ? I just checked and don't have this problem with 5.4.1 and varnish 3.0.2. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Ceph plugin for collectd
Hello Dan ! Thanks for following up ! Excerpts from Dan Ryder (daryder)'s message of 2014-03-14 19:17:42 +0100: > Hi, > > I was looking through the collectd mailer-list archives and read through > some discussion about a ceph plugin for collectd. > > It doesn't seem like this code ever got added to the official collectd > code. Is there a reason for this? I saw > https://github.com/ceph/collectd-4.10.1 on github and used the ceph > plugin here to get a working version. > > I forked the main collectd code and modified the ceph plugin to fix a few > things. > > Is there a way I can commit these plugins to the official collectd code? Last time ceph was mentioned on the list, the question was if using libyajl (which the curl_json plugin uses) over libjson (that the ceph plugin uses) was possible. The question remains open AFAIK. From the maintenance and packaging point of view, it's really better not to depend on different libraries providing the same features. On the other hand, if libjson is considered better than libyajl, why not consider changing curl_json to use it ? In both cases, the amount of code actually using libjson/libyajl doesn't seem to be huge. Do you think this is something that could be done ? BTW, I'm going to close the current pull-request for this plugin (https://github.com/collectd/collectd/pull/522) because it seems to have went out of control: 26 files changed, 9018 insertions(+), 7955 deletions(-) If you have a clean working branch with the ceph plugin in it's current state, please submit it on github so people can test it, and maybe take over the supporting lib switch if you don't. Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
[collectd] help using write_redis
Hello, Is someone successfully using the write_redis plugin ? I can't make any sense of the data stored in redis by this plugin. A snippet of code (in any language) that handles the data pulled out of redis would be a great help. Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] curl_jason plugin is not sending data to rabbitMQ server
Excerpts from Mariano González's message of 2014-02-10 16:59:25 +0100: > Hi Marc! > > Here's my request/answer using curl -v: > > $ curl -v -H "X-Auth-Token: my_token" https://rackspace_api.url/current | > [...] > > GET /v1.0/account_id/loadbalancers/lb_id/usage/current HTTP/1.1 > > User-Agent: curl/7.33.0 > > Host: rackspace_api.url > > Accept: */* > > X-Auth-Token: my_token So here are the exact HTTP headers the curl CLI is sending to the server ^^ > But, I got a doubt in the next item, now I've this set up in collectd.conf > > > https://rackspace_api.url/usage/current";> >Instance "Load_Balancer" > Header "X-Auth-Token: my_token" And here is the only header we are sure collectd is sending ^^ My suggestion was to try to add the other headers to see if this can help. My first bet would be on "Accept: */*". I had a similar problem in the past with curl_json on a jetty-based service, but just guessing... Hope this helps ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] curl_jason plugin is not sending data to rabbitMQ server
Excerpts from Mariano González's message of 2014-02-06 13:32:56 +0100: > Hi Marc. > > The request I run manually is: > > curl -H "X-Auth-Token: my_token" > https://API_url/1.0/account_id/loadbalancers/lb_id/usage/current | python > -m json.tool > > And collectd uses the curl_json plugin, which I don't know what it requests > (I assume it uses curl, but I'm not sure how). > > Could you guide me on how to tcdump this connections please? Actually, ngrep is a similar tool which output is easier to read when you're only interested in the "application layer". Try: sudo ngrep -d any -W byline port 80 I see you're using https though. If you can't switch to plain http, you can probably forget about dumping the network traffic. "curl -v" will show you what headers are sent in the HTTP(s) request. Then try to use collectd's "Header" option to the curl_json plugin and make it match the headers sent by the curl CLI tool. As the curl CLI tool works, I'm pretty sure what you are attempting is feasible. It's just a matter of finding the correct parameters. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Add debug symbols to compiled collectd binary
Excerpts from Christopher Fitzner's message of 2014-01-28 10:33:38 +0100: > Hey, > > can anyone give me any tips on compiling the collectd binary with debug > symbols and then use gdb to load it for plugin debugging? I've run the > configure script with these options: CFLAGS="-g -O0" as suggested on the > homepage but when I try to run the compiled binary It looks like some of > the data is in .libs/collectdS.o which is deleted near the end of the make > process. > > Messing around I made a copy of the file before it was deleted to load up > however even with that file I'm having trouble get gdb to stop anywhere > useful to me. I try to set a break point at one of my functions and it > says theres no function with that name. Doesn't "./configure --enable-debug [...]" do the job ? Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] curl_jason plugin is not sending data to rabbitMQ server
Excerpts from Mariano González's message of 2014-01-31 15:01:13 +0100: > Hi there. > I've implemented curl_jason plugin to recolect and send LoadBalancer > metrics to my RabbitMQ to be graphed in Graphite. > Thing is, it's not sending any data, while it is working just fine (and > great) with other plugins like memory, cpu, df root, network, etc. I've > tried to troubleshoot following this suggestion: > http://serverfault.com/questions/499378/collectd-stores-nan-instead-of-correct-value-in-ubuntu-12-04, > but there're no issues coming out. > > Here's my collectd.conf: https://gist.github.com/Mariano-gon/8732467 > > Here're the last lines of collectd.log when I start it: > https://gist.github.com/Mariano-gon/8732488 > > The request is made against Rackspace API where my LoadBalancer is located, > and if run manually, the curl gets me a json response perfectly normal. Just a random guess: does the rackspace API require you to send an "Accept" encoding header ? By default collectd doesn't send any. Try tcpdumping the HTTP traffic and compare what headers collectd sends vs when you run curl manually. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] How to contribute my code ?
Hello, Excerpts from Dennis Zou (yunzou)'s message of 2014-01-17 10:36:47 +0100: > Hi, > Recently I developed a new plugin, but I'm not so clear about how to > commit my code? > Anyone can point me to the right direction or give me some clues? The usual way nowadays is to fork the project on github and send a pull-request against the "master" branch: https://github.com/collectd/collectd/ If you're not familiar with git/github, just send your patch to the mailing-list for review. Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Problem with java plugin
Excerpts from Paulo Silva's message of 2014-01-11 11:38:58 +0100: > Hi, > > I think I've found the problem. > Nice catch ! > [...] > > So it seams that when I load the java plugin some of the other plugins > start using my locale setting to display numbers (I'm Portuguese and we use > "," instead of "." as a decimal mark) so I fixed the problem running this > before starting collectd: > > unset LC_NUMERIC LANG > > Should I open a bug report for this? Yes please: https://github.com/collectd/collectd/issues Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] write_http behaviour on POST failure?
Hello Otis, Excerpts from Otis Gospodnetic's message of 2013-12-12 17:56:59 +0100: > Thanks Mark! > > Would it make sense to improve that plugin and let it fall-back to writing > to disk, say by using one of the csv output plugins, in case of failure? > Even if nothing in Collectd itself is able to pick up that data and re-send > it at a later time, it would allow others to put things like file tailing > agents there that, should any data be written to disk, could pick it up and > resend it. I'm not sure "fallback to writing to $somewhere_safer" would fit very well in the collectd model. What I would do in your place is enable > 1 write plugins to maximise the chances at least one of them is able to save the data somewhere. The CSV plugin is a dead simple one which would fit quite well in this case I think. NB: not all write plugins are prefixed with "write_" for historical reasons. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] write_http behaviour on POST failure?
Hello, Excerpts from Otis Gospodnetic's message of 2013-12-12 04:41:46 +0100: > Hi, > > What does https://collectd.org/wiki/index.php/Plugin:Write_HTTP do when the > destination (there data is being POSTed) is either not up or returns a > non-200 or times out or ... ? > > Is the data dropped and lost? > Or is it written somewhere to be sent later? It will be lost. An error will be emitted if you have logging enabled. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Logging to files instead of RRD
Hello, Excerpts from Igor Kryltsov's message of 2013-11-30 18:52:24 +0100: > Hi, > > I can not find anywhere a way to write data collected by collectd to > files to be able to pickup them with logstash and send to > elasticsearch. > > Did I miss it and it is possible? If not do you have such plans? It > makes sense since log files and system stats can be shipped in this > case to a central location with logstash/elasticsearch and logstash > can output to graphite. The first idea that comes to mind would be to use collectd'd "csv" plugin, and logstash's "file" input plugin on these CSV files + some basic grok to parse the lines. But you might be more interested by this new plugin (not part of any release yet) which implements collectd's network protocol in logstash: https://github.com/logstash/logstash/blob/master/lib/logstash/inputs/collectd.rb Hope this helps ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] time series data in between specific time periods.
Excerpts from Kaushal Shriyan's message of 2013-11-18 14:16:41 +0100: > Hi, > > I have setup collectd on CentOS 6.4. Are there options to select time range > for example 15/05/2013 to 23/05/2013? I mean to look for time series data > in between this period This really depends on what storage backend you are saving your data into. If you are running a recent version of collectd, there are at least a dozen output plugins, several of which have the time-range feature you're looking for: graphite, postgresql, mongodb, etc. Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Bundling plugins (e.g. Plugin:Redis)
Excerpts from Otis Gospodnetic's message of 2013-11-01 21:11:38 +0100: > Hi, > > I'm trying to trace https://collectd.org/wiki/index.php/Plugin:Redis . > This plugin doesn't appear to be included in any Collectd package out > there. But some others, like mysql or ngingx, seem to be included. > > In case of https://collectd.org/wiki/index.php/Plugin:Redis - I > actually don't even see where to go to get this plugin, if it's > available somewhere else, where one would grab it and compile it on > their own. Yes, the redis plugin is notoriously annoying to build, as no distro (that I'm aware of) ships the required credis library. So it's usually disabled by default in packages available out there. If you're going down the path of building collectd yourself, you might want to try merging this patch in: https://github.com/collectd/collectd/pull/464 It changes the library supporting the redis plugin to the more popular hiredis, which is usually available with yum/apt-get/whatever install. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] symbol not found when building
Excerpts from Khushil Dep's message of 2013-08-13 17:58:19 +0200: > I get the following message in the configure output when trying to build: > > libstatgrab . . . . . no (symbol sg_init not found) Are you building for linux ? On this OS, libstatgrab is not required to run collectd. > [root@05e534b1-9281-44b2-9734-974da75e775e ~/collectd-5.3.1]# nm -D > /opt/local/lib/libstatgrab.so.9.0.0 | grep sg_ For building against libstatgrab, you'll need a file named libstatgrab.a, which typically comes in the linux distros in a package named libstatgrab-devel or libstatgrab-dev. Hope this helps ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Java API for collectd
Excerpts from Vedran Bartonicek's message of 2013-08-06 21:10:48 +0200:
> If I remember right, there was configure option "--enable-java" which has
> to be used as well together with "--with-java".
> -Vedran
This is indeed what is done in the RPM specfile in contrib/
configure is called with:
--enable-java --with-java=%{java_home}/
(on RHEL6 the %{java_home} macro expands to /usr/lib/jvm/java, which is a
symlink to /etc/alternatives/java_sdk, which itself points to
/usr/lib/jvm/java-1.6.0-openjdk.x86_64)
Also, the following packages are required for building this plugin:
java-devel jpackage-utils
Marc
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Collecd ping syslog
Excerpts from DeCamps, Miguel's message of 2013-07-23 22:12:12 +0200: > Hello everyone > > I am trying to figure out how I can get success pings from the ping > plugin in the collectd agent to the local syslog. I can clearly see when > a ping does not succeed a log entry in /var/log/syslog; but upon recovery > of the pings I do not see a log entry into my syslog file. > > I need this so I can integrate it into my OMD instance for remote agents > I have distributed. Any help would be greatful! There isn't any notion of failed/recovered "state". If a ping fails, a log message is emitted. If it succeeds, nothing is logged. That's all. I don't know about how OMD works, but would I typically check the values emitted by the ping plugin (from one of the available write plugins, or using collectd's threshold system) and raise an alarm if the value is above/below a certain level. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] processes plugin process by name - percent of all cpus feature
Hello, Excerpts from William Salt's message of 2013-06-17 02:00:35 +0200: > Im using the processes plugin to find specific processes via regex, and > monitoring their cpu usage, this is the total time spent in jiffies > (milliseconds) across all cores. > > In graphite, i am dividing it by the number of cpu cores i have on a > machine then turning that into percent. this gives me a percentage of total > cpu usage across all cores for that process. > However, i cannot ascertain in graphite the exact number of cpus a node > has, it differs greatly, and its not scalable to have static values in > graph query strings like this, nor efficient to work this out when > rendering a graph. Still not very nice and efficient, but you can use a variant of this[1] to count the number of cpus using graphite functions. IE: this would return the number of CPUs as seen by collectd's spu plugin: sumSeries(offset(scale(collectd.your_hostname.cpu-*.cpu-idle, 0), 1)) Another thing that comes to mind: since collectd 5.3 there's an "aggregation" plugin, which you can use to sum up the metrics of every cpu on each host. You could then use these aggregated metrics for your percentage calculation. There's a similar feature in graphite if you run the carbon-aggregator in front of carbon-cache. > I wondered if someone could patch the plugin to allow a flag to produce the > average cpu use across all cores? grabbing the number of cpus > from /proc/cpuinfo and dividing the metric by that? bonus if it had a flag > to output a percent! Vedran Bartonicek has 2 pull-requests doing this sort of thing for the load and df plugins. Have look at PRs #343 and #344 on github.com/collectd/collectd Marc [1] http://obfuscurity.com/2013/05/Graphite-Tip-Counting-Number-of-Metrics-Reported ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] volume plugin
Hello, Excerpts from Chad Malfait's message of 2013-03-31 23:48:03 +0200: > > I was looking for a plugin to gather volume (lvm) metrics, and couldn't > seem to find one. > > I have since written one using lvm2app and it appears to work fine. I am > basically just gathering the volumegroup name and available free space in > the volume. I was thinking this may be something that i could push > upstream for the community to take a look at? Oh, great idea ! I could definitely make use of such a plugin :-) > I was curious if someone could point me in the right direction on how to > contribute to collectd? If you have a github account, fork https://github.com/collectd/collectd, commit your patch in a topic branch based on the master branch, and send a pull request. You can also just send your patch to the mailing-list using git-format-patch and git-send-email. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] collect 4 compilation problem - yajl
Hello, Excerpts from Pavel Kasparek's message of 2013-03-04 14:56:56 +0100: > Hello all, > > I'm trying to compile collect 4 but got the following error: > > action_instance_data_json.c:93: undefined reference to `yajl_gen_alloc2' > > Full error is listed below. > The system is Linux CentOS 6, with the following installed: > yajl-1.0.7-3.el6.x86_64 > yajl-devel-1.0.7-3.el6.x86_64 > > The version of collect 4 is 4.0.0. This is a very old version of collectd, released in 2007. Have you a good reason not to use a more recent version ? The plugins depending on yajl build well on CentOS6 with the latest 4.x as well as the 5.x series of collectd. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] libstatgrab required for collectd (and -devel to build)
Hello, Excerpts from Bruno Mairlot's message of 2013-01-08 14:37:55 +0100: > > I noticed today that with the .spec file currently available, the > buildrequires needs libstatgrab-devel and the .rpm needs libstatgrab > packages, although this package is not available from CentOS or RedHat > base repository. You need to activate EPEL repository to get it. > > Although it is not a real hindrance, is it possible to build the package > without libstatgrab in order to get a clean 5.2.0 package without > resorting to EPEL or is it some kind of core features ? I didn't see it > as a plugin, but is required for the main package. As far as I can tell, libstatgrab is optional, no plugin has a hard requirement on this library. Referring to the README, it's used "by various plugins to collect statistics on systems other than Linux and/or Solaris." The debian package for 5.2 isn't built against libstatgrab and all the plugins mentioning this lib in their source code are part of the package and are working as expected. I would be surprised if it was any different on RHEL. I'm going to update PR#220 accordingly ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Patch : contrib/redhat/collectd.spec
Hello Bruno, Excerpts from Bruno Mairlot's message of 2013-01-02 15:29:23 +0100: > > You'll find bellow a patch to apply on the collectd.spec file of the > original archive 5.2.0 > > I have worked on a new .spec file to allow for building RPMs for > CentOS/RHEL, it should probably work on Fedora too, although I haven't > tested (yet). > > I tried to build it with as much plugin possible, but I limited them to > those that could be built with the standard package from RedHat-Base (= > CentOS-Base) in order to keep it the most lightweight possible. It could > eventually be adjusted to allow for more flexibility or create multiple > subpackage with the specific plugin but I don't know enough about the > the plugins hierarchy to do that right now. There has been a common effort going forward for a couple of weeks about this RPM specfile: https://github.com/collectd/collectd/pull/220 AFAICT everything you included in your patch is already part of this pull-request. I warmly invite you to join the effort, test this specfile on your platform/environment and give feedback if it doesn't match your needs. Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Filter chains and GenericJMX plugin. Help Needed
Hello, Excerpts from Josh Behrends's message of Fri Jun 22 22:52:33 +0200 2012: > Hello, > > I'm looking for some help! I'm using Collectd 5.1.0 to collect JMX metrics > using the GenericJMX plugin and then sending the metrics over to Graphite. > The issue I'm having is I need to rename the metric "Type" that's being > sent to graphite. The format Collectd is currently sending: > .GenericJMX... The "Type" > [...] I have no experience with collectd's Filter mechanism, but I've done something similar to what you describe using carbon's rewrite feature. Have a look at conf/rewrite-rules.conf.example in your carbon installation. NB: you'll need to have the carbon aggregator service running to use this. Hope this helps ! Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
[collectd] [PATCH 1/3] Varnish plugin: added missing fields up to varnish 2.1.5.
Signed-off-by: Marc Fournier
---
src/collectd.conf.in |7 ++
src/types.db |6 +
src/varnish.c| 255 +
3 files changed, 164 insertions(+), 104 deletions(-)
diff --git a/src/collectd.conf.in b/src/collectd.conf.in
index 94cf2a9..f1cc29d 100644
--- a/src/collectd.conf.in
+++ b/src/collectd.conf.in
@@ -932,14 +932,21 @@
# CollectCache true
# CollectBackend true
# CollectConnections true
+# CollectDirectorDNS false
# CollectSHM true
# CollectESI false
# CollectFetch false
# CollectHCB false
+# CollectObjects false
+# CollectPurge false
+# CollectSession false
# CollectSMA false
# CollectSMS false
# CollectSM false
+# CollectStruct false
# CollectTotals false
+# CollectUptime false
+# CollectVCL false
# CollectWorkers false
#
#
diff --git a/src/types.db b/src/types.db
index e6345ab..5248ace 100644
--- a/src/types.db
+++ b/src/types.db
@@ -6,6 +6,7 @@ apache_requests value:DERIVE:0:U
apache_scoreboard value:GAUGE:0:65535
ath_nodes value:GAUGE:0:65535
ath_stat value:DERIVE:0:U
+backends value:GAUGE:0:65535
bitratevalue:GAUGE:0:4294967295
bytes value:GAUGE:0:U
cache_operationvalue:DERIVE:0:U
@@ -103,6 +104,7 @@ node_octets rx:DERIVE:0:U, tx:DERIVE:0:U
node_rssi value:GAUGE:0:255
node_stat value:DERIVE:0:U
node_tx_rate value:GAUGE:0:127
+objectsvalue:DERIVE:0:U
operations value:DERIVE:0:U
percentvalue:GAUGE:0:100.1
pg_blksvalue:DERIVE:0:U
@@ -129,6 +131,7 @@ ps_rss
value:GAUGE:0:9223372036854775807
ps_stacksize value:GAUGE:0:9223372036854775807
ps_state value:GAUGE:0:65535
ps_vm value:GAUGE:0:9223372036854775807
+purge_operationvalue:DERIVE:0:U
queue_length value:GAUGE:0:U
recordsvalue:GAUGE:0:U
requests value:GAUGE:0:U
@@ -137,12 +140,14 @@ route_etx value:GAUGE:0:U
route_metric value:GAUGE:0:U
routes value:GAUGE:0:U
serial_octets rx:DERIVE:0:U, tx:DERIVE:0:U
+session_operation value:DERIVE:0:U
signal_noise value:GAUGE:U:0
signal_power value:GAUGE:U:0
signal_quality value:GAUGE:0:U
snrvalue:GAUGE:0:U
spam_check value:GAUGE:0:U
spam_score value:GAUGE:U:U
+struct value:DERIVE:0:U
swap_iovalue:DERIVE:0:U
swap value:GAUGE:0:1099511627776
tcp_connectionsvalue:GAUGE:0:4294967295
@@ -161,6 +166,7 @@ total_time_in_msvalue:DERIVE:0:U
total_values value:DERIVE:0:U
uptime value:GAUGE:0:4294967295
users value:GAUGE:0:65535
+vclvalue:GAUGE:0:65535
vcpu value:GAUGE:0:U
virt_cpu_total value:DERIVE:0:U
virt_vcpu value:DERIVE:0:U
diff --git a/src/varnish.c b/src/varnish.c
index 859df21..99f224a 100644
--- a/src/varnish.c
+++ b/src/varnish.c
@@ -23,110 +23,6 @@
* Florian octo Forster
**/
-/**
- * Current list of what is monitored and what is not monitored (yet)
- * {{{
- * Field name Description Monitored
- * -- --- -
- * uptime Child uptime N
- * client_conn Client connections accepted Y
- * client_drop Connection dropped, no sess Y
- * client_req Client requests received Y
- * cache_hitCache hitsY
- * cache_hitpassCache hits for pass Y
- * cache_miss Cache misses Y
- * backend_conn Backend conn. success Y
- * backend_unhealthyBackend conn. not attempted Y
- * backend_busy Backend conn. too manyY
- * backend_fail Backend conn. failuresY
- * backend_reuseBackend conn. reuses Y
- * backend_toolate Backend conn. was closed Y
- * backend_recycle Backend conn. recyclesY
- * backend_unused Backend conn. unused Y
- * fetch_head Fetch headY
- * fetch_length Fetch with Length Y
- * fetch_chunkedFetch chunked Y
- * fetch_eofFetch EOF Y
- * fetch_badFetch had bad
[collectd] [PATCH 2/3] Varnish plugin: indentation and ordering consistency.
Signed-off-by: Marc Fournier
---
src/collectd.conf.in |6 +-
src/varnish.c| 138 +-
2 files changed, 72 insertions(+), 72 deletions(-)
diff --git a/src/collectd.conf.in b/src/collectd.conf.in
index f1cc29d..bf8238a 100644
--- a/src/collectd.conf.in
+++ b/src/collectd.conf.in
@@ -929,20 +929,20 @@
# If you prefer defining another instance you can do
# so by using
#
-# CollectCache true
# CollectBackend true
+# CollectCache true
# CollectConnections true
# CollectDirectorDNS false
-# CollectSHM true
# CollectESI false
# CollectFetch false
# CollectHCB false
# CollectObjects false
# CollectPurge false
# CollectSession false
+# CollectSHM true
+# CollectSM false
# CollectSMA false
# CollectSMS false
-# CollectSM false
# CollectStruct false
# CollectTotals false
# CollectUptime false
diff --git a/src/varnish.c b/src/varnish.c
index 99f224a..4fac591 100644
--- a/src/varnish.c
+++ b/src/varnish.c
@@ -34,20 +34,20 @@
struct user_config_s {
char *instance;
+ _Bool collect_backend;
_Bool collect_cache;
_Bool collect_connections;
- _Bool collect_esi;
- _Bool collect_backend;
_Bool collect_dirdns;
+ _Bool collect_esi;
_Bool collect_fetch;
_Bool collect_hcb;
_Bool collect_objects;
_Bool collect_purge;
_Bool collect_session;
_Bool collect_shm;
+ _Bool collect_sm;
_Bool collect_sma;
_Bool collect_sms;
- _Bool collect_sm;
_Bool collect_struct;
_Bool collect_totals;
_Bool collect_uptime;
@@ -109,6 +109,32 @@ static int varnish_submit_derive (const char
*plugin_instance, /* {{{ */
static void varnish_monitor (const user_config_t *conf, struct varnish_stats
*VSL_stats) /* {{{ */
{
+ if (conf->collect_backend)
+ {
+ /* Backend conn. success */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "success", VSL_stats->backend_conn);
+ /* Backend conn. not attempted */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "not-attempted", VSL_stats->backend_unhealthy);
+ /* Backend conn. too many */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "too-many", VSL_stats->backend_busy);
+ /* Backend conn. failures */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "failures", VSL_stats->backend_fail);
+ /* Backend conn. reuses*/
+ varnish_submit_derive (conf->instance, "backend",
"connections", "reuses",VSL_stats->backend_reuse);
+ /* Backend conn. was closed*/
+ varnish_submit_derive (conf->instance, "backend",
"connections", "was-closed",VSL_stats->backend_toolate);
+ /* Backend conn. recycles */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "recycled", VSL_stats->backend_recycle);
+ /* Backend conn. unused*/
+ varnish_submit_derive (conf->instance, "backend",
"connections", "unused",VSL_stats->backend_unused);
+ /* Backend conn. retry */
+ varnish_submit_derive (conf->instance, "backend",
"connections", "retries", VSL_stats->backend_retry);
+ /* Backend requests mades */
+ varnish_submit_derive (conf->instance, "backend", "requests",
"requests", VSL_stats->backend_req);
+ /* N backends */
+ varnish_submit_gauge (conf->instance, "backend", "backends",
"n_backends", VSL_stats->n_backend);
+ }
+
if (conf->collect_cache)
{
/* Cache hits */
@@ -149,52 +175,26 @@ static void varnish_monitor (const user_config_t *conf,
struct varnish_stats *VS
varnish_submit_derive (conf->instance, "esi",
"total_operations", "error", VSL_stats->esi_errors);
}
- if (conf->collect_backend)
- {
- /* Backend conn. success */
- varnish_submit_derive (conf->instance, "backend",
"connections", "success" , VSL_stats->backend_conn);
- /* Backend conn. not attempted */
[collectd] varnish plugin updates
Hello, Here are a couple of patches I would like to suggest for inclusion before the 5.0 release. The first one adds the possibility to collect all the remaining stats defined in varnish's stat_field.h. Rationale: - I wanted to make graphs out of varnish's cache purge activity, and couldn't because we arbitrarily decided that it wasn't useful to collect these values. - more stats were added to stat_field.h in varnish release 2.1.4 (backend_retry, dir_dns_lookups, dir_dns_failed, dir_dns_hit, dir_dns_cache_full ) and in 2.1.5 (fetch_1xx, fetch_204, fetch_304). - the nagios plugin for varnish is unnecessarily complicated to build, and doesn't seem actively maintained. Using collectd-nagios instead makes much more sense to me. The 2 other patches are code cleanup and a minor bugfix. You'll also find these patches here: https://github.com/mfournier/collectd/tree/varnish-plugin-updates Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
[collectd] [PATCH 3/3] Varnish plugin: missing definition in default config.
Signed-off-by: Marc Fournier
---
src/varnish.c |1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/src/varnish.c b/src/varnish.c
index 4fac591..60999b3 100644
--- a/src/varnish.c
+++ b/src/varnish.c
@@ -453,6 +453,7 @@ static int varnish_config_apply_default (user_config_t
*conf) /* {{{ */
conf->collect_totals = 0;
conf->collect_uptime = 0;
conf->collect_vcl = 0;
+ conf->collect_workers = 0;
return (0);
} /* }}} int varnish_config_apply_default */
--
1.7.2.3
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello, On Sun, Jun 27, 2010 at 08:41:19AM +0200, Jerome Renard wrote: > Hi Marc, > > [...] > > > You'll find an implementation suggestion in this branch: > > > > http://github.com/mfournier/collectd/commits/varnish-data-reorg > > I have applied your patches and I had to modifiy some minor things [1]. I agree meaning is more eloquent with this change, but it breaks the consistency where the names used to matche the names from varnishstat ... > > > > - would it make more sense to name the directories after the configuration > > option ? ie: CollectWorkers -> varnish-instance-workers. They actually > > match the varnish values names (same than returned by "varnishstat -l"). > > Sounds good to me. ... but in any case it's probably more comprehensive to have the directories match the name of the configuration items. I've pushed a trivial patch renaming a couple of values this way onto my github branch, as we seem to agree on this. > Data really looks better organized but I am still unsure this is the way > to go. Well, it's should at least be easier to make useful graphs with collection3 now :-) Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello, > On Fri, Jun 11, 2010 at 10:51:03AM +0200, Florian Forster wrote: > > > > I'm currently thinking about moving information from the type instance > > to the plugin instance. [...] > > > > If we move some of the information into the plugin instance, the data is > > structured better: > > > > host/varnish-default-storage-file/... > > .../total_requests > > .../requests-outstanding > > .../bytes-allocated > > .../bytes-free > > host/varnish-default-storage-mem/... > > .../total_requests > > .../requests-outstanding > > .../bytes-outstanding > > .../bytes-allocated > > .../bytes-free > > host/varnish-default-storage-synth/... > > .../total_requests > > .../requests-outstanding > > .../bytes-outstanding > > .../bytes-allocated > > .../bytes-free > > You'll find an implementation suggestion in this branch: > > http://github.com/mfournier/collectd/commits/varnish-data-reorg As requested, I changed the strncat/sstrncpy functions for one ssnprintf. The resulting behaviour remains the same as before. I also pushed another patch adding a whitespace in front of each opening bracket, as it seems to be the coding style used in the rest of the project. Cheers, Marc signature.asc Description: Digital signature ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello Jérôme, > I added Marc's branch as a remote branch so I have this in my > .git/config file : > > [...] > [remote "octo"] > url = git://github.com/octo/collectd.git > fetch = +refs/heads/jr/varnish:refs/remotes/octo/jr/varnish > [remote "mfournier"] > url = http://github.com/mfournier/collectd.git > fetch = +refs/heads/master:refs/remotes/mfournier/master > [...] I have something looking like: [remote "jeromer"] url = git://github.com/jeromer/collectd.git fetch = +refs/heads/*:refs/remotes/jeromer/* [remote "octo"] url = git://github.com/octo/collectd.git fetch = +refs/heads/*:refs/remotes/octo/* [remote "mf"] url = g...@github.com:mfournier/collectd.git fetch = +refs/heads/*:refs/remotes/mf/* This allows git to know about every branch in each of the 3 remote repositories. Then, I often have a "gitk --all" running. This helps a lot understanding how all these commits are stacked. > But whenever I run git merge mfournier/master git just merges > everything (34 commits) and not only the one I want. Did I do something wrong > ? Hmm, I'm not 100% sure but I think that if you pass 2 commits (remember of a branch as a label attached to a commit) to "merge", git will do an octopus (=multiple) merge between both branch heads AND the current branch. I guess this is not what you were expecting... You probably want something like this, which should inform you of your master branch getting fast-forwarded: git checkout master git merge mfournier/master I hope this will help ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Collectd-nagios - percentage of swap used
Hello, > I've been trying for three hours now and I seem unable to get the > right collectd-nagios command line to get the percentage of swap used. > > I get the right values when I do the following: > > yha...@grimston:/var/lib/collectd/rrd/yhamon-dev/swap$ sudo -u > nagios /usr/bin/collectd-nagios -s /var/run/collectd-unixsock -H > yhamon-dev -n swap/swap-free OKAY: 0 critical, 0 warning, 1 okay | > value=5.09403e+08 > yha...@grimston:/var/lib/collectd/rrd/yhamon-dev/swap$ sudo -u > nagios /usr/bin/collectd-nagios -s /var/run/collectd-unixsock -H > yhamon-dev -n swap/swap-used OKAY: 0 critical, 0 warning, 1 okay | > value=1.06496e+06 > > But impossible to get a percentage, whatever I try. > Any help would be highly appreciated, as I won't get very far with > just this :( collectd-nagios can only check against values found in RRD files. AFAIK, collectd's "swap" plugin doesn't record this value. Why don't you use nagios's check_swap plugin ? It has exactly the option you are looking for: $ /usr/lib/nagios/plugins/check_swap -w 100% SWAP WARNING - 99% free (1955 MB out of 1983 MB) |swap=1955MB;1983;0;0;1983 By the way, I use collectd-nagios to check "vmem/vmpage_io-swap", which I find more relevant to monitor than the percentage of swap used. I hope this helps ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
[collectd] issue on shutdown with 4.10
Hello, On systems running 4.10.0 or 37e18082, I discovered that the socket created by the unixsocket plugin was sometimes not removed on shutdown, which prevents the plugin from working with this error at the next restart: unixsock plugin: bind failed: Address already in use This problem is non-systematic. I tried to strace collectd, but didn't find any obvious error. But I did notice that when the socket fails to get removed, strace exits with an error: $ strace -f -s0 -o /tmp/strace.txt -p 30699 Process 30699 attached with 10 threads - interrupt to quit Process 30708 detached Process 30709 detached Process 30707 detached Process 30705 detached Process 30704 detached PANIC: handle_group_exit: 30703 leader 30699 PANIC: handle_group_exit: 30702 leader 30699 PANIC: handle_group_exit: 30701 leader 30699 PANIC: handle_group_exit: 30700 leader 30699 When the socket gets removed, strace ends without "panicking". NB: I checked file and dir modes, collectd can create/remove the socket file without restriction. Any suggestions how I could dig further in order to find out what's going on ? Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello,
On Fri, Jun 11, 2010 at 10:51:03AM +0200, Florian Forster wrote:
>
> I'm currently thinking about moving information from the type instance
> to the plugin instance. [...]
>
> If we move some of the information into the plugin instance, the data is
> structured better:
>
> host/varnish-default-storage-file/...
> .../total_requests
> .../requests-outstanding
> .../bytes-allocated
> .../bytes-free
> host/varnish-default-storage-mem/...
> .../total_requests
> .../requests-outstanding
> .../bytes-outstanding
> .../bytes-allocated
> .../bytes-free
> host/varnish-default-storage-synth/...
> .../total_requests
> .../requests-outstanding
> .../bytes-outstanding
> .../bytes-allocated
> .../bytes-free
You'll find an implementation suggestion in this branch:
http://github.com/mfournier/collectd/commits/varnish-data-reorg
As I'm no C expert, please be indulgent :-P Comments welcome !
This modification allow this config:
CollectCache true
CollectConnections true
CollectBackend false
CollectSHM false
CollectTotals true
CollectWorkers true
CollectCache false
CollectBackend false
CollectSHM false
to dispatch the values in directories following this structure:
lonquimay.wrk.lsn.camptocamp.com/
lonquimay.wrk.lsn.camptocamp.com/varnish-default-cache
lonquimay.wrk.lsn.camptocamp.com/varnish-default-cache/cache_result-hitpass.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-default-cache/cache_result-hit.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-default-cache/cache_result-miss.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-default-client
lonquimay.wrk.lsn.camptocamp.com/varnish-default-client/connections-accepted.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-default-client/connections-dropped.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-default-client/connections-received.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-client
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-client/connections-accepted.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-client/connections-dropped.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-client/connections-received.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/threads-worker.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_requests-dropped.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_requests-overflowed.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_requests-queued.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_threads-created.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_threads-failed.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-n_wrk/total_threads-limited.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_bytes-body-bytes.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_bytes-header-bytes.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_operations-fetches.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_operations-pass.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_operations-pipe.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_requests-requests.rrd
lonquimay.wrk.lsn.camptocamp.com/varnish-lonquimay-s/total_sessions-sessions.rrd
A couple of thoughts:
- as I understood, the plugin instance name is limited to 64 characters
(DATA_MAX_NAME_LEN). This could be an issue with long instance names. As
the trailing part gets truncated, we could end up having
varnish-quite_long_instance_name_more_than_64_characters_long-{sm,sma,sms}/bytes-allocated.rrd
ending as one same file.
- would it make more sense to name the directories after the configuration
option ? ie: CollectWorkers -> varnish-instance-workers. They actually
match the varnish values names (same than returned by "varnishstat -l").
> Here, "default" is the Varnish instance name (which would no longer be
> allowed to be empty).
This is in the 2nd patch in my "varnish-data-reorg" branch.
Cheers,
Marc
signature.asc
Description: Digital signature
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello, > In any case, I'll keep you informed of any trouble encountered. > > > > Did you nothing any (new ?) issue so far ? I actually found a small issue: plugin_dispatch_values: Dataset not found: total_counters I fixed this error in this small patch: http://github.com/mfournier/collectd/commit/eda2991a5f4f9491c29a881b03de4340d9832c99 Marc signature.asc Description: Digital signature ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello, > On Fri, Jun 11, 2010 at 08:40:30AM +0200, Jerome Renard wrote: > > Are you still running the plugin on your production machines ? I was planning to but actually I spent some time struggling to build a decent rpm package. Finally the kind fedora folks have more or less done this for me [¹] :-) As I try to avoid entropy on production servers by not circumventing the package manager, the next step for me is to build a nice package with the source from the HEAD of Florian's branch. Then push it on the machines... In any case, I'll keep you informed of any trouble encountered. > > Did you nothing any (new ?) issue so far ? I just spent some time testing with different "instance" parameters, which work just fine. I was also working on a reasonable varnish config for contrib/collection3/etc/collection.conf, but maybe I should wait until Florian settles down with the way instances and types are named... > If we move some of the information into the plugin instance, the data is > structured better: This seems quite sensible to me. > > Do you think the plugin should monitor more stuff or is that > > sufficient for now ? > > I think the following might be interesting: > > s_sess Total Sessions > s_reqTotal Requests > s_pipe Total pipe > s_pass Total pass > s_fetch Total fetch > s_hdrbytes Total header bytes > s_bodybytes Total body bytes I would find the stuff about running threads would be nice too: n_wrkN worker threads n_wrk_create N worker threads created n_wrk_failed N worker threads not created n_wrk_maxN worker threads limited n_wrk_queue N queued work requests n_wrk_overflow N overflowed work requests n_wrk_drop N dropped work requests > > @Florian : > > Do you think we should focus on the Wiki page that will document the > > plugin now ? > > I already set up a preliminary wiki page at [0]. Feel free to edit it to > your liking ;) You need to log-in to upload example graphs, though. I'll be glad to upload a couple of graphs as soon as I have some "real" ones. I've added a note concerning this linking issue Florian diagnosed. Marc [¹] https://bugzilla.redhat.com/show_bug.cgi?id=536725 signature.asc Description: Digital signature ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Hello Florian, Thanks for your help and explanations ! I definitely learned a couple of things today :-) > > /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnish.so: > > undefined reference to `strlcpy' > > That's new. I thought the GNU libc did implement "strlcpy", too..? According to wikipedia, it's not the case: http://en.wikipedia.org/wiki/Strlcpy#Criticism Cheers, Marc signature.asc Description: Digital signature ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
Jérôme, > can you please pull the changes from my branch: > - http://github.com/jeromer/collectd/ > > I just pushed Florian's patches and if you follow what I said in my > previous email [1] about pkg-config you > should be able to compile the Varnish plugin on your Debian (I did not > try on RHEL yet, sorry). As soon as you wrote about this -lvarnishcompat trick, I was able to build the varnish plugin on redhat 5.5 without any trouble. Later I even applied the patch to the 4.9 branch and built it successfully. Thanks for figuring that out ! I understood that the main varnish developer was also a freebsd developer. This might explain why strlcpy is used, and the reason of this varnishcompat library for linux... In any case, the plugin seems to work fine. Values are written to RRD files every now and then, and I didn't notice any coredumps or dreadful log messages. I'll continue to play around with this plugin until I have graphs of my varnish production machines. So I'll let you know of any other issue I come across. Thanks ! Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
> could you look for relevant information in "config.log", the file > created by the configure script? Something like > > grep -C 10 varnish config.log Here you are: | #define HAVE_PTHREAD_H 1 | #define HAVE_LIBPTHREAD 1 | #define HAVE_RRD_H 1 | #define HAVE_THREADSAFE_LIBRRD 1 | #define HAVE_LIBXMMS 0 | /* end confdefs.h. */ | #include configure:43973: result: no configure:44008: checking for yajl/yajl_parse.h configure:44015: result: no configure:44178: Checking for varnishapi using /usr/bin/pkg-config configure:44224: checking varnish/varnishapi.h usability configure:44236: gcc -c -g -O2 -I/usr/include/varnish conftest.c >&5 configure:44242: $? = 0 configure:44246: test -z || test ! -s conftest.err configure:44249: $? = 0 configure:44252: test -s conftest.o configure:44255: $? = 0 configure:44265: result: yes configure:44269: checking varnish/varnishapi.h presence configure:44279: gcc -E -I/usr/include/varnish conftest.c configure:44285: $? = 0 configure:44305: result: yes configure:44340: checking for varnish/varnishapi.h configure:44347: result: yes configure:44373: checking for VSL_OpenStats in -lvarnishapi configure:44403: gcc -o conftest -g -O2 -I/usr/include/varnishconftest.c -lvarnishapi -lvarnishapi>&5 /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnishapi.so: undefined reference to `VRE_compile' /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnishapi.so: undefined reference to `VRE_exec' collect2: ld returned 1 exit status configure:44409: $? = 1 configure: failed program was: | /* confdefs.h. */ | | #define PACKAGE_NAME "collectd" | #define PACKAGE_TARNAME "collectd" | #define PACKAGE_VERSION "4.10.0.36.g25e6243" | #define PACKAGE_STRING "collectd 4.10.0.36.g25e6243" | #define PACKAGE_BUGREPORT "" -- config.status:1120: creating src/collectd.conf config.status:1120: creating src/owniptc/Makefile config.status:1120: creating src/libcollectdclient/Makefile config.status:1120: creating src/libcollectdclient/libcollectdclient.pc config.status:1120: creating src/liboconfig/Makefile config.status:1120: creating bindings/Makefile config.status:1120: creating bindings/java/Makefile config.status:1224: creating src/config.h config.status:1874: executing depfiles commands configure:55091: configuring in libltdl configure:55208: running /bin/sh './configure' --prefix=/opt/collectd '--prefix' '/opt/collectd' '--disable-all-plugins' '--enable-varnish' --enable-ltdl-convenience --cache-file=/dev/null --srcdir=. configure:55409: error: "Some plugins are missing dependencies - see the summary above for details" ## ## ## Cache variables. ## ## ## ac_cv_build=x86_64-redhat-linux-gnu ac_cv_build_alias=x86_64-redhat-linux-gnu ac_cv_c_compiler_gnu=yes ac_cv_c_const=yes -- ac_cv_header_sys_vfs_h=yes ac_cv_header_sys_vfstab_h=no ac_cv_header_sys_wait_h=yes ac_cv_header_syslog_h=yes ac_cv_header_tcrdb_h=no ac_cv_header_termios_h=yes ac_cv_header_time=yes ac_cv_header_unistd_h=yes ac_cv_header_utmp_h=yes ac_cv_header_utmpx_h=yes ac_cv_header_varnish_varnishapi_h=yes ac_cv_header_vm_anon_h=no ac_cv_header_wordexp_h=yes ac_cv_header_xfs_xqm_h=no ac_cv_header_yajl_yajl_parse_h=no ac_cv_host=x86_64-redhat-linux-gnu ac_cv_host_alias=x86_64-redhat-linux-gnu ac_cv_lib_IOKit_IOServiceGetMatchingServices=no ac_cv_lib_curl_curl_easy_init=yes ac_cv_lib_dl_dlopen=yes ac_cv_lib_dld_shl_load=no -- ac_cv_lib_kvm_kvm_getprocs=no ac_cv_lib_kvm_kvm_getswapinfo=no ac_cv_lib_kvm_kvm_nlist=no ac_cv_lib_kvm_kvm_openfiles=no ac_cv_lib_oconfig_oconfig_parse_fh=no ac_cv_lib_pcap_pcap_open_live=no ac_cv_lib_pthread_pthread_create=yes ac_cv_lib_resolv_res_search=no ac_cv_lib_rrd_th_rrd_update_r=yes ac_cv_lib_rrd_th_rrdc_update=no ac_cv_lib_varnishapi_VSL_OpenStats=no ac_cv_lib_xml2_xmlXPathEval=yes ac_cv_member_kstat_io_t_nwrites=no ac_cv_member_kstat_io_t_nwritten=no ac_cv_member_kstat_io_t_writes=no ac_cv_member_kstat_io_t_wtime=no ac_cv_member_struct_if_data_ifi_ibytes=no ac_cv_member_struct_if_data_ifi_ierrors=no ac_cv_member_struct_if_data_ifi_opackets=no ac_cv_member_struct_ip_mreqn_imr_ifindex=yes ac_cv_member_struct_kinfo_proc_ki_pid=no > > /usr/lib64/libvarnish.so > > /usr/lib64/libvarnishapi.so > > /usr/lib64/libvarnishcompat.so > > Also, an "ldd" of each of those files would be interesting for the RHEL > box. On Debian Squeeze I know it's broken :/ /usr/lib64/libvarnish.so: librt.so.1 => /lib64/librt.so.1 (0x003ebce0) libnsl.so.1 => /lib64/libnsl.so.1 (0x003ebe20) libm.so.6 => /lib64/libm.so.6 (0x003ebbe0) libpcre.so.0 => /lib64/libpcre.so.0 (0x0034f9a0) libc.so.6 => /lib64/libc.so.6 (0x003ebb20) libpthread.so.0 => /lib64/libpthread.so.0 (0x003ebca0) /lib64/ld-linux-x86-64.so.2 (0x003ebae0) /usr/lib64/libvarnishapi.so: libc.so.6 => /lib64/libc.so.6 (0x003ebb20) /lib
Re: [collectd] Varnish plugin
Hello, > Other than that, there are two new "features": The configure script now > uses pkg-config to determine the correct C- and LD-flags. Unfortunately, > the Debian package is broken in this respect so that building under > Debian currently needs some hands-on hacking fixing. I opened bug > reports but I doubt this will be fixed soon – a similar bug report has > been open since November 2009 :( I'm no longer able to build the varnish plugin using your branch. I suppose this is related to this issue you mention ? A few relevant pieces below: $ git checkout 24da3d76 Previous HEAD position was e73f418... Varnish plugin: Added SM Monitor HEAD is now at 24da3d7... configure.in: Use pkg-config to check for libvarnishapi. $ ./build.sh + autoheader + aclocal + libtoolize --ltdl --copy --force + automake --add-missing --copy + autoconf $ ./configure --prefix /opt/collectd --disable-all-plugins --enable-varnish [...] libvarnish . . . . . no (symbol VSL_OpenStats not found) [...] varnish . . . . . . . no (dependency error) [...] configure: error: "Some plugins are missing dependencies - see the summary above for details" $ ./configure --prefix /opt/collectd --disable-all-plugins --enable-varnish --with-libvarnish=/usr/include/ [...] libvarnish . . . . . /usr/include/ [...] varnish . . . . . . . no (dependency error) [...] configure: error: "Some plugins are missing dependencies - see the summary above for details" $ rpm -ql varnish-libs-devel-2.1.2-1.el5 /usr/include/varnish /usr/include/varnish/shmlog.h /usr/include/varnish/shmlog_tags.h /usr/include/varnish/stat_field.h /usr/include/varnish/stats.h /usr/include/varnish/varnishapi.h /usr/lib64/libvarnish.so /usr/lib64/libvarnishapi.so /usr/lib64/libvarnishcompat.so /usr/lib64/libvcl.so /usr/lib64/pkgconfig/varnishapi.pc /usr/share/doc/varnish-libs-devel-2.1.2 /usr/share/doc/varnish-libs-devel-2.1.2/LICENSE $ pkg-config --libs varnishapi -lvarnishapi $ grep VSL_OpenStats /usr/include/varnish/* /usr/include/varnish/varnishapi.h:struct varnish_stats *VSL_OpenStats(const char *varnish_name); This problem occurs on RHEL 5.5 (latest stable release) as well as debian squeeze (testing). Can you elaborate on the "hands-on fixing" you mentioned ? > Unfortunately I'm flying blind because I don't have a Varnish instance > running anywhere. It'd be awesome if you could test the changes for me. I was about to do that, when I run into this build problem ;-) I'll test Jérôme's branch meanwhile and let you know of any other issues I encounter. Marc signature.asc Description: Digital signature ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
Re: [collectd] Varnish plugin
> I created a new plugin to monitor a Varnish [1] instance. > > So far the plugin monitor statistics about cache (hit/misses) and > connections but could monitor more in the future. As a varnish user, I'm glad to know I'll be able to have collectd make nice graphs from it soon ! Many thanks for your work so far Jérôme, I'm really looking forward to use it ! Let me know if I can help testing or something. As the varnishstat engine gathers a quite large number of values (around a hundred as of varnish 2.1), I think it would be a pity to arbitrarily only graph half a dozen of them in collectd. Wouldn't it be an idea to let the user choose which ones he wants to graph in collectd.conf ? What do you think ? > The patch is attached to this message, it has been generated against > the master branch of my local collectd copy. Just out of curiosity, is there any chance this plugin will build on older collectd versions such as 4.5 ? > Yeah I already thought about that and I was not sure what to do here. > Even though it is in theory possible to run multiple varnishd > instances on the same host, in practice it is quite common to have > only one instance per host. AFAIK, the default varnish instance name is the host's FQDN. Maybe would it be an idea have the instance name appear in the path where the rrd files get saved (sort of like the cpu and disk plugins do) ? Although I agree that in 99% of the cases people would have only one instance per host. # varnishstat -1 > /dev/null # echo $? 0 # varnishstat -1 -n foo > /dev/null Cannot open /var/lib/varnish/foo/_.vsl: No such file or directory # echo $? 1 # varnishstat -1 -n $(hostname --fqdn) > /dev/null # echo $? 0 Cheers, Marc ___ collectd mailing list collectd@verplant.org http://mailman.verplant.org/listinfo/collectd
