Re: [collectd] Curl Header

[Marc Fournier]

Hello,

Poil  writes:
> It does not seem possible to pass a custom header in the curl plugins 
> documentation (Like a header: host) .
>
> But when I check the source code, what I understand is it seems 
> possible, anyone can confirm or not ? (and have an example ?)

I just double-checked on collectd 5.6, but as far as I can tell, this
feature works, for all 3 curl, curl_json and curl_xml plugins. Also, the
documentation/examples are correct.

This feature has been introduced in version 5.3. Are you running a recent
enough version ?

Marc

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] Version 5.6.2 available.

[Marc Fournier]

Florian Forster  writes:


> Download
> 
>
> The new version is available in source-code form from collectd's
> download page. The direct download links are:
>
>   * http://collectd.org/files/collectd-5.5.2.tar.bz2
> SHA-256: cc0b4118a91e5369409ced22d1d8a85c1a400098419414160c1839268ecad0c6

NB: the link should be http://collectd.org/files/collectd-5.6.2.tar.bz2

The sha256sum is valid for collectd-5.6.2.tar.bz2 and obviously wrong
for collectd-5.5.2.tar.bz2

Thanks !

Marc

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] collectd inside a container monitoring the host

[Marc Fournier]

Maurizio Vitale  writes:

> There's another way that consists in LD_PRELOADing a shared object that
> intercepts all calls to fopen and prepend a prefix to filenames.

I like this approach, and actually have implemented such a wrapper and
have been using it for a couple of weeks now. So far so good.

See https://github.com/collectd/collectd/pull/2041

This way, no need to modify or rebuild collectd itself. It should work
with any existing or future version, with out-of-tree plugins, and even
with other tools than collectd ;-)

Marc

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] write_http plugin - dynamic tag

[Marc Fournier]

Not sure if this answers your question, but starting with 5.6, the
following config should allow you to add arbitry key-values to outgoing
metrics:


  
URL "http://localhost:4567/collectd-post;
Format "json"
  


PostCacheChain "PostCache"

  

  Plugin "^cpu$"


  MetaData "foo" "bar"

  
  Target "write"


Running with the config above, values such as this get POSTed by
collectd's write_http plugin:

{"values":[136247],"dstypes":["derive"],"dsnames":["value"],"time":1478019107.077,"interval":1.000,"host":"localhost","plugin":"cpu","plugin_instance":"2","type":"cpu","type_instance":"idle","meta":{"foo":"bar"}},

HTH,
Marc

Shirly Radco  writes:

> Would really appreciate if you can elaborate how...
>
> Thank you in advance,
>
> Shirly Radco
>
> BI Software Engineer
> Red Hat Israel Ltd.
> 34 Jerusalem Road
> Building A, 4th floor
> Ra'anana, Israel 4350109
>
>
> On Thu, Oct 13, 2016 at 1:36 PM, Aurelien ROUGEMONT 
> wrote:
>
>> On 10/13/2016 12:29 PM, Shirly Radco wrote:
>> > Hi,
>> >
>> > Is there an option to dynamically generate the tag in the collectd
>> > write_http plugin?
>> > (Use the record values to add as tag)
>> >
>> > Thank you,
>> >
>> > Shirly Radco
>> >
>> > BI Software Engineer Red Hat Israel Ltd. 34 Jerusalem Road Building A,
>> > 4th floor Ra'anana, Israel 4350109
>> >
>> >
>> >
>> > ___
>> > collectd mailing list
>> > collectd@verplant.org
>> > https://mailman.verplant.org/listinfo/collectd
>> >
>> Hello Shirly,
>>
>> I've done just that in the kairosdb format for write_http
>>
>> Aurelien
>>
>> --
>> Aurelien ROUGEMONT
>> be...@binaries.fr
>>
>>
>> ___
>> collectd mailing list
>> collectd@verplant.org
>> https://mailman.verplant.org/listinfo/collectd
>>
> ___
> collectd mailing list
> collectd@verplant.org
> https://mailman.verplant.org/listinfo/collectd

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] Documentation to write Plugins

[Marc Fournier]

Thijs Cramer  writes:

> Hi all,
>
> Is there any documentation / quickstart guide available on how to
> develop Plugins?
> I've been unable to find any.


You might also want to take a look at the plugin skeleton:

https://github.com/collectd/collectd/blob/master/contrib/examples/myplugin.c

Marc

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] parse errors on predefined postgresql plugin queries

[Marc Fournier]

Hello,

Aaron Vinson  writes:

> I'm attempting to use collectd postgresql plugin to monitor a Postgres
> 9.5.2 RDS instance. I've created a user with the correct permissions and
> tested the queries manually. The predefined "transactions" query works as
> expected but many of the others such as "queries" and "query_plans" result
> in the following error:

This same issue got mentioned on the IRC channel a couple of days ago,
so I opened https://github.com/collectd/collectd/issues/1905 to keep
track of it.

Please feel free to chime in if you have any updates.

Thanks for reporting this issue !

Marc

___
collectd mailing list
collectd@verplant.org
https://mailman.verplant.org/listinfo/collectd

Re: [collectd] how to set a threshold using postgresql plugin

[Marc Fournier]

Gerardo Herzig  writes:

> Hi all. Im sending collectd values from a postgres DB to our collectd central 
> server.
> In DB, there is using postgres plugin like:
>
> 
>   Statement "select sum(numbackends) as backends, sum(tup_inserted) as 
> inserts, sum(tup_updated) as updates, sum(tup_deleted) as deletes 
>   from pg_stat_database;"
>   
> Type gauge
> InstancePrefix total_backends
> ValuesFrom "backends"
>   
> 
> 
> Interval 10
> Host localhost
> User postgres
> Query db_movements
> 
>
> And, in the "central" collectd server i want to set a threshold for that 
> server, so
> " If that db server is over 100 connections in use, ALERT "
>
> At that point, i got confused about how to get it work.
> I have something like that:
>
> 
> 
>   
> FailureMax 100.0
>   
> 
>  
>
>
> It throws no error, but even if i put a FailureMax of 2 or so, alert is not 
> raising.
> So, anyone can help me to know to set that alert?

I don't think the above specification will match the value you're trying
to track.

Try something like this:


  
Instance "template1"

  Instance "total_backends"
  FailureMax 100.0

  


You might want to enable the unixsock plugin and run
"collectdctl listval" to make sure your specification actually matches
an existing element name.

Also, take a look at https://collectd.org/wiki/index.php/Naming_schema

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Exec script

[Marc Fournier]

Collectd User <collectd@siimnet.dk> writes:

>> On 24. feb. 2016, at 21.51, Marc Fournier <marc.fourn...@camptocamp.com> 
>> wrote:
>> 
>> Collectd User <collectd@siimnet.dk <mailto:collectd@siimnet.dk>> 
>> writes:
>> 
>>>> On 15. feb. 2016, at 10.55, Collectd User <collectd@siimnet.dk> wrote:
>>>> 
>>>> Fellow Collectors,
>>>> 
>>>> Trying my first exec plugin script, but it’s not working 100%.
>>>> 
>>>> I’ve added this type in /usr/share/collectd/types.db on the end point 
>>>> collecting the data:
>>>> 
>>>> ssdhealth   health:GAUGE:0:100.1, hours:GAUGE:0:4294967295
>>>> 
>>>> and submits data on stdout like this:
>>>> 
>>>> PUTVAL d1r1n1/ssdhealth-1I:1:3/ssdhealth 1452173939:100.00:2227
>>>> 
>>>> but after coming through my logstash and into elasticsearch, I’m missing 
>>>> health and hours fields in stored ES documents and wondering why only 
>>>> these two rather important fileds are missing…
>>>> 
>>>> any hints on howto debug this appreciated, TIA!
>>> Any one?
>> 
>> Mmmh, not sure exactly how the collectd protocol decoder works on the
>> logstash side, but I could imagine it needs the matching types in
>> types.db. Did you try adding your "ssdhealth" type to logstash's
>> types.db ?
>> 
>> Other than that, maybe try enabling the cvs plugin to make sure
>> everything works fine on the collectd side.
> Thanks for hinted, started a new collectd with a different config file with 
> just my exec collector and output directly to csv an dit shows values fine :)
> Will try to dig further into logstash…

This recent post on the logstash forum seems to be the problem you're
having, doesn't it ?

https://discuss.elastic.co/t/new-collectd-types-do-not-show-up-in-logstash-es/42727

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] modbus plugin spamming console with debug info

[Marc Fournier]

Hello Eric !

Eric Sandeen  writes:

> I hadn't noticed this before, but after running on a systemd system it
> comes to the forefront:
>
>> Message from syslogd@sandeen at Feb 24 22:39:48 ...
>>  collectd:<21><00><00><00><05><01><04><02><00><02>
>
>> Broadcast message from systemd-journ...@sandeen.net (Wed 2016-02-24 22:39:48 
>> CST):
>
>> collectd[22528]: <22><00><00><00><05><01><04><02><00><19>
>
> on every modbus communication.  ;)
>
> The plugin sets modbus_set_debug(ctx, 1); - that seems a bit odd to do 
> unconditionally.
>
> Has anyone else run into this?  Setting to debug should almost certainly be 
> optional,
> AFAICT.

Thanks for pointing this out !

Here's a patch which should prevent this:
https://github.com/collectd/collectd/pull/1561

Does it sound an acceptable solution ?

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Exec script

[Marc Fournier]

Collectd User  writes:

>> On 15. feb. 2016, at 10.55, Collectd User  wrote:
>> 
>> Fellow Collectors,
>> 
>> Trying my first exec plugin script, but it’s not working 100%.
>> 
>> I’ve added this type in /usr/share/collectd/types.db on the end point 
>> collecting the data:
>> 
>> ssdhealth   health:GAUGE:0:100.1, hours:GAUGE:0:4294967295
>> 
>> and submits data on stdout like this:
>> 
>> PUTVAL d1r1n1/ssdhealth-1I:1:3/ssdhealth 1452173939:100.00:2227
>> 
>> but after coming through my logstash and into elasticsearch, I’m missing 
>> health and hours fields in stored ES documents and wondering why only these 
>> two rather important fileds are missing…
>> 
>> any hints on howto debug this appreciated, TIA!
> Any one?

Mmmh, not sure exactly how the collectd protocol decoder works on the
logstash side, but I could imagine it needs the matching types in
types.db. Did you try adding your "ssdhealth" type to logstash's
types.db ?

Other than that, maybe try enabling the cvs plugin to make sure
everything works fine on the collectd side.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] AMQP Subscription

[Marc Fournier]

Theo Bot  writes:

Hello,

> I have a collectd server setup on a linux platform having an influxdb as a
> beackend. Is it possible to receive values via AMQP (RabbitMQ). And if so
> what would be the message format to process these values and what are the
> other requirements?

Yes, the amqp plugin supports both subscribing and publishing to an AMQP
broker. I never used this feature myself, but my understanding is that
you need to publish messages using the PUTVAL format (see the
collectd-unixsock(5) manpage) and set "text/collectd" as "Content-Type".

HTH,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Configure reporting interval on network plugin

[Marc Fournier]

Excerpts from Joel Serrano's message of 2016-02-15 20:41:58 -0800:
> 
> Has anyone successfully configured the network plugin in collectd to send
> metrics every X seconds? (like 10 seconds).

You can't control how frequently collected data is flushed to the write
target(s). Only how frequently read targets are probed.

In the case of the network plugin, the data gets flushed once its buffer
is full, which size defaults to 1.5kB. Are you collecting very little
data with a relatively high read interval ? If not you might want to
fire tcpdump to check you aren't loosing packets (remember the protocol
is UDP-based).

Note that the values dispatched include the collection time timestamp.
So even if there is some delay before they reach their final
destination, they should be stored with the correct time reference.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Problem using the write_sensu plugin

[Marc Fournier]

Excerpts from Joshua J. Kugler's message of 2016-02-03 15:27:33 -0900:
> [...]
> Metrics false
> Notifications true
> NotificationHandler "pagerduty"
> NotificationHandler "irc"

collectd dispatches 2 type of events:
 - metrics ("value of %free memory is N" every few seconds)
 - notifications ("warning: %free memory is below N")

Some write plugins only handle metrics (rrdtool), others only handle
notifications (notify_email). A 3rd category handle both (write_riemann,
write_sensu). Hence the Metrics, Notifications, MetricHandler and
NotificationHandler options to the write_sensu plugin.

According to your config, you've disabled metrics dispatching. The
easiest way to make sure the collectd->sensu link is working would be to
flip the Metrics boolean and add a MetricHandler. Or alternatively,
change thresholds so that notifications get raised. See
collectd-threshold(5).

If you don't want to store the metrics collected at all, you can use a
PostCacheChain to send them all to /dev/null. This should stop the "No
write callback has been registered" log message:

PostCacheChain "PostCache"

  Target "stop"


But I agree with you, the plugin could be slightly improved to make it
emit a couple of log messages during initalisation. It isn't obvious to
figure that out currently.

HTH,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] collectd on Windows

[Marc Fournier]

Excerpts from François-Xavier Choinière's message of 2016-01-19 00:08:05 +:
> Hi,
> I manage a network with hybrid OS, Linux and ... Windows :-(.
> I tried sscserv but the support is not good and it's not open source.
> I've tried to compile collectd on Windows with cygwin and the core seems to
> compile.
> Any of you think it's a good idea to compile the core on Windows and add a
> specific plugin for Windows.
> I can put some time on it. Like that you could have an official Windows
> port.

I'm aware the Stackdriver folks have done some work in this direction.
Unfortunately it didn't get to the point of a formal review/merge
request.

Maybe would you like to get in touch with them and take over this
project ?

Take a look at https://github.com/kele/collectd/branches/all and
https://github.com/kele/collectd/pulls

Have fun :-)

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] how to build a debian package of a daily snapshot?

[Marc Fournier]

Excerpts from Svante Karlsson's message of 2015-05-11 23:55:46 +0200:
 I finally managed to build both ubuntu 12  14 packages based on pkg-debian
 - if I skipped the patches and relaxed the dependency check on my home
 built packages.
 
 I also had to add packages libvirt-bin dpatch to be able to build.
 
 Finally I had to build protobuf and protobuf-c for both platforms (and
 librdkafka for ubuntu 12)

Great :-)

 If/when you start building the packages do you plan to provide them for
 download somewhere? That would be awesome!

I'm still messing around with this, but yes, packages are already built
(hopefully) each time new patches are commited. See the README of
https://github.com/collectd/collectd-ci for the where.

Once the last details are sorted out, I'll send a more formal email to
the mainling-list with all the details.

In the meantime, fell free to give these packages a try. Feedback is of
course welcome :)

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] how to build a debian package of a daily snapshot?

[Marc Fournier]

Hello Svante,

I still have a couple of details to sort out so nothing official yet,
but I've been working on making nightly builds of collectd for debian,
ubuntu LTS and centos.

So if you feel like giving this a try, recent packages for your target
distros are ready for consumption. Details here:
https://github.com/collectd/collectd-ci

By the way, an important fix to write_kafka has been merged this WE, so
I recommend using the current tip of the master branch if you need this
plugin.

Anwsering your questions regarding building your own packages:

 - libprotobuf-c-dev has been renamed to libprotobuf-c0-dev in ubuntu
   12.04 and 14.04. You should edit debian/control to reflect this
   change.

 - https://github.com/collectd/pkg-debian.git doesn't know yet about
   current collectd master (which has write_kafka). Have a look at
   https://github.com/mfournier/pkg-debian/branches/all for the
   differences which allow the nightly builds to work.

 - ubuntu 12.04 doesn't provide a package for librdkafka, so you'll need
   to backport it or install the library manually. The nightly builds I
   mentioned above don't provide write_kafka on ubuntu 12.04 because of
   this.

Hope that helps !

Marc

Excerpts from Svante Karlsson's message of 2015-05-10 23:00:06 +0200:
 I'm struggling to build a deb package for internal distribution on our
 servers. I need to have kafka-writer plugin so I think I need to build from
 a snapshot.
 
 this is what I'm doing
 
 sudo apt-get update
 sudo apt-get upgrade
 sudo apt-get install liboping-dev libmysqlclient-dev libpq-dev
 librabbitmq-dev  libvirt-bin librdkafka-dev
 sudo apt-get install po-debconf dpatch iptables-dev javahelper
 libcurl4-gnutls-dev libcurl4-gnutls-dev libcurl3-gnutls-dev libdbi0-dev
 libesmtp-dev
 sudo apt-get install libganglia1-dev libglib2.0-dev libhiredis-dev
 liblvm2-dev libmemcached-dev libmodbus-dev libmnl-dev
 sudo apt-get install libmysqlclient-dev libnotify-dev libopenipmi-dev
 libpcap0.8-dev
 sudo apt-get install libperl-dev libprotobuf-c0-dev librabbitmq-dev
 librrd-dev libsensors4-dev libsnmp-dev perl
 sudo apt-get install libtokyocabinet-dev libtokyotyrant-dev
 libupsclient-dev libvarnish-dev libvirt-dev libyajl-dev
 sudo apt-get install default-jdk protobuf-c-compiler  libow-dev libprotobuf
 
 
 git clone https://github.com/collectd/pkg-debian.git
 wget
 http://snapshots.tokkee.org/collectd/2015-03-17/collectd-5.4.2.694.g9817e72.tar.gz
 -Ocollectd-5.4.2.694.g9817e72.tar.gz
 tar xvzf collectd-5.4.2.694.g9817e72.tar.gz -C pkg-debian
 --strip-components=1
 cd pkg-debian
 dpkg-buildpackage -us -uc
 
 
 I've tried to build and install
 git clone https://github.com/google/protobuf.git
 git clone https://github.com/protobuf-c/protobuf-c.git
 
 but can't find dependency   libprotobuf-c-dev
 
 right now I'm running ubuntu 14.04 but I also need to get this working on
 12.04. I realize that I'm doing something wrong - could you please give me
 a hint on how to produce the deb package?
 
 thanks in advance
 svante

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] python plugin.

[Marc Fournier]

Excerpts from Dan Dong's message of 2015-03-10 16:19:06 +0100:
 Hi, All,
   Could anyone hint me how to install the python plugin of collectd? I
 downloaded the collectd-5.4.2.tar.gz but after:
 $./configure --prefix=/opt/collectd --enable-python
 $sudo make all install
 
 Where can I find the python plugin? Is it supposed to be under:
 /opt/collectd/lib/collectd/plugins/python/ ?
 
 But nothing there under /opt/collectd/lib/collectd/plugins/

You should be looking for /opt/collectd/lib/collectd/python.so.

After running ./configure, a summary is printed out which should tell
you which plugins won't be built and why not.

Maybe the configure script can't figure out where python is located on
your system ? Try adding --with-python=/path/to/python to ./configure in
this case.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] collectd not logging

[Marc Fournier]

Excerpts from Tom Throckmorton's message of 2014-10-28 02:34:32 +0100:
 On Mon, Oct 27, 2014 at 6:41 PM, Alison Chan chan7...@kettering.edu wrote:
 
  Hello,
  I'm developing a few new collectd plugins so I am interested in
  debug-level logs. However, when I configure logging no file is
  created. What should I do to solve this problem?
 
 
 From
 http://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_logfile
 :
 
 Please note that *debug* is only available if collectd has been compiled
 with debugging support.
 
 It's probably not enabled in the build you're using.  You can verify that
 logging generally works by switching to LogLevel info.
 
 The solution would be to rebuild and --enable-debug.

FYI, upcoming collectd 5.5 will fallback to info log level in this
case. Avoiding this confusing situation where nothing gets logged at
all.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Plugin varnish relocation error at startup of collectd [Varnish V4.0.2]

[Marc Fournier]

Excerpts from John Platter's message of 2014-10-19 11:59:03 +0200:
  I have a working collectd environment but as soon as I enable 
  configure this plugin then the collectd daemon stops and generates an
  error as follows. Please help. If you require more info then please
  let me know.

Yes, it's a known problem with varnish 4. You can keep track of it and
add your comments, if you have some, to this issue:
https://github.com/collectd/collectd/issues/618

Thanks !
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] bind plugin failing

[Marc Fournier]

Excerpts from Chip Marshall's message of 2014-09-26 20:26:00 +0200:
 I'm having some trouble with the bind plugin, and I could use
 some guidance in tracking down what the problem is. At the
 moment, I'm just getting read-function of plugin `bind' failed.
 in the log, followed by various Will suspend it for times.
 [...]
 I'm using collectd 5.4.1 on FreeBSD 10.0-RELEASE with BIND 9.10.1.

I believe your issue is adressed by this patch:
https://github.com/collectd/collectd/pull/742

It will be part of collectd 5.5, but can probably easily be applied to
the 5.4 branch if you'd like to benefit from it immediatly.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] ldap plugin for collectd ?

[Marc Fournier]

Hello Kimo,

Excerpts from Kimo Rosenbaum's message of 2014-06-04 17:56:13 +0200:
 
 A few years ago I started on an openldap plugin:
 
 http://mailman.verplant.org/pipermail/collectd/2011-January/004336.html 
 (although this seems to have truncated the email)
 
 Here is the original email and patch I sent:
 
 https://gist.github.com/kimor79/684e2ef4f48037e9

I just wanted to inform you I made a bunch of improvements to your
openldap plugin recently:

https://github.com/mfournier/collectd/compare/kimor79:kimor79/openldap...openldap-improvements

If by any chance you still use it, I'd be interested by your feedback !

Andrew: if you're still planning to use this plugin, I was able to build
it against an existing installation of collectd using:
  gcc -DHAVE_CONFIG_H -Wall -Werror -g -O2 -shared -fPIC \
-I/usr/include/collectd/core/ -ldl -lldap -o openldap.so openldap.c
(after installing the openldap-devel/libldap2-dev package)

My plan is to test this on a bunch of servers for a bit, then once I'm
happy with it, rebase the branch against current master and (re)submit
it for inclusion.

Thanks !
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] generating composite metrics

[Marc Fournier]

Hello,

Excerpts from Gerardo Herzig's message of 2014-06-25 18:14:10 +0200:
 Hi all. Im wondering if it is possible to generate a metric as a
 calculation between others.
 
 Example I: Know the rate between the used memory and the total
 memory.  Example II: Know the rate between In use apache threads
 against total apache threads.

As a side note, the master branch has a patch which allows the memory
plugin to emit percentages instead of absolute values. Maybe this could
solve example 1 in a way ?

 I know how to make a graph as this sort of calculations, but i also
 need to generate an alert when, say, 75% of the apache's threads are
 in use, so the graphical solutions does not cover all my needs.
 
 So i guess the correct solution is to generate my own .rrd files
 according to existing .rrd files. I just dont have any idea how.

I like using riemann for this sort of use-cases (http://riemann.io).
Basically, you just use collectd to collect metrics from your systems
and send them to a riemann server using the write_riemann plugin. Then
do the calculations/aggregations/correlations (and optionnally, the
alerting based on the result) in riemann.

If you're tied to having the resulting values in RRD files (which
riemann doesn't know about), you can run riemann embedded in collectd
using the java language binding plugin. This way you can do the filering
and calculations you want using riemann, and have the results passed
down to collectd's rrdtool/rrdcached plugin. Search for collmann on
github for the details.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] varnish plugin, not found

[Marc Fournier]

Excerpts from L Cruzero's message of 2014-05-23 15:38:06 +0200:
 Hi, i've been using collectd for a few months to gather based system stats,
 recently i've tried unsuccessfully to use the varnish plugin.
 
 on start:
 
 stdout:
 
 Starting collectd: Could not find plugin varnish
 [...]
 collectd-5.1.0-1.el6.rft.x86_64
 
 
 varnish-release-3.0-1.el6.noarch

As neither of these packages are part of rhel6, I guess you have enabled
additional repositories from where you installed them. It's important
that you install the same libvarnishapi version that was used to build
the collectd plugin (it will typically be found in the same repo).

If you don't find the exact match of varnish + collectd versions you
wish in a repository out there, you'll have to build collectd yourself.
There's a rpm specfile for el5  el6 in contrib/redhat/, in collectd's
source tree.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Which plug-ins support collection interval on a per-plugin basis feature introduced from 5.2

[Marc Fournier]

Hello,

Excerpts from andrewbooty .'s message of 2014-05-09 23:22:22 +0200:
 I've been testing the 5.4.1 release of collectd and assumed that most core
 plug-ins now supported the Interval feature to override the global setting.
 For ethstat, interface and table this hasn't been implemented yet. So
 wondering if there is a list of which plug-ins do currently support their
 own interval settings. I could check the source code to see if Interval is
 included in the callback function. But if Interval support was included in
 say the wiki  plugin table that would be helpful.

*All* the plugins support custom intervals since 5.2.0. This is how you
have to load a plugin with a non-default interval:

  LoadPlugin df
Interval 3600
  /LoadPlugin

What is a bit confusing, is that some plugins implement this or some
similar behaviour themselves for various reasons.
https://collectd.org/documentation/manpages/collectd.conf.5.shtml#plugin_netapp
is an example of such a plugin.

So writing something like this *could* work for some plugins, but is not
the correct way of using this feature:

  LoadPlugin df
  Plugin df
Interval 3600
  /Plugin

Could there be some misunderstanding on your side ? Or do you have
reasons to believe there's something wrong with the ethstat, interface
and table plugins and not the others ?

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Varnish plugin : Objects LRU nuked are not reported

[Marc Fournier]

Hello,

Excerpts from smainklh's message of 2014-04-04 15:49:16 +0200:
 Hello,
 
 I'm currently benching a varnish server and there's an essential value i need 
 to monitor : LRU Nuked.
 Actually all other metrics are well gathered by collectd but for this one i 
 always get a 0.
 However when i look to the varnishstat the value increases.
 
 varnishstat -1 | grep n_lru_nuked
 n_lru_nuked  680  .   N LRU nuked objects
 
 Do you know how to solve this issue please ?

What version of collectd  varnish are you running ? I just checked and
don't have this problem with 5.4.1 and varnish 3.0.2.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Ceph plugin for collectd

[Marc Fournier]

Hello Dan !

Thanks for following up !

Excerpts from Dan Ryder (daryder)'s message of 2014-03-14 19:17:42 +0100:
 Hi,
 
 I was looking through the collectd mailer-list archives and read through
 some discussion about a ceph plugin for collectd.
 
 It doesn't seem like this code ever got added to the official collectd
 code. Is there a reason for this? I saw
 https://github.com/ceph/collectd-4.10.1 on github and used the ceph
 plugin here to get a working version.
 
 I forked the main collectd code and modified the ceph plugin to fix a few
 things.
 
 Is there a way I can commit these plugins to the official collectd code?

Last time ceph was mentioned on the list, the question was if using libyajl
(which the curl_json plugin uses) over libjson (that the ceph plugin uses)
was possible. The question remains open AFAIK. From the maintenance and
packaging point of view, it's really better not to depend on different
libraries providing the same features.

On the other hand, if libjson is considered better than libyajl, why not
consider changing curl_json to use it ?

In both cases, the amount of code actually using libjson/libyajl doesn't
seem to be huge. Do you think this is something that could be done ?

BTW, I'm going to close the current pull-request for this plugin
(https://github.com/collectd/collectd/pull/522) because it seems to have
went out of control:
 26 files changed, 9018 insertions(+), 7955 deletions(-)

If you have a clean working branch with the ceph plugin in it's current
state, please submit it on github so people can test it, and maybe take
over the supporting lib switch if you don't.

Thanks !
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

[collectd] help using write_redis

[Marc Fournier]

Hello,

Is someone successfully using the write_redis plugin ? I can't make any
sense of the data stored in redis by this plugin. A snippet of code
(in any language) that handles the data pulled out of redis would be a
great help.

Thanks !
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] curl_jason plugin is not sending data to rabbitMQ server

[Marc Fournier]

Excerpts from Mariano González's message of 2014-02-10 16:59:25 +0100:
 Hi Marc!
 
 Here's my request/answer using curl -v:
 
 $ curl -v -H X-Auth-Token: my_token https://rackspace_api.url/current |
 [...]
  GET /v1.0/account_id/loadbalancers/lb_id/usage/current HTTP/1.1
  User-Agent: curl/7.33.0
  Host: rackspace_api.url
  Accept: */*
  X-Auth-Token: my_token

So here are the exact HTTP headers the curl CLI is sending to the server ^^

 But, I got a doubt in the next item, now I've this set up in collectd.conf
 
 Plugin curl_json
   URL https://rackspace_api.url/usage/current;
Instance Load_Balancer
 Header X-Auth-Token: my_token

And here is the only header we are sure collectd is sending ^^

My suggestion was to try to add the other headers to see if this can help.

My first bet would be on Accept: */*. I had a similar problem in the past
with curl_json on a jetty-based service, but just guessing...

Hope this helps !

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Add debug symbols to compiled collectd binary

[Marc Fournier]

Excerpts from Christopher Fitzner's message of 2014-01-28 10:33:38 +0100:
 Hey,
 
 can anyone give me any tips on compiling the collectd binary with debug
 symbols and then use gdb to load it for plugin debugging?  I've run the
 configure script with these options: CFLAGS=-g -O0 as suggested on the
 homepage but when I try to run the compiled binary It looks like some of
 the data is in .libs/collectdS.o which is deleted near the end of the make
 process.
 
 Messing around I made a copy of the file before it was deleted to load up
 however even with that file I'm having trouble get gdb to stop anywhere
 useful to me.  I try to set a break point at one of my functions and it
 says theres no function with that name.

Doesn't ./configure --enable-debug [...] do the job ?

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] curl_jason plugin is not sending data to rabbitMQ server

[Marc Fournier]

Excerpts from Mariano González's message of 2014-01-31 15:01:13 +0100:
 Hi there.
 I've implemented curl_jason plugin to recolect and send LoadBalancer
 metrics to my RabbitMQ to be graphed in Graphite.
 Thing is, it's not sending any data, while it is working just fine (and
 great) with other plugins like memory, cpu, df root, network, etc. I've
 tried to troubleshoot following this suggestion:
 http://serverfault.com/questions/499378/collectd-stores-nan-instead-of-correct-value-in-ubuntu-12-04,
 but there're no issues coming out.
 
 Here's my collectd.conf: https://gist.github.com/Mariano-gon/8732467
 
 Here're the last lines of collectd.log when I start it:
 https://gist.github.com/Mariano-gon/8732488
 
 The request is made against Rackspace API where my LoadBalancer is located,
 and if run manually, the curl gets me a json response perfectly normal.

Just a random guess: does the rackspace API require you to send an Accept
encoding header ? By default collectd doesn't send any.

Try tcpdumping the HTTP traffic and compare what headers collectd sends vs
when you run curl manually.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] How to contribute my code ?

[Marc Fournier]

Hello,

Excerpts from Dennis Zou (yunzou)'s message of 2014-01-17 10:36:47 +0100:
 Hi,
 Recently  I  developed a  new plugin, but I'm not so clear about how to 
 commit my code?
 Anyone can point me to the right direction or give  me some clues?

The usual way nowadays is to fork the project on github and send a
pull-request against the master branch:
https://github.com/collectd/collectd/

If you're not familiar with git/github, just send your patch to the
mailing-list for review.

Thanks !

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] write_http behaviour on POST failure?

[Marc Fournier]

Hello,

Excerpts from Otis Gospodnetic's message of 2013-12-12 04:41:46 +0100:
 Hi,
 
 What does https://collectd.org/wiki/index.php/Plugin:Write_HTTP do when the
 destination (there data is being POSTed) is either not up or returns a
 non-200 or times out or ... ?
 
 Is the data dropped and lost?
 Or is it written somewhere to be sent later?

It will be lost. An error will be emitted if you have logging enabled.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] time series data in between specific time periods.

[Marc Fournier]

Excerpts from Kaushal Shriyan's message of 2013-11-18 14:16:41 +0100:
 Hi,
 
 I have setup collectd on CentOS 6.4. Are there options to select time range
 for example 15/05/2013 to 23/05/2013?  I mean to look for time series data
 in between this period

This really depends on what storage backend you are saving your data into.

If you are running a recent version of collectd, there are at least a dozen
output plugins, several of which have the time-range feature you're looking
for: graphite, postgresql, mongodb, etc.

Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Collecd ping syslog

[Marc Fournier]

Excerpts from DeCamps, Miguel's message of 2013-07-23 22:12:12 +0200:
 Hello everyone
 
 I am trying to figure out how I can get success pings from the ping
 plugin in the collectd agent to the local syslog. I can clearly see when
 a ping does not succeed a log entry in /var/log/syslog; but upon recovery
 of the pings I do not see a log entry into my syslog file.
 
 I need this so I can integrate it into my OMD instance for remote agents
 I have distributed. Any help would be greatful!

There isn't any notion of failed/recovered state. If a ping fails, a log
message is emitted. If it succeeds, nothing is logged. That's all.

I don't know about how OMD works, but would I typically check the values
emitted by the ping plugin (from one of the available write plugins, or
using collectd's threshold system) and raise an alarm if the value is
above/below a certain level.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] processes plugin process by name - percent of all cpus feature

[Marc Fournier]

Hello,

Excerpts from William Salt's message of 2013-06-17 02:00:35 +0200:
 Im using the processes plugin to find specific processes via regex, and
 monitoring their cpu usage, this is the total time spent in jiffies
 (milliseconds) across all cores.
 
 In graphite, i am dividing it by the number of cpu cores i have on a
 machine then turning that into percent. this gives me a percentage of total
 cpu usage across all cores for that process.
 However, i cannot ascertain in graphite the exact number of cpus a node
 has, it differs greatly, and its not scalable to have static values in
 graph query strings like this, nor efficient to work this out when
 rendering a graph.

Still not very nice and efficient, but you can use a variant of this[1] to
count the number of cpus using graphite functions. IE: this would return
the number of CPUs as seen by collectd's spu plugin:
sumSeries(offset(scale(collectd.your_hostname.cpu-*.cpu-idle, 0), 1))

Another thing that comes to mind: since collectd 5.3 there's an
aggregation plugin, which you can use to sum up the metrics of every cpu
on each host. You could then use these aggregated metrics for your
percentage calculation. There's a similar feature in graphite if you run
the carbon-aggregator in front of carbon-cache.

 I wondered if someone could patch the plugin to allow a flag to produce the
 average cpu use across all cores? grabbing the number of cpus
 from /proc/cpuinfo and dividing the metric by that? bonus if it had a flag
 to output a percent!

Vedran Bartonicek has 2 pull-requests doing this sort of thing for the load
and df plugins. Have look at PRs #343 and #344 on
github.com/collectd/collectd

Marc

[1] 
http://obfuscurity.com/2013/05/Graphite-Tip-Counting-Number-of-Metrics-Reported

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Patch : contrib/redhat/collectd.spec

[Marc Fournier]

Hello Bruno,

Excerpts from Bruno Mairlot's message of 2013-01-02 15:29:23 +0100:
 
 You'll find bellow a patch to apply on the collectd.spec file of the 
 original archive 5.2.0
 
 I have worked on a new .spec file to allow for building RPMs for 
 CentOS/RHEL, it should probably work on Fedora too, although I haven't 
 tested (yet).
 
 I tried to build it with as much plugin possible, but I limited them to 
 those that could be built with the standard package from RedHat-Base (= 
 CentOS-Base) in order to keep it the most lightweight possible. It could 
 eventually be adjusted to allow for more flexibility or create multiple 
 subpackage with the specific plugin but I don't know enough about the 
 the plugins hierarchy to do that right now.

There has been a common effort going forward for a couple of weeks about
this RPM specfile: https://github.com/collectd/collectd/pull/220

AFAICT everything you included in your patch is already part of this
pull-request. I warmly invite you to join the effort, test this specfile on
your platform/environment and give feedback if it doesn't match your needs.

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Filter chains and GenericJMX plugin. Help Needed

[Marc Fournier]

Hello,

Excerpts from Josh Behrends's message of Fri Jun 22 22:52:33 +0200 2012:
 Hello,
 
 I'm looking for some help!  I'm using Collectd 5.1.0 to collect JMX metrics
 using the GenericJMX plugin and then sending the metrics over to Graphite.
   The issue I'm having is I need to rename the metric Type that's being
 sent to graphite.  The format Collectd is currently sending:
  MachineName.GenericJMX.InstanceName.Type.TypeInstance  The Type
 [...]

I have no experience with collectd's Filter mechanism, but I've done
something similar to what you describe using carbon's rewrite feature.

Have a look at conf/rewrite-rules.conf.example in your carbon installation.
NB: you'll need to have the carbon aggregator service running to use this.

Hope this helps !

Cheers,
Marc

___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

[collectd] [PATCH 1/3] Varnish plugin: added missing fields up to varnish 2.1.5.

[Marc Fournier]

Signed-off-by: Marc Fournier marc.fourn...@camptocamp.com
---
 src/collectd.conf.in |7 ++
 src/types.db |6 +
 src/varnish.c|  255 +
 3 files changed, 164 insertions(+), 104 deletions(-)

diff --git a/src/collectd.conf.in b/src/collectd.conf.in
index 94cf2a9..f1cc29d 100644
--- a/src/collectd.conf.in
+++ b/src/collectd.conf.in
@@ -932,14 +932,21 @@
 #  CollectCache true
 #  CollectBackend true
 #  CollectConnections true
+#  CollectDirectorDNS false
 #  CollectSHM true
 #  CollectESI false
 #  CollectFetch false
 #  CollectHCB false
+#  CollectObjects false
+#  CollectPurge false
+#  CollectSession false
 #  CollectSMA false
 #  CollectSMS false
 #  CollectSM false
+#  CollectStruct false
 #  CollectTotals false
+#  CollectUptime false
+#  CollectVCL false
 #  CollectWorkers false
 #   /Instance
 #/Plugin
diff --git a/src/types.db b/src/types.db
index e6345ab..5248ace 100644
--- a/src/types.db
+++ b/src/types.db
@@ -6,6 +6,7 @@ apache_requests value:DERIVE:0:U
 apache_scoreboard  value:GAUGE:0:65535
 ath_nodes  value:GAUGE:0:65535
 ath_stat   value:DERIVE:0:U
+backends   value:GAUGE:0:65535
 bitratevalue:GAUGE:0:4294967295
 bytes  value:GAUGE:0:U
 cache_operationvalue:DERIVE:0:U
@@ -103,6 +104,7 @@ node_octets rx:DERIVE:0:U, tx:DERIVE:0:U
 node_rssi  value:GAUGE:0:255
 node_stat  value:DERIVE:0:U
 node_tx_rate   value:GAUGE:0:127
+objectsvalue:DERIVE:0:U
 operations value:DERIVE:0:U
 percentvalue:GAUGE:0:100.1
 pg_blksvalue:DERIVE:0:U
@@ -129,6 +131,7 @@ ps_rss  
value:GAUGE:0:9223372036854775807
 ps_stacksize   value:GAUGE:0:9223372036854775807
 ps_state   value:GAUGE:0:65535
 ps_vm  value:GAUGE:0:9223372036854775807
+purge_operationvalue:DERIVE:0:U
 queue_length   value:GAUGE:0:U
 recordsvalue:GAUGE:0:U
 requests   value:GAUGE:0:U
@@ -137,12 +140,14 @@ route_etx value:GAUGE:0:U
 route_metric   value:GAUGE:0:U
 routes value:GAUGE:0:U
 serial_octets  rx:DERIVE:0:U, tx:DERIVE:0:U
+session_operation  value:DERIVE:0:U
 signal_noise   value:GAUGE:U:0
 signal_power   value:GAUGE:U:0
 signal_quality value:GAUGE:0:U
 snrvalue:GAUGE:0:U
 spam_check value:GAUGE:0:U
 spam_score value:GAUGE:U:U
+struct value:DERIVE:0:U
 swap_iovalue:DERIVE:0:U
 swap   value:GAUGE:0:1099511627776
 tcp_connectionsvalue:GAUGE:0:4294967295
@@ -161,6 +166,7 @@ total_time_in_msvalue:DERIVE:0:U
 total_values   value:DERIVE:0:U
 uptime value:GAUGE:0:4294967295
 users  value:GAUGE:0:65535
+vclvalue:GAUGE:0:65535
 vcpu   value:GAUGE:0:U
 virt_cpu_total value:DERIVE:0:U
 virt_vcpu  value:DERIVE:0:U
diff --git a/src/varnish.c b/src/varnish.c
index 859df21..99f224a 100644
--- a/src/varnish.c
+++ b/src/varnish.c
@@ -23,110 +23,6 @@
  *   Florian octo Forster octo at verplant.org
  **/
 
-/**
- * Current list of what is monitored and what is not monitored (yet)
- * {{{
- * Field name   Description   Monitored
- * --   ---   -
- * uptime   Child uptime  N
- * client_conn  Client connections accepted   Y
- * client_drop  Connection dropped, no sess   Y
- * client_req   Client requests received  Y
- * cache_hitCache hitsY
- * cache_hitpassCache hits for pass   Y
- * cache_miss   Cache misses  Y
- * backend_conn Backend conn. success Y
- * backend_unhealthyBackend conn. not attempted   Y
- * backend_busy Backend conn. too manyY
- * backend_fail Backend conn. failuresY
- * backend_reuseBackend conn. reuses  Y
- * backend_toolate  Backend conn. was closed  Y
- * backend_recycle  Backend conn. recyclesY
- * backend_unused   Backend conn. unused  Y
- * fetch_head   Fetch headY
- * fetch_length Fetch with Length Y
- * fetch_chunkedFetch chunked Y
- * fetch_eofFetch EOF

Re: [collectd] Varnish plugin

[Marc Fournier]

Hello,

On Sun, Jun 27, 2010 at 08:41:19AM +0200, Jerome Renard wrote:
 Hi Marc,
 
 [...]

  You'll find an implementation suggestion in this branch:
 
  http://github.com/mfournier/collectd/commits/varnish-data-reorg
 
 I have applied your patches and I had to modifiy some minor things [1].

I agree meaning is more eloquent with this change, but it breaks the
consistency where the names used to matche the names from varnishstat ...

 
   - would it make more sense to name the directories after the configuration
    option ? ie: CollectWorkers - varnish-instance-workers. They actually
    match the varnish values names (same than returned by varnishstat -l).
 
 Sounds good to me.

... but in any case it's probably more comprehensive to have the
directories match the name of the configuration items. I've pushed a
trivial patch renaming a couple of values this way onto my github branch,
as we seem to agree on this.

 Data really looks better organized but I am still unsure this is the way
 to go.

Well, it's should at least be easier to make useful graphs with collection3
now :-)

Cheers,
Marc


___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Varnish plugin

[Marc Fournier]

Hello,

 On Fri, Jun 11, 2010 at 10:51:03AM +0200, Florian Forster wrote:
  
  I'm currently thinking about moving information from the type instance
  to the plugin instance. [...]
  
  If we move some of the information into the plugin instance, the data is
  structured better:
  
host/varnish-default-storage-file/...
  .../total_requests
  .../requests-outstanding
  .../bytes-allocated
  .../bytes-free
host/varnish-default-storage-mem/...
  .../total_requests
  .../requests-outstanding
  .../bytes-outstanding
  .../bytes-allocated
  .../bytes-free
host/varnish-default-storage-synth/...
  .../total_requests
  .../requests-outstanding
  .../bytes-outstanding
  .../bytes-allocated
  .../bytes-free
 
 You'll find an implementation suggestion in this branch:
 
 http://github.com/mfournier/collectd/commits/varnish-data-reorg

As requested, I changed the strncat/sstrncpy functions for one ssnprintf.
The resulting behaviour remains the same as before.

I also pushed another patch adding a whitespace in front of each opening
bracket, as it seems to be the coding style used in the rest of the
project.

Cheers,
Marc



signature.asc
Description: Digital signature
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

[collectd] issue on shutdown with 4.10

[Marc Fournier]

Hello,

On systems running 4.10.0 or 37e18082, I discovered that the socket created
by the unixsocket plugin was sometimes not removed on shutdown, which
prevents the plugin from working with this error at the next restart:

  unixsock plugin: bind failed: Address already in use

This problem is non-systematic. I tried to strace collectd, but didn't find
any obvious error. But I did notice that when the socket fails to get
removed, strace exits with an error:

$ strace -f -s0 -o /tmp/strace.txt -p 30699
Process 30699 attached with 10 threads - interrupt to quit
Process 30708 detached
Process 30709 detached
Process 30707 detached
Process 30705 detached
Process 30704 detached
PANIC: handle_group_exit: 30703 leader 30699
PANIC: handle_group_exit: 30702 leader 30699
PANIC: handle_group_exit: 30701 leader 30699
PANIC: handle_group_exit: 30700 leader 30699

When the socket gets removed, strace ends without panicking.

NB: I checked file and dir modes, collectd can create/remove the socket
file without restriction.

Any suggestions how I could dig further in order to find out what's going
on ?

Thanks !
Marc


___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Varnish plugin

[Marc Fournier]

Hello,

 Other than that, there are two new features: The configure script now
 uses pkg-config to determine the correct C- and LD-flags. Unfortunately,
 the Debian package is broken in this respect so that building under
 Debian currently needs some hands-on hacking fixing. I opened bug
 reports but I doubt this will be fixed soon – a similar bug report has
 been open since November 2009 :(

I'm no longer able to build the varnish plugin using your branch. I suppose
this is related to this issue you mention ?

A few relevant pieces below:

$ git checkout 24da3d76
Previous HEAD position was e73f418... Varnish plugin: Added SM Monitor
HEAD is now at 24da3d7... configure.in: Use pkg-config to check for 
libvarnishapi.
$ ./build.sh
+ autoheader
+ aclocal
+ libtoolize --ltdl --copy --force
+ automake --add-missing --copy
+ autoconf

$ ./configure --prefix /opt/collectd --disable-all-plugins --enable-varnish
[...]
libvarnish  . . . . . no (symbol VSL_OpenStats not found)
[...]
varnish . . . . . . . no (dependency error)
[...]
configure: error: Some plugins are missing dependencies - see the summary 
above for details

$ ./configure --prefix /opt/collectd --disable-all-plugins --enable-varnish 
--with-libvarnish=/usr/include/
[...]
libvarnish  . . . . . /usr/include/
[...]
varnish . . . . . . . no (dependency error)
[...]
configure: error: Some plugins are missing dependencies - see the summary 
above for details

$ rpm -ql varnish-libs-devel-2.1.2-1.el5
/usr/include/varnish
/usr/include/varnish/shmlog.h
/usr/include/varnish/shmlog_tags.h
/usr/include/varnish/stat_field.h
/usr/include/varnish/stats.h
/usr/include/varnish/varnishapi.h
/usr/lib64/libvarnish.so
/usr/lib64/libvarnishapi.so
/usr/lib64/libvarnishcompat.so
/usr/lib64/libvcl.so
/usr/lib64/pkgconfig/varnishapi.pc
/usr/share/doc/varnish-libs-devel-2.1.2
/usr/share/doc/varnish-libs-devel-2.1.2/LICENSE

$ pkg-config --libs varnishapi
-lvarnishapi

$ grep VSL_OpenStats /usr/include/varnish/*
/usr/include/varnish/varnishapi.h:struct varnish_stats *VSL_OpenStats(const 
char *varnish_name);


This problem occurs on RHEL 5.5 (latest stable release) as well as debian
squeeze (testing).

Can you elaborate on the hands-on fixing you mentioned ?


 Unfortunately I'm flying blind because I don't have a Varnish instance
 running anywhere. It'd be awesome if you could test the changes for me.

I was about to do that, when I run into this build problem ;-)
I'll test Jérôme's branch meanwhile and let you know of any other issues I
encounter.

Marc



signature.asc
Description: Digital signature
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Varnish plugin

[Marc Fournier]

 could you look for relevant information in config.log, the file
 created by the configure script? Something like
 
   grep -C 10 varnish config.log

Here you are:

| #define HAVE_PTHREAD_H 1
| #define HAVE_LIBPTHREAD 1
| #define HAVE_RRD_H 1
| #define HAVE_THREADSAFE_LIBRRD 1
| #define HAVE_LIBXMMS 0
| /* end confdefs.h.  */
| #include yajl/yajl_parse.h
configure:43973: result: no
configure:44008: checking for yajl/yajl_parse.h
configure:44015: result: no
configure:44178: Checking for varnishapi using /usr/bin/pkg-config
configure:44224: checking varnish/varnishapi.h usability
configure:44236: gcc -c -g -O2  -I/usr/include/varnish   conftest.c 5
configure:44242: $? = 0
configure:44246: test -z 
 || test ! -s conftest.err
configure:44249: $? = 0
configure:44252: test -s conftest.o
configure:44255: $? = 0
configure:44265: result: yes
configure:44269: checking varnish/varnishapi.h presence
configure:44279: gcc -E  -I/usr/include/varnish   conftest.c
configure:44285: $? = 0
configure:44305: result: yes
configure:44340: checking for varnish/varnishapi.h
configure:44347: result: yes
configure:44373: checking for VSL_OpenStats in -lvarnishapi
configure:44403: gcc -o conftest -g -O2  -I/usr/include/varnishconftest.c 
-lvarnishapi -lvarnishapi5
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnishapi.so: 
undefined reference to `VRE_compile'
/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnishapi.so: 
undefined reference to `VRE_exec'
collect2: ld returned 1 exit status
configure:44409: $? = 1
configure: failed program was:
| /* confdefs.h.  */
|
| #define PACKAGE_NAME collectd
| #define PACKAGE_TARNAME collectd
| #define PACKAGE_VERSION 4.10.0.36.g25e6243
| #define PACKAGE_STRING collectd 4.10.0.36.g25e6243
| #define PACKAGE_BUGREPORT 
--
config.status:1120: creating src/collectd.conf
config.status:1120: creating src/owniptc/Makefile
config.status:1120: creating src/libcollectdclient/Makefile
config.status:1120: creating src/libcollectdclient/libcollectdclient.pc
config.status:1120: creating src/liboconfig/Makefile
config.status:1120: creating bindings/Makefile
config.status:1120: creating bindings/java/Makefile
config.status:1224: creating src/config.h
config.status:1874: executing depfiles commands
configure:55091: configuring in libltdl
configure:55208: running /bin/sh './configure' --prefix=/opt/collectd  
'--prefix' '/opt/collectd' '--disable-all-plugins' '--enable-varnish' 
--enable-ltdl-convenience --cache-file=/dev/null --srcdir=.
configure:55409: error: Some plugins are missing dependencies - see the 
summary above for details

##  ##
## Cache variables. ##
##  ##

ac_cv_build=x86_64-redhat-linux-gnu
ac_cv_build_alias=x86_64-redhat-linux-gnu
ac_cv_c_compiler_gnu=yes
ac_cv_c_const=yes
--
ac_cv_header_sys_vfs_h=yes
ac_cv_header_sys_vfstab_h=no
ac_cv_header_sys_wait_h=yes
ac_cv_header_syslog_h=yes
ac_cv_header_tcrdb_h=no
ac_cv_header_termios_h=yes
ac_cv_header_time=yes
ac_cv_header_unistd_h=yes
ac_cv_header_utmp_h=yes
ac_cv_header_utmpx_h=yes
ac_cv_header_varnish_varnishapi_h=yes
ac_cv_header_vm_anon_h=no
ac_cv_header_wordexp_h=yes
ac_cv_header_xfs_xqm_h=no
ac_cv_header_yajl_yajl_parse_h=no
ac_cv_host=x86_64-redhat-linux-gnu
ac_cv_host_alias=x86_64-redhat-linux-gnu
ac_cv_lib_IOKit_IOServiceGetMatchingServices=no
ac_cv_lib_curl_curl_easy_init=yes
ac_cv_lib_dl_dlopen=yes
ac_cv_lib_dld_shl_load=no
--
ac_cv_lib_kvm_kvm_getprocs=no   
ac_cv_lib_kvm_kvm_getswapinfo=no
ac_cv_lib_kvm_kvm_nlist=no
ac_cv_lib_kvm_kvm_openfiles=no  
ac_cv_lib_oconfig_oconfig_parse_fh=no
ac_cv_lib_pcap_pcap_open_live=no
ac_cv_lib_pthread_pthread_create=yes
ac_cv_lib_resolv_res_search=no  
ac_cv_lib_rrd_th_rrd_update_r=yes
ac_cv_lib_rrd_th_rrdc_update=no 
ac_cv_lib_varnishapi_VSL_OpenStats=no
ac_cv_lib_xml2_xmlXPathEval=yes 
ac_cv_member_kstat_io_t_nwrites=no
ac_cv_member_kstat_io_t_nwritten=no
ac_cv_member_kstat_io_t_writes=no
ac_cv_member_kstat_io_t_wtime=no
ac_cv_member_struct_if_data_ifi_ibytes=no
ac_cv_member_struct_if_data_ifi_ierrors=no
ac_cv_member_struct_if_data_ifi_opackets=no
ac_cv_member_struct_ip_mreqn_imr_ifindex=yes
ac_cv_member_struct_kinfo_proc_ki_pid=no



  /usr/lib64/libvarnish.so
  /usr/lib64/libvarnishapi.so
  /usr/lib64/libvarnishcompat.so
 
 Also, an ldd of each of those files would be interesting for the RHEL
 box. On Debian Squeeze I know it's broken :/

/usr/lib64/libvarnish.so:
librt.so.1 = /lib64/librt.so.1 (0x003ebce0)
libnsl.so.1 = /lib64/libnsl.so.1 (0x003ebe20)
libm.so.6 = /lib64/libm.so.6 (0x003ebbe0)
libpcre.so.0 = /lib64/libpcre.so.0 (0x0034f9a0)
libc.so.6 = /lib64/libc.so.6 (0x003ebb20)
libpthread.so.0 = /lib64/libpthread.so.0 (0x003ebca0)
/lib64/ld-linux-x86-64.so.2 (0x003ebae0)
/usr/lib64/libvarnishapi.so:
libc.so.6 = /lib64/libc.so.6 (0x003ebb20)

Re: [collectd] Varnish plugin

[Marc Fournier]

Jérôme,

 can you please pull the changes from my branch:
 - http://github.com/jeromer/collectd/
 
 I just pushed Florian's patches and if you follow what I said in my
 previous email [1] about pkg-config you
 should be able to compile the Varnish plugin on your Debian (I did not
 try on RHEL yet, sorry).

As soon as you wrote about this -lvarnishcompat trick, I was able to build
the varnish plugin on redhat 5.5 without any trouble. Later I even applied
the patch to the 4.9 branch and built it successfully. Thanks for figuring
that out !

I understood that the main varnish developer was also a freebsd developer.
This might explain why strlcpy is used, and the reason of this
varnishcompat library for linux...

In any case, the plugin seems to work fine. Values are written to RRD files
every now and then, and I didn't notice any coredumps or dreadful log
messages.

I'll continue to play around with this plugin until I have graphs of my
varnish production machines. So I'll let you know of any other issue I come
across.

Thanks !
Marc


___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd

Re: [collectd] Varnish plugin

[Marc Fournier]

Hello Florian,

Thanks for your help and explanations ! I definitely learned a couple of
things today :-)

  /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64/libvarnish.so: 
  undefined reference to `strlcpy'
 
 That's new. I thought the GNU libc did implement strlcpy, too..?

According to wikipedia, it's not the case:
http://en.wikipedia.org/wiki/Strlcpy#Criticism

Cheers,
Marc



signature.asc
Description: Digital signature
___
collectd mailing list
collectd@verplant.org
http://mailman.verplant.org/listinfo/collectd