Re: [Puppet Users] Re: Still no nagios joy
On 10/4/13 1:39 PM, Deepak Giridharagopal wrote: So, puppet agent --test --environment production fails to create resources findable via REST (and in fact removes them if they're present), but puppet apply foo.pp (with foo.pp containing the statements above) makes them findable (and they show up in nagios_hosts.cfg on the nagios server). PuppetDB will only return resources for nodes which it haven't been marked inactive...is it possible that's happened for some of your nodes? Hitting http://localhost:8080/v2/nodes on your puppetdb server should dump out everything puppetdb knows about your hosts, including when they were deactivated (or null if they've not been deactivated). All the nodes show up via the URL above with null deactivation dates. -- David Thompson Waisman Center Brain Imaging and Behavior Lab 1500 Highland Ave. Room T133 Madison, WI 53705-2280 (608) 265-6608 dthompson (at) waisman (dot) wisc (dot) edu -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On Thursday, October 3, 2013 10:48:10 AM UTC-5, David Thompson wrote: On 10/3/13 9:07 AM, jcbollinger wrote: So, puppet agent --test --environment production fails to create resources findable via REST (and in fact removes them if they're present), but puppet apply foo.pp (with foo.pp containing the statements above) makes them findable (and they show up in nagios_hosts.cfg on the nagios server). With that, I'm ready to call bug. Nevertheless, I hope to be able to help you find a viable workaround, and perhaps to characterize the issue better. though I don't expect using that form will change anything. I'm more interested, though, in the back end. That is, is puppetdb for some reason failing to serve up resources that are in fact recorded in the back-end DB, or are the resources either not making it to the DB in the first place, or are they being purged from the DB between being entered and being queried? That's a very good question. I can certainly find catalog_resource and resource_params records that *look* like they contain appropriate Nagios_host resources, but the REST API says no, and if I run the actual sql query out of the postgres statement log, it says no. The sql query itself is a complex enough to be beyond my db-mojo to understand why it's not finding any Nagios_host resources. If there's any way you can provide the relevant data and query, I daresay someone around here -- maybe me -- will be able to figure out why the query doesn't return any rows. But that might be better deferred, as I think there may be another avenue to pursue first. It looks like enlightenment lies in understanding the difference between the puppet agent and puppet apply runs. Indeed so. You have or can create statement logs showing what happens when you use Puppet apply to export the Nagios_host resources such that they can later be found, and also showing what happens when the same resources are exported as part of catalog compilation. There must be some difference, hopefully more than the identity of the db client machine. That seems a logical place to look for clues. John -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On 10/4/13 8:52 AM, jcbollinger wrote:
It shouldn't make a difference, I don't think, but how are those
declarations positioned in your original code? Are they in a class /
definition / node block, or are they at top level?
In the file modules/nagios_config/manifests/init.pp, I have the
functional equivalent of:
class nagios_config {
if $im_supposed_to_be_monitored {
notify {Nagios Base: $fqdn $hostname $ipaddress: }
@@nagios_host { $fqdn:
ensure = present,
alias = $hostname,
address = $ipaddress,
use = linux-server,
}
}
}
...then in site.pp 'include nagios_config'.
Also, is there any chance of resource parameter overrides being in
play? Specifically, overrides of the Nagios_hosts' 'ensure' parameters?
Nope. I don't do any resource overrides in any of my manifests.
--
David Thompson
Waisman Center Brain Imaging and Behavior Lab
1500 Highland Ave. Room T133
Madison, WI 53705-2280
(608) 265-6608
dthompson (at) waisman (dot) wisc (dot) edu
--
You received this message because you are subscribed to the Google Groups Puppet
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On Thu, Oct 3, 2013 at 9:48 AM, David Thompson
dthomp...@waisman.wisc.eduwrote:
On 10/3/13 9:07 AM, jcbollinger wrote:
And you're certain that $nagios_server evaluates to true on the machines
that nevertheless fail to get the expected Nagios_host resources
applied? You could probe that by changing the above to
if $nagios_server {
Nagios_host | |
notify { 'I am a nagios server': }
} else {
notify { 'I am not a nagios server': }
}
That will also help you catch the case that the class containing the
above is not included in the catalog at all, for then you would see
neither the Notify message in the agent log. You can also watch your
query log for the query by which Puppet retrieves all the wanted
Nagios_host resources; if nothing else is fruitful then the details of
the query might reveal some useful information.
First to answer. I have one computer that runs nagios. That's the (only)
computer that has $nagios_server defined. The computers that I want to
monitor have the @@nagios_host {} resource. I've verified this up and down.
I've done some more testing, and there's something very funny going on.
In my manifests, I have the following:
notify {Nagios Base: $fqdn $hostname $ipaddress: }
@@nagios_host { $fqdn:
ensure = present,
alias = $hostname,
address = $ipaddress,
use = linux-server,
}
On the hosts to be monitored, I see the notify, and, as I've said, I see
the db inserts, but then nothing comes up in the REST query. HOWEVER, if I
take the lines above and put them in a separate .pp file and puppet apply
that file, then I see the resources via the REST query. My understanding
was that once a resource gets defined, that's the end of the story, but
it's clearly not what's happening here.
So, puppet agent --test --environment production fails to create
resources findable via REST (and in fact removes them if they're present),
but puppet apply foo.pp (with foo.pp containing the statements above)
makes them findable (and they show up in nagios_hosts.cfg on the nagios
server).
PuppetDB will only return resources for nodes which it haven't been marked
inactive...is it possible that's happened for some of your nodes? Hitting
http://localhost:8080/v2/nodes on your puppetdb server should dump out
everything puppetdb knows about your hosts, including when they were
deactivated (or null if they've not been deactivated).
though I don't expect using that form will change anything. I'm more
interested, though, in the back end. That is, is puppetdb for some
reason failing to serve up resources that are in fact recorded in the
back-end DB, or are the resources either not making it to the DB in the
first place, or are they being purged from the DB between being entered
and being queried?
That's a very good question. I can certainly find catalog_resource and
resource_params records that *look* like they contain appropriate
Nagios_host resources, but the REST API says no, and if I run the actual
sql query out of the postgres statement log, it says no. The sql query
itself is a complex enough to be beyond my db-mojo to understand why it's
not finding any Nagios_host resources.
You said earlier that some nagios hosts are getting configured, at least
on some machines, but I didn't completely understand which hosts are
getting configured on which machines. Are there machines that get all
the expected hosts? Are there nagios_hosts that get configured on all
machines? More generally, what is the nature of the correlation (that
you have recognized) between declared nagios host resources and the
machines on which puppet manages those resources?
Yes, sometimes I've gotten some partial Nagios_host data generated on the
nagios server, but never in any discernible or repeatable form.
It looks like enlightenment lies in understanding the difference between
the puppet agent and puppet apply runs.
--
David Thompson
Waisman Center Brain Imaging and Behavior Lab
1500 Highland Ave. Room T133
Madison, WI 53705-2280
(608) 265-6608
dthompson (at) waisman (dot) wisc (dot) edu
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
puppet-users+unsubscribe@**googlegroups.compuppet-users%2bunsubscr...@googlegroups.com
.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at
http://groups.google.com/**group/puppet-usershttp://groups.google.com/group/puppet-users
.
For more options, visit
https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out
.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post
Re: [Puppet Users] Re: Still no nagios joy
On Wednesday, October 2, 2013 9:44:14 AM UTC-5, David Thompson wrote:
The puppet nagios collector:
if $nagios_server {
Nagios_host ||
}
And you're certain that $nagios_server evaluates to true on the machines
that nevertheless fail to get the expected Nagios_host resources applied?
You could probe that by changing the above to
if $nagios_server {
Nagios_host | |
notify { 'I am a nagios server': }
} else {
notify { 'I am not a nagios server': }
}
That will also help you catch the case that the class containing the above
is not included in the catalog at all, for then you would see neither the
Notify message in the agent log. You can also watch your query log for the
query by which Puppet retrieves all the wanted Nagios_host resources; if
nothing else is fruitful then the details of the query might reveal some
useful information.
The query I'm using:
curl -G 'http://localhost:8080/v3/resources' --data-urlencode
'query=[=, type, Nagios_host]'
(I tried updating to the latest-and-greatest puppetdb yesteday. No
change. I get the same from the v2 URI.)
For what it's worth, it looks like you could simplify that to just
curl -G 'http://localhost:8080/v3/resources/Nagios_host'
though I don't expect using that form will change anything. I'm more
interested, though, in the back end. That is, is puppetdb for some reason
failing to serve up resources that are in fact recorded in the back-end DB,
or are the resources either not making it to the DB in the first place, or
are they being purged from the DB between being entered and being queried?
You said earlier that some nagios hosts are getting configured, at least on
some machines, but I didn't completely understand which hosts are getting
configured on which machines. Are there machines that get all the expected
hosts? Are there nagios_hosts that get configured on all machines? More
generally, what is the nature of the correlation (that you have recognized)
between declared nagios host resources and the machines on which puppet
manages those resources?
John
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On 10/3/13 9:07 AM, jcbollinger wrote:
And you're certain that $nagios_server evaluates to true on the machines
that nevertheless fail to get the expected Nagios_host resources
applied? You could probe that by changing the above to
if $nagios_server {
Nagios_host | |
notify { 'I am a nagios server': }
} else {
notify { 'I am not a nagios server': }
}
That will also help you catch the case that the class containing the
above is not included in the catalog at all, for then you would see
neither the Notify message in the agent log. You can also watch your
query log for the query by which Puppet retrieves all the wanted
Nagios_host resources; if nothing else is fruitful then the details of
the query might reveal some useful information.
First to answer. I have one computer that runs nagios. That's the
(only) computer that has $nagios_server defined. The computers that I
want to monitor have the @@nagios_host {} resource. I've verified this
up and down.
I've done some more testing, and there's something very funny going on.
In my manifests, I have the following:
notify {Nagios Base: $fqdn $hostname $ipaddress: }
@@nagios_host { $fqdn:
ensure = present,
alias = $hostname,
address = $ipaddress,
use = linux-server,
}
On the hosts to be monitored, I see the notify, and, as I've said, I see
the db inserts, but then nothing comes up in the REST query. HOWEVER,
if I take the lines above and put them in a separate .pp file and
puppet apply that file, then I see the resources via the REST query.
My understanding was that once a resource gets defined, that's the end
of the story, but it's clearly not what's happening here.
So, puppet agent --test --environment production fails to create
resources findable via REST (and in fact removes them if they're
present), but puppet apply foo.pp (with foo.pp containing the
statements above) makes them findable (and they show up in
nagios_hosts.cfg on the nagios server).
though I don't expect using that form will change anything. I'm more
interested, though, in the back end. That is, is puppetdb for some
reason failing to serve up resources that are in fact recorded in the
back-end DB, or are the resources either not making it to the DB in the
first place, or are they being purged from the DB between being entered
and being queried?
That's a very good question. I can certainly find catalog_resource and
resource_params records that *look* like they contain appropriate
Nagios_host resources, but the REST API says no, and if I run the actual
sql query out of the postgres statement log, it says no. The sql query
itself is a complex enough to be beyond my db-mojo to understand why
it's not finding any Nagios_host resources.
You said earlier that some nagios hosts are getting configured, at least
on some machines, but I didn't completely understand which hosts are
getting configured on which machines. Are there machines that get all
the expected hosts? Are there nagios_hosts that get configured on all
machines? More generally, what is the nature of the correlation (that
you have recognized) between declared nagios host resources and the
machines on which puppet manages those resources?
Yes, sometimes I've gotten some partial Nagios_host data generated on
the nagios server, but never in any discernible or repeatable form.
It looks like enlightenment lies in understanding the difference between
the puppet agent and puppet apply runs.
--
David Thompson
Waisman Center Brain Imaging and Behavior Lab
1500 Highland Ave. Room T133
Madison, WI 53705-2280
(608) 265-6608
dthompson (at) waisman (dot) wisc (dot) edu
--
You received this message because you are subscribed to the Google Groups Puppet
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
[Puppet Users] Re: Still no nagios joy
On Tuesday, October 1, 2013 10:50:42 AM UTC-5, David Thompson wrote:
I continue to be unable to get puppetdb to create nagios resources.
Nitpick: puppetdb does not create resources of any type, in any sense.
That leaves me uncertain whether you are saying that the exported
nagios_host resources you describe next fail to be recorded in the DB, or
whether you are saying that despite being recorded, they are not
subsequently collected.
I
have a very standard nagios host definition:
notify {Nagios Base: $fqdn $hostname $ipaddress: }
@@nagios_host { $fqdn:
ensure = present,
alias = $hostname,
address = $ipaddress,
use = linux-server,
}
...and the notify shows up on subsequent agent runs, so I know the
statement is being activated. I also know that the facts and catalogs
are winding up in the postgres database.
Do you know whether the exported nagios_host resources in particular are
ending up in the DB?
Despite all this, both the nagios collector and the puppetdb rest API
only show the Nagios_host resources generated on the nagios server itself.
Could you be more specific? What are the forms of the collector and query
URIs you are using?
Do you actually have PuppetDB configured as your storeconfigs back end?
Following the docs
(http://docs.puppetlabs.com/puppetdb/latest/connect_puppet_master.html),
you should have
storeconfigs = true
storeconfigs_backend = puppetdb
in the [master] section of the master's puppet.conf, *and* thin_storeconfigs
and async_storeconfigs should be disabled (the default).
John
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On 10/2/13 9:09 AM, jcbollinger wrote:
On Tuesday, October 1, 2013 10:50:42 AM UTC-5, David Thompson wrote:
I continue to be unable to get puppetdb to create nagios resources.
Nitpick: puppetdb does not create resources of any type, in any sense.
That leaves me uncertain whether you are saying that the exported
nagios_host resources you describe next fail to be recorded in the DB,
or whether you are saying that despite being recorded, they are not
subsequently collected.
Yes, it would be better to say that the generated nagios_host.cfg file
contains far fewer entries than are indicated by the number of hosts
containing the nagios_host resource shown below.
I
have a very standard nagios host definition:
notify {Nagios Base: $fqdn $hostname $ipaddress: }
@@nagios_host { $fqdn:
ensure = present,
alias = $hostname,
address = $ipaddress,
use = linux-server,
}
...and the notify shows up on subsequent agent runs, so I know the
statement is being activated. I also know that the facts and catalogs
are winding up in the postgres database.
Do you know whether the exported nagios_host resources in particular are
ending up in the DB?
If I turn on postgres statement logging, I can see e.g.:
LOG: execute S_2: INSERT INTO catalog_resources
(catalog,resource,type,title,tags,exported,file,line) VALUES
($1,$2,$3,$4,$5,$6,$7,$8)
DETAIL: parameters: $1 = 'b0ddc055494f58babc8f0085245517d3424e1e63', $2
= '5b5706ff5fe112b04f3722b5ce65d699c80a0e04', $3 = 'Nagios_host', $4 =
'c101.keck.waisman.wisc.edu', $5 =
'{node,nagios_host,x86_64_sci6,class,nagios,c101.keck.waisman.wisc.edu}', $6
= 't', $7 =
'/systypes/noarch/puppet/production/modules/nagios/manifests/init.pp',
$8 = '125'
(...and then c101 shows up in neither the generated nagios config nor in
the REST query...)
Despite all this, both the nagios collector and the puppetdb rest API
only show the Nagios_host resources generated on the nagios server
itself.
Could you be more specific? What are the forms of the collector and
query URIs you are using?
The puppet nagios collector:
if $nagios_server {
Nagios_host ||
}
The query I'm using:
curl -G 'http://localhost:8080/v3/resources' --data-urlencode
'query=[=, type, Nagios_host]'
(I tried updating to the latest-and-greatest puppetdb yesteday. No
change. I get the same from the v2 URI.)
Do you actually have PuppetDB configured as your storeconfigs back end?
Following the docs
(http://docs.puppetlabs.com/puppetdb/latest/connect_puppet_master.html),
you should have
| storeconfigs = true
storeconfigs_backend = puppetdb
in the [master] section of the master's puppet.conf,/and/ thin_storeconfigs
andasync_storeconfigs should be disabled (the default).
Yes, this is the configuration I have on all the hosts. In addition, I
have the following routes.yaml on the puppet server/puppetdb server (1
host does both):
---
master:
facts:
terminus: puppetdb
cache: yaml
--
David Thompson
Waisman Center Brain Imaging and Behavior Lab
1500 Highland Ave. Room T133
Madison, WI 53705-2280
(608) 265-6608
dthompson (at) waisman (dot) wisc (dot) edu
--
You received this message because you are subscribed to the Google Groups Puppet
Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: Still no nagios joy
On 10/2/13 9:44 AM, David Thompson wrote: Do you know whether the exported nagios_host resources in particular are ending up in the DB? Just to eliminate postgres as a possible problem source, I changed the database.ini and switched (back) to the embedded hsqldb database. Same result with the embedded database. Could you be more specific? What are the forms of the collector and query URIs you are using? curl -G 'http://localhost:8080/v3/resources' --data-urlencode 'query=[=, type, Nagios_host]' Strange that although I can see the record inserts in the catalog_resources and resource_params relations (in either postgres or hsqldb), the query says there is no such resource. -- David Thompson Waisman Center Brain Imaging and Behavior Lab 1500 Highland Ave. Room T133 Madison, WI 53705-2280 (608) 265-6608 dthompson (at) waisman (dot) wisc (dot) edu -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
