[Puppet Users] PuppetDB curl and puppetdbquery tentative to filter nodes on an array type parameter including a value
Hello puppeters,
I have check_mk::host resource exported by my nodes, and this resources
contain a host_tags parameter that is an array.
I am trying to query PuppetDB to filer my nodes to retrieve all the nodes
exporting this Check_mk::Host resources containing a specific value (nat) in the
host_tags array.
My ultime goal is to be able to write something like this in my manifests, to
retrieve all the check_mk hosts that are behind NAT:
--8<--
$nat_nodes = query_nodes('@@Check_mk::Host{host_tags = "nat"}')
--8<--
With the = meaning that the nat tag is included in the host_tags array. (I
would love to have a construction like the in operator of the puppet language
[1]...)
So I started playing with the puppet query module [2], the following works:
--8<--
puppet query nodes "@@Check_mk::Host['node1.domain.tld']"
--8<--
But not the following:
--8<--
puppet query nodes '@@Check_mk::Host{host_tags = "nat"}'
--8<--
So I started looking at the raw PuppetDB API using curl to see if I was able to
find the correct syntaxe.
I was able to filter the nodes on the complete host_tags parameter value:
--8<--
curl --cacert /var/lib/puppet/ssl/certs/ca.pem \
--cert /var/lib/puppet/ssl/certs/puppet.domain.tld.pem \
--key /var/lib/puppet/ssl/private_keys/puppet.domain.tld.pem \
'https://puppet.domain.tld:8081/v4/resources' \
-X GET \
--data-urlencode 'query=["and",
["and",
["=", "type", "Check_mk::Host"],
["=", "exported", true]],
["=", ["parameter", "host_tags" ], [ "wan", "nat", "RedHat" ]]]'
--8<--
The output is something like this:
--8<--
[ {
"tags" : [ "profile::check_mk", "check_mk::host", "baseclass", "agent",
"host", "class", "profile", "check_mk::agent", "node1.domain.tld",
"check_mk", "node" ],
"file" :
"/etc/puppet/environments/production/modules/check_mk/manifests/agent.pp",
"type" : "Check_mk::Host",
"title" : "node1.domain.tld",
"line" : 28,
"resource" : "0405e0d6caa542a1fc1f348f42524f1f9f9e4875c",
"environment" : "production",
"certname" : "node1.domain.tld",
"parameters" : {
"host_tags" : [ "wan", "nat", "RedHat" ]
},
"exported" : true
},
(...)
]
--8<--
Then I saw in the v4 API documentation [3] that the equality operator on arrays
should math if any one of their elements match, so I tried something like this:
--8<--
curl --cacert /var/lib/puppet/ssl/certs/ca.pem \
--cert /var/lib/puppet/ssl/certs/puppet.domain.tld.pem \
--key /var/lib/puppet/ssl/private_keys/puppet.domain.tld.pem \
'https://puppet.domain.tld:8081/v4/resources' \
-X GET \
--data-urlencode 'query=["and",
["and",
["=", "type", "Check_mk::Host"],
["=", "exported", true]],
["=", ["parameter", "host_tags" ], "nat"]]'
--8<--
But it does not return me anything, the previously matched nodes are not
returned, is such a filtering possible?
Was someone able to do something like this?
If someone have any idea on how to proceed with this... Thanks in advance!
Refs:
[1]- https://docs.puppetlabs.com/puppet/3.7/reference/lang_expressions.html#in
[2]- https://github.com/dalen/puppet-puppetdbquery
[3]-
https://docs.puppetlabs.com/puppetdb/2.2/api/query/v4/operators.html#equality
Best,
Baptiste
--
\,,/_[-_-]_\,,/
"What has not been examined impartially has not been well
examined. Skepticism is therefore the first step toward truth."
[Denis Diderot, "Pensees philosophiques"]
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/20141015170259.GQ6530%40monster.maatg.fr.
For more options, visit https://groups.google.com/d/optout.
Re: [Puppet Users] Re: chaining of create_resources
Le 09/04/14 à 15:15, jcbollinger téléscripta : > Other reports suggest so, but there are other possibilities, such as the > graph not going with the version of the manifests and/or data presented. That's why I provided a complete vagrant project allowing to easily and confidently test such things. > I am not inclined to set up a test environment for your code, but all past > reports -- of which there have been several -- are consistent that resource > references cannot be expressed in Hiera data. And that's perfectly > sensible. OK. As it is very easy with vagrant, I did destroy the VM and restart from scratch (I am using a debian base box from puppetlabs, with puppetlabs and plopopertaions puppet modules) and added two more resources (postgresql package and service) and the dependency are present in the graphs files: https://github.com/gnubila-france/puppet-vagrant-playground/blob/master/graphs/relationships.png In case anyone is interested in checking or testing this I added the graphs files (.dot and .png) to the github repository https://github.com/gnubila-france/puppet-vagrant-playground The README should be sufficient, only vagrant and builder/ruby/rubygems (or puppet and r10k) are needed, and most of the things are documented and automatized, there is just a documented workaround for an annoying bug with VirtualBox 4.3.10. > John Best, Baptiste -- \,,/_[-_-]_\,,/ There was a young lady named Riddle Who had an untouchable middle. She had many friends Because of her ends, Since it isn't the middle you diddle. pgp66zP19q8ow.pgp Description: PGP signature
Re: [Puppet Users] Re: Unable to define order of modules
Hello,
Le 07/11/11 à 15:30, jcbollinger téléscripta :
> For the record, that is not an implementation of my recommendation.
> It is not safe for class sudo::apdater to include that new class,
> except in the most technical of senses.
Indeed.
> The point is to have a source of truth separate from apdater::client
> for that class and class sudo both to draw on, but you have that
> relationship reversed on one side. That just lengthens the dependency
> chain you already had (now sudo depends on newclass depends on
> apdater::client), which doesn't gain you anything. For this to work
> you need {sudo, apdater::client} depends on newclass.
>
> Replace "newclass" with "global variable" or "external data" as you
> like.
OK.
> > > The separate class option is great, but it is distinct from the other
> > > two only if the new class can choose the variable's value based only
> > > on nodes' facts.
> >
> > It does not seem to be possible for me.
>
> Whatever logic you use now either to set $sudo_allow_aptdater_user or
> to choose whether to include class apdater::client can certainly be
> put in a separate class. Depending on what the logic is, it might be
> ugly / unmaintainable / painful to move it there, but not impossible.
For now the logic is that I manually add the adapter::client to the
concerned nodes, so it is quite difficult to put this as it is in
another class. (I am sure it would be possible, I can have the separate
class that extlookup a file to see if the node should have the
adapter::client class, but now, for my present need, it seems to be too
much)
> > By the way I was able to order the module application using run stages,
> > so thanks for pointing me to this direction.
> > I don't find this to be the perfect/cleanest solution for my problem,
> > but at least it is working as needed and expected :)
>
> I am glad you have found a solution.
Thanks for your time/help.
> Good luck,
Do I look so lost? ;)
> John
Regards,
Baptiste
--
\,,/_[-_-]_\,,/
http://asocial.ws/gwarf
Down to the Banana Republics,
Down to the tropical sun.
Go the expatriated Americans,
Hoping to find some fun.
Some of them go for the sailing,
Caught by the lure of the sea.
Trying to find what is ailing,
Living in the land of the free.
Some of them are running from lovers,
Leaving no forward address.
Some of them are running tons of ganja,
Some are running from the IRS.
Late at night you will find them,
In the cheap hotels and bars.
Hustling the senoritas,
While they dance beneath the stars.
-- Jimmy Buffet, "Banana Republics"
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Unable to define order of modules
Hello,
Le 03/11/11 à 15:45, jcbollinger téléscripta :
> On Nov 2, 9:57 am, Baptiste Grenier wrote:
> > Le 31/10/11 16:09, jcbollinger t l scripta :
> > > On Oct 28, 10:07 am, Baptiste Grenier
> > > wrote:
> > > You have an evaluation order problem, and you have tried to fix it
> > > by constraining the application order. Instead, you want something like
> > > this:
> >
> > > [...]
> >
> > > class sudo {
> > > # KEY CHANGE:
> > > include "aptdater::client"
> >
> > The problem is that I want to be able to include the sudo class without
> > including the aptdater::client class: some node can have sudo managed,
> > but won't be configured for apt-dater.
>
> There are several ugly ways you could do this. The clean way would be
> to pull the $aptdater::client::sudo_allow_aptdater_user variable out
> to a separate class that both class sudo and class aptdater::client
> can safely include (which implies that it is not parameterized),
I tried it quickly, with a test like this in the separate class:
-8<
if (inline_template("<%= classes.include?('aptdater::client') %>") == 'true')
{
$sudo_allow_aptdater_user = true
} else {
$sudo_allow_aptdater_user = false
}
-8<
But it still dependent of the order of the include of the classes...
> or to global scope,
I am already using global-scoped variable for a similar thing in another
module, but I am not too much satisfied with this approach, that's why I
tried other ways.
> or to external data loaded in both places via extlookup() or hiera.
I will also take a look at this possibility, thanks.
> If you're using an ENC then global scope is a good bet; otherwise
> external data is probably better.
We are not using an ENC for now, I will probably test external data.
> The separate class option is great, but it is distinct from the other
> two only if the new class can choose the variable's value based only
> on nodes' facts.
It does not seem to be possible for me.
By the way I was able to order the module application using run stages,
so thanks for pointing me to this direction.
I don't find this to be the perfect/cleanest solution for my problem,
but at least it is working as needed and expected :)
> > I could perhaps use a parametrized class like with something like an
> > aptdater_client_enabled paramter, but I don't find this very nice...
> > It's why I am trying to test a variable to see if aptdater::client is
> > enabled.
> It constitutes a class dependency for one class to rely on data
> belonging to another. If the former class must not directly or
> indirectly 'include' the latter, then it is inconsitent to have such a
> dependency at all.
Yes, I understand your point.
> Similarly, it is always a poor idea for a class to depend on whether
> a separate class has been included, which is the most fundamental
> data belonging to that class.
Yes, I saw that this leads to unedefined variable access which isn't
really clean.
I will see if I can think of a better way of doing this. (and taking in
account the time I have for this...)
> John
Thanks for your help!
Regards,
Baptiste
--
\,,/_[-_-]_\,,/
Finagle's Second Law:
No matter what the anticipated result, there will always be
someone eager to (a) misinterpret it, (b) fake it, or (c) believe it
happened according to his own pet theory.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Unable to define order of modules
Hello,
Le 31/10/11 à 16:09, jcbollinger téléscripta :
> On Oct 28, 10:07 am, Baptiste Grenier
> wrote:
>
> > I am trying to define the modules' run order like this:
>
> You are running up against a common source of confusion: the
> difference between order of manifest evaluation (on the master) and
> order of catalog application (on the client). You appear to be
> conflating the two into "run order", but in general there is no need
> for them to be the same, and indeed, one sometimes wants them to
> differ.
>
> Evaluation order affects (only) the compilation of your manifests into
> a catalog, and the key mechanisms available for influencing it are
> (1) the 'require' and 'include' functions (*not* the 'require'
> resource metaparameter), and
> (2) lexical order within individual manifest files
>
> Application order affects (only) the order in which resources are
> applied to your nodes, but that's irrelevant if catalog compilation
> fails. The user-accessible mechanisms for influencing it are
> (1) the 'require' resource metaparameter and its friends,
> (2) the arrow syntax for defining resource relationships,
> (3) run stages, and
> (4) the 'require' function (which also affects order of manifest
> evaluation)
Thanks for all this information, I will do some more tests to see if I
can achieve my goal (see next paragraph).
> You have an evaluation order problem, and you have tried to fix it by
> constraining the application order. Instead, you want something like
> this:
>
> [...]
>
> class sudo {
> # KEY CHANGE:
> include "aptdater::client"
The problem is that I want to be able to include the sudo class without
including the aptdater::client class: some node can have sudo managed,
but won't be configured for apt-dater.
I could perhaps use a parametrized class like with something like an
aptdater_client_enabled paramter, but I don't find this very nice...
It's why I am trying to test a variable to see if aptdater::client is
enabled.
Is there another better/cleaner/simpler way of doing this?
> John
Regards,
Baptiste
--
\,,/_[-_-]_\,,/
http://asocial.ws/gwarf
I'm having a RELIGIOUS EXPERIENCE ... and I don't take any DRUGS
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Unable to define order of modules
Hello,
I am trying to define the modules' run order like this:
--8<
... % cat test.pp
class baseclass {
include sudo
}
class sudo {
file { '/tmp/sudoers':
ensure => 'present',
content => template('/tmp/sudoers.erb'),
}
notify { "class_sudo": message => "class sudo" }
}
class aptdater::client {
$sudo_allow_aptdater_user = true
notify { "class_aptdater_client": message => "class aptdater_client" }
}
class { "sudo": require => Class["aptdater::client"] }
node 'monster.maatg.fr' {
include baseclass
include aptdater::client
}
# vim: set tw=80 et smarttab sw=2 ts=2 softtabstop=2 nocindent noautoindent:
--8<
..d % cat /tmp/sudoers.erb
# DEBUG: has_variable?("aptdater::client:sudo_allow_aptdater_user") = <%=
has_variable?("aptdater::client::sudo_allow_aptdater_user") %>
# DEBUG: scope.lookupvar("aptdater::client:sudo_allow_aptdater_user") = <%=
scope.lookupvar("aptdater::client::sudo_allow_aptdater_user") %>
--8<
d.. % puppet apply test.pp
warning: Scope(Class[Sudo]): Could not look up qualified variable
'aptdater::client::sudo_allow_aptdater_user'; class aptdater::client has not
been evaluated at /tmp/sudoers.erb:1
warning: Scope(Class[Sudo]): Could not look up qualified variable
'aptdater::client::sudo_allow_aptdater_user'; class aptdater::client has not
been evaluated
notice: class aptdater_client
notice: /Stage[main]/Aptdater::Client/Notify[class_aptdater_client]/message:
defined 'message' as 'class aptdater_client'
notice: class sudo
notice: /Stage[main]/Sudo/Notify[class_sudo]/message: defined 'message' as
'class sudo'
notice: Finished catalog run in 0.04 seconds
--8<
... % cat /tmp/sudoers
# DEBUG: has_variable?("aptdater::client:sudo_allow_aptdater_user") = false
# DEBUG: scope.lookupvar("aptdater::client:sudo_allow_aptdater_user") =
undefined
--8<
But you can see that the order does not seems to be respected.
If I change the order of the includes in the node manifests and remove
the "class { "sudo": require => Class["aptdater::client"] }" it works as
expected:
--8<
... 1 % cat test.pp
class baseclass {
include sudo
}
class sudo {
file { '/tmp/sudoers':
ensure => 'present',
content => template('/tmp/sudoers.erb'),
}
notify { "class_sudo": message => "class sudo" }
}
class aptdater::client {
$sudo_allow_aptdater_user = true
notify { "class_aptdater_client": message => "class aptdater_client" }
}
#class { "sudo": require => Class["aptdater::client"] }
node 'monster.maatg.fr' {
include aptdater::client
include baseclass
}
# vim: set tw=80 et smarttab sw=2 ts=2 softtabstop=2 nocindent noautoindent:
--8<
... 1 % puppet apply test.pp
notice: class aptdater_client
notice: /Stage[main]/Aptdater::Client/Notify[class_aptdater_client]/message:
defined 'message' as 'class aptdater_client'
notice: /Stage[main]/Sudo/File[/tmp/sudoers]/content: content changed
'{md5}588dda775c2c1dcfa204510682a43458' to
'{md5}a1b441f7f7b2c922ef06125e69265c39'
notice: class sudo
notice: /Stage[main]/Sudo/Notify[class_sudo]/message: defined 'message' as
'class sudo'
notice: Finished catalog run in 0.04 seconds
--8<
d.. 1 % cat /tmp/sudoers
# DEBUG: has_variable?("aptdater::client:sudo_allow_aptdater_user") = true
# DEBUG: scope.lookupvar("aptdater::client:sudo_allow_aptdater_user") = true
--8<
But I would like to be able to use metaparameters or puppet functions to
force the order of the modules.
I also tried things like adding to aptdater::client class
--8<
Class['aptdater::client'] -> Class['sudo']
--8<
or adding
--8<
file { '/tmp/plop':
notify => File['/tmp/sudoers'],
}
--8<
or
--8<
file { '/tmp/plop':
notify => Class['sudo'],
}
--8<
But without any success.
What am I doing wrong?
Regards,
Baptiste
--
\,,/_[-_-]_\,,/
BOFH Excuse #132:
SCSI Chain overterminated
pgpXdZpiQtlp6.pgp
Description: PGP signature
[Puppet Users] Exporting variables between manifests/nodes
Hello,
Here is what I am trying to do:
- in a class aptdater::client I want to export the $fqdn
- in a class aptdater::master I want to retrieve all the exported $fqdn
to fill an erb template of a configuration file
Here is what I tried:
# modules/aptdater/manifests/init.pp
class aptdater {
# some not relevant resources
}
# modules/aptdater/manifests/master.pp
class aptdater::master inherits aptdater {
# list of hosts that will be added to the hosts.conf
$hosts = [$fqdn]
(...)
}
# modules/aptdater/manifests/client.pp
class aptdater::client inherits aptdater {
$aptdater::master::hosts += $fqdn
(...)
}
# modules/aptdater/templates/hosts.conf.erb
[Hosts]
Hosts=<% scope.lookupvar('aptdater::master::hosts').each do |i| -%>
<%= i + ";" -%>
<% end -%>
I have two nodes:
# manifests/nodes/master.node.plop.org.pp
node 'master.node.plop.org' {
include aptdater::master
}
# manifests/nodes/client.node.plop.org.pp
node 'client.node.plop.org' {
include aptdater::client
}
But it does not work, the Hosts line does only contain the $fqdn of the
master node.
I took a look at exported resource, but here it's not a resource that I
want to be exported, it's just a variable/fact.
Is this possible to achieve such a goal?
Thanks,
Baptiste
--
\,,/_[-_-]_\,,/
Start the day with a smile. After that you can be your nasty old self again.
pgpEivdri1XC8.pgp
Description: PGP signature
