Re: [Puppet Users] Re: Getting all variable occurrences from Hiera
On Friday, August 17, 2012 1:26:21 PM UTC-5, Tim Mooney wrote:
>
>
> Consider:
>
> :hierarchy: - secure/fqdn/%{clientcert}
> - fqdn/%{clientcert}
> - secure/location/%{location}
> - location/%{location}
> - secure/common
> - common
>
>
> common.yaml:
> ---
> ntp:
>type: client
>servers:
> - 10.0.0.1
> - 10.0.0.2
>
> location/datacenter1.yaml:
> ntp:
>servers:
> - 10.1.0.101
> - 10.1.0.102
>
> fqdn/clock1.example.com.yaml:
> ntp:
>type: server
>
>
>
>
> Hopefully, the intent is relatively clear: the defaults for all systems
> are to be an ntp client and to use ntp servers with the IP addresses from
> the common.yaml, however everyone in location=datacenter1 should have
> the list of servers overridden, and the one system with
> fqdn=clock1.example.com should have its type=server.
>
> As hiera currently works, I don't believe there's any way to "merge" these
> types of nested data from multiple sources in the hierarchy. If I try
> lookup ntp['type'] for any of the systems in location=datacenter1, it
> will be empty, because type is not specified (duplicated) for the
> location/datacenter1.yaml.
>
> Because of this, what ends up happening is that you instead need to use
> a flat namespace
>
Well, yes, because a flat namespace is all that hiera natively provides.
Although an hierarchical namespace might be useful,
1. I am not in favor of implementing such a thing via the same
structures that are used to define complex data values. When I write or
read an Hiera YAML file, I don't want to be uncertain about what is data
and what is name.
2. You can use the structure of your data names to provide for
namespacing, however, as in fact your example does. Although that seems
unnatural to you, it seems perfectly natural to me. Perhaps it's just a
question of mindset and perspective.
3. Alternatively, you could use hiera_array() to obtain all the
applicable values for the "namespace" object you want, and merge them via a
separate function call. I wouldn't think the needed function would be hard
to write. Inasmuch as that function would look a lot like parts of hiera's
existing internals, perhaps it would make sense to have hiera provide it.
John
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/vhlm56eROs4J.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Getting all variable occurrences from Hiera
In regard to: Re: [Puppet Users] Re: Getting all variable occurrences from...: I'll comment in the ticket as well, I see you have, and you've closed it out. In Tim Mooney's example, which I think is the usual case for hiera data, hiera doesn't have an enumeration of all the 'type: client' nodes, nor should it. What I'm asking for (or at least, about) is indeed somewhat different than what Alexander was asking for. I'm a little surprised at the quick dismissal. It seems reasonable to me that since at least YAML supports data structures *and* puppet supports hashes and arrays and nesting of same, one might choose to design the configuration "namespace" for a particular class to use an actual data structure in hiera. I've been watching the list very closely for several months for any official statement from Puppetlabs folks about best practices with hiera data, and your statement is really the first I've seen. What you've said has a pretty clear implication: don't use nested data structures in hiera, because they're not going to work in the way that I think most people would expect them to work -- you can't merge different sub-keys from different parts of your hierarchy. In other words, we shouldn't use something like this: ntp: type: 'client' servers: - 10.1.2.3 - 10.1.2.4 allow_query: - '10.1.0.0/255.255.0.0' - '10.2.0.0/255.255.0.0' - '10.3.0.0/255.255.0.0' - '2001:4930::/:::' We should instead use: ntp_type: 'client' ntp_servers: - 10.1.2.3 - 10.1.2.4 ntp_allow_query: - '10.1.0.0/255.255.0.0' - '10.2.0.0/255.255.0.0' - '10.3.0.0/255.255.0.0' - '2001:4930::/:::' You could pretty easily write a custom parser function to return the data structures you want. Perhaps if I was as adept in ruby as many of the Puppetlabs employees, that would be pretty easy, but unfortunately I'm not. I do appreciate the response! It's really nice to know the thoughts of the people that are the experts on this. It helps me to plan the direction for our environment. Tim -- Tim Mooney tim.moo...@ndsu.edu Enterprise Computing & Infrastructure 701-231-1076 (Voice) Room 242-J6, IACC Building 701-231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Getting all variable occurrences from Hiera
I'll comment in the ticket as well, but one problem I have with this approach is that hiera doesn't necessarily have a list of all the nodes. It's an artifact of your data design that the mac/ip for each node is available as an enumerable list. In Tim Mooney's example, which I think is the usual case for hiera data, hiera doesn't have an enumeration of all the 'type: client' nodes, nor should it. You could pretty easily write a custom parser function to return the data structures you want. On Monday, August 20, 2012 2:53:20 AM UTC-7, Alexander Swen wrote: > > Agreed, for the original case. >> >> However, the original suggestion matches closely with an issue I've run >> into as well, which also had me thinking that something like "hiera_all" >> (I was thinking it should be "hiera_merge") would be needed. > > > I've just filed a feature request: > http://projects.puppetlabs.com/issues/16033 please vote on it when you > need it as well. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/TyA5C59_OgYJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Getting all variable occurrences from Hiera
> > Agreed, for the original case. > > However, the original suggestion matches closely with an issue I've run > into as well, which also had me thinking that something like "hiera_all" > (I was thinking it should be "hiera_merge") would be needed. I've just filed a feature request: http://projects.puppetlabs.com/issues/16033 please vote on it when you need it as well. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/0_d3_ynteOAJ. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Getting all variable occurrences from Hiera
In regard to: [Puppet Users] Re: Getting all variable occurrences from...:
That yaml file should keep alle the specific params of that host. and
every other module should be able to take advantage of that information.
This is not only the case for dhcp, but for dns as well. and of course the
/etc/network/interfaces file of the host itself.
And to avoid needless redundancy of information, certain values should be
written into that file maximum once. so, the ipaddress for example must not
be written in some dns config yaml file as well. and in the hostname.yaml.
Should you decide to change the ipaddress, you definitely forget either the
bind or the dhcp config.
Certainly data duplication should be avoided, and that could be aided by
putting all information for each host into a single hash for that host.
Those per-host hashes could equally well (for this purpose) appear in
separate YAML files or all together as values in a larger hash in one YAML
file.
Agreed, for the original case.
However, the original suggestion matches closely with an issue I've run
into as well, which also had me thinking that something like "hiera_all"
(I was thinking it should be "hiera_merge") would be needed.
Consider:
:hierarchy: - secure/fqdn/%{clientcert}
- fqdn/%{clientcert}
- secure/location/%{location}
- location/%{location}
- secure/common
- common
common.yaml:
---
ntp:
type: client
servers:
- 10.0.0.1
- 10.0.0.2
location/datacenter1.yaml:
ntp:
servers:
- 10.1.0.101
- 10.1.0.102
fqdn/clock1.example.com.yaml:
ntp:
type: server
Hopefully, the intent is relatively clear: the defaults for all systems
are to be an ntp client and to use ntp servers with the IP addresses from
the common.yaml, however everyone in location=datacenter1 should have
the list of servers overridden, and the one system with
fqdn=clock1.example.com should have its type=server.
As hiera currently works, I don't believe there's any way to "merge" these
types of nested data from multiple sources in the hierarchy. If I try
lookup ntp['type'] for any of the systems in location=datacenter1, it
will be empty, because type is not specified (duplicated) for the
location/datacenter1.yaml.
Because of this, what ends up happening is that you instead need to use
a flat namespace, so you design things like this
common.yaml:
ntp_type: client
ntp_servers:
- 10.0.0.1
- 10.0.0.2
location/datacenter1.yaml:
ntp_servers:
- 10.1.0.101
- 10.1.0.102
fqdn/clock1.example.com.yaml:
ntp_type: server
As I've said before on the list, this strikes me as a bit unnatural.
Tim
--
Tim Mooney tim.moo...@ndsu.edu
Enterprise Computing & Infrastructure 701-231-1076 (Voice)
Room 242-J6, IACC Building 701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
