[Puppet Users] Re: What's your hiera tree look like?
I couldn't get hiera integrated with PE 3.0.1 to work with nested
hierarchies, so this is what my hierarchy looks like:
:hierarchy:
- defaults
- %{clientcert}
- %{domain}
- %{osfamily}
- %{environment}
- global
Regards,
Alex
On Tuesday, March 19, 2013 2:30:16 PM UTC-7, Larry Fast wrote:
My Hiera tree is turning into a chaotic mess. I'm guessing that my tree
structure is one problem source. So I'm wondering how other real users have
setup hiera. Here are the core problems that are giving me headaches.
# I need to version control my modules separately but hiera data is
monolithic. If I need to change anything anywhere, I need to release a new
version of the whole tree.
# I can't easily review the final output from hiera within puppet because
the traversal is dependent on facts. I need to feed the facts into Hiera
before I get anything meaningful.
# Can't easily compare the config of two different servers. (really the
same as the previous)
Here's my Hiera tree. I'm very interested to know how others deal with
these issues. Does Puppet Enterprise include Hiera mgmt tools?
- %{environment}/%{project}/%{server_env}/%{module_name}
- %{environment}/%{project}/%{server_env}/common
- %{environment}/%{project}/%{role_name}-role/%{module_name}
- %{environment}/%{project}/%{role_name}-role/common
- %{environment}/%{project}/%{module_name}
- %{environment}/%{location}/%{module_name}
- %{environment}/%{module_name}
- %{environment}/common
NOTES:
Each Environment is a copy of our git repo and includes the hiera tree.
Project, Server_env, Role_name Location are custom facts
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users.
For more options, visit https://groups.google.com/groups/opt_out.
[Puppet Users] Re: What's your hiera tree look like?
I use Netgroups (with LDAP as backend) to group my nodes. Netgroups is an old NIS thing, which works beautifully. You can group hosts, and each netgroup, can contain other netgroups, users or hosts. This way, a host can be a member of several netgroups - and I use netgroup membership to determine something akin to roles. Currently I have to use IN and netgroups.include in my templates. $netgroups - is a paramater my script adds (by way of external_nodes = /path/to/script). One thing I would very much like, is to hiera to support lookup, based on netgroups (which is an array). That is definetely the missing part in my puzzle. I use the same netgroups for many other things (such as access control - I have a central file (used by PAM) which defines which groups can access which netgroups). -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: What's your hiera tree look like?
On Fri, Mar 22, 2013 at 04:22:19PM -0700, Aaron Mills wrote: The biggest pain point for us is that having hiera + puppet in the same repo feels like extra work. If they're going to live together, why even use hiera? Why not just set variables at the node level? For me, here are some reasons to want to do it that way (I'm looking at moving to Hiera right now): 1. Hiera stuff is set before everything, which fixes a lot of scoping issues in puppet. 2. You can drive other tools off the Hiera YAML; you can basically use it as an inventory system. Multiple sources of truth is *bad*. Alternatively, you can have an inventory system generate Hiera YAML very easily; generating well-formed puppet instructions with reasonable flexibility is *HARD* (I've written a system to do this, trust me). 3. Tweaking a YAML value means you are, in some sense at least, tweaking *data*, but if you're messing with puppet variables you're always messing with *code*, with all the joy that entails. -Robin -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] Re: What's your hiera tree look like?
On 25 March 2013 10:12, Robin Lee Powell rlpow...@digitalkingdom.orgwrote: On Fri, Mar 22, 2013 at 04:22:19PM -0700, Aaron Mills wrote: The biggest pain point for us is that having hiera + puppet in the same repo feels like extra work. If they're going to live together, why even use hiera? Why not just set variables at the node level? For me, here are some reasons to want to do it that way (I'm looking at moving to Hiera right now): These are exactly the reasons I moved to hiera when I switched my environment over to puppet 3 from 2.6. It was mostly about separating my data from the code and making my modules easy to read and portable. My modules and node templates were getting insane and so hard to read (I started using puppet 0.24 so it was the only way to do it at the time apart from setting up and ENC) Nail all this to a puppetmaster with a puppetdb backend and set up foreman as an ENC and everything is lovely. Especially my node definitions which are only used for adding the occasional define and occasionally adding a module include for testing before I add it to my nodes in foreman. 1. Hiera stuff is set before everything, which fixes a lot of scoping issues in puppet. 2. You can drive other tools off the Hiera YAML; you can basically use it as an inventory system. Multiple sources of truth is *bad*. Alternatively, you can have an inventory system generate Hiera YAML very easily; generating well-formed puppet instructions with reasonable flexibility is *HARD* (I've written a system to do this, trust me). 3. Tweaking a YAML value means you are, in some sense at least, tweaking *data*, but if you're messing with puppet variables you're always messing with *code*, with all the joy that entails. -Robin -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[Puppet Users] Re: What's your hiera tree look like?
Late to the party, but wanted to add my $.02
We also keep hiera, manifests, and modules in a single repository at the
moment. Since we use dynamic environments for development (as outlined
here: https://puppetlabs.com/blog/git-workflow-and-puppet-environments), we
also have a node-level variable called $our_environment that specifies
whether a box lives in dev/test/prod (We talk about environment being
where the puppet manifest is in the development cycle, and
$our_environment being where the node is in the development cycle). So
our hierarchy looks like this:
- %{our_environment}/%{fqdn}
- %{our_environment}/role_%{role}
- %{our_environment}
- common
Hiera finds its data in: /usr/local/puppet/%{environment}/hieradata
A node definition would look like:
node 'foo' {
$our_environment = 'development'
include mysql
include some::submodule
}
When in doubt, we set hiera variables at the %{fqdn} level. So there's some
duplication of common settings, but we avoid having too many questions
about where variables are getting set. Putting too many search layers into
hiera just reintroduces the problems that were supposed to be solved by
hiera in the first place (i.e. getting rid of the inherits stuff).
The biggest pain point for us is that having hiera + puppet in the same
repo feels like extra work. If they're going to live together, why even use
hiera? Why not just set variables at the node level? Rhetorical questions,
to be sure. But we have to live with hiera + puppet in the same repo since
otherwise we wouldn't be able to do dynamic environments (at least not
while developing modules that use hiera).
By the way - if you're asking for a wish list. I'd REALLY like to have
hiera be able to look in more than one directory path for configuration
variables. This way we could keep secure configuration information separate
from our normal DVCS. hiera-gpg ameliorates this somewhat, but I'd prefer
to solve that problem with plain YAML and a separate, secure path on the
puppet master.
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
[Puppet Users] Re: What's your hiera tree look like?
Hi Andy, I guess my most important request for Hiera 2 is diagnostic information. It's probably a broader request for better diagnostics in puppet as a whole. Here's what I dream of ... 1. The ability to run Puppet on the puppetmaster using the cached Facts from a server. 2. A postmortem (sorry, retrospective) output that includes a dump of all internal variables. Probably a .yaml file. Ideally each variable should be able to state how it was set. Eg. Came from Hiera/qa_env/zookeeper_role/zabbix_module OR set at file:line module/zabbix/manifest/init.pp:136 To answer some questions about how we are using Hiera more generally Puppet - we only use the yaml backend - we create custom facts for each server inside a puppet module (soon to be an ENC): location, project, role server_env. The last could be renamed as server_collection or service_group. - Each Puppet Environment is a git clone of our repo. - Development activity is delivered into the tip and then we create a branch for the deployment pipeline: integration=staging=production. IMO, we need the branch because our puppet code is still young. When things are more stable we may be able to push straight from the tip. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
