Re: [Puppet Users] New to hiera
On Sunday, May 5, 2013 8:43:52 PM UTC-5, Josh wrote: Thanks for the replies! Although it's a bit tardy, I'd like to add a word or two here. First, I think it's a mischaracterization to say heira is the recommended way to do things now. Certainly, hiera is the favored interface with external data, and I have said before that hiera finally makes parameterized classes usable. That's not at all the same as hiera being recommended for *every*thing, however. You should not feel that hiera is being imposed on you. Its integration with the Puppet core is a big advance -- the most significant in Puppet 3 -- but to the extent that you already have a manifest set that works in 2.7 without hiera, you have no particular need to convert to using hiera. While I don't like the idea of reaching in to a class to set its variables, Do you mean to *get* its variables? Puppet variables can be set only in the scope to which they belong, and even then only once. It is not possible to reach in to a class to set its variables. On the other hand, a class can reach *out* to obtain values for its variables from an external source. Prevailing Puppet wisdom holds that it is good practice to separate data from classes. given the design flaws in Puppet with parameterized classes, Although I have never been a fan of parameterized classes, with Puppet 3 I no longer attribute the related flaws to parameterized classes themselves, but rather to the parameterized-class declaration syntax, and somewhat to the intentional blurring of the distinction between classes and resources. It's not a particular problem to single out certain variables that characterize the allowed variation in the state described by a given class -- i.e. parameters -- but it is a *fundamental* problem to bind data to parameters in a manner that depends on parse order. Now that there are alternatives to doing the latter, those are separable issues. I devote verbiage to this issue because the word flaw suggests something that can be fixed, but although class parameterization is basically ok (if overrated), the problems with binding data to classes via unrestricted DSL statements (parameterized-style class declarations) are not fixable. That does not reflect a design flaw so much as the nature of the problem. I agree that this is a strength of Heira (so far, the only one, IMHO). Maybe this irks me because I'm also a developer who started with C++. This is actually a hard place from which to come to Puppet. Puppet DSL uses some OO terms and has syntax and structure reminiscent of some OO languages, but it is essential to understand that it is NOT object-oriented in any but the most basic sense. Your developer instincts will send you the wrong signals when you see the words class or inherit, for example. Coming from C/C++ may be especially hard, because include means something radically different in Puppet than in the C world, and define and defined types are very different beasts than C preprocessor macros. Also, the overall mindset attending development in imperative languages is just plain wrong for writing Puppet manifests. C++ (for example) focuses on actions, whereas Puppet DSL is almost entirely about state. Imperative programmers are typically inclined to ask the wrong question: How do I make Puppet do foo?. The correct question is usually How do I model foo-result in Puppet? If you've overcome these issues then I offer my congratulations to you. I looked for docs and didn't find much beyond random blog posts, there are no docs, really... I did read that article. Yes, this a significant weakness for hiera. I am sympathetic to your dissatisfaction there. I'm obviously a stalker. My (senior) co-worker that I'm training on Puppet is very much a pouncer. On Friday I had to say no you actually need to read what it says when trying to get him to use source instead of content (file type). He couldn't copy and paste, oh noes! I understand, though. I read a couple of the articles linked from that one, too, and there are lots of problems with them, and with the example Ashley gave (which the articles mention also). The NTP argument is pretty weak, IMHO. I don't think it's a great NTP module, I guess (I don't use it). =P No offense. =) I think the premise of supporting a new distro or OS (there is more than Linux out there, you know) with just variable flips almost silly. It practically requires modifying everything that will use that new OS. If all you do is flip a variable value, the module probably isn't very comprehensive. I almost always have functional differences for different OSes, like deploying an option file for a port, using .d files for some OSes and not for others, different templates, etc., not just a different package name. You're right, or course, that in many cases the differences from one OS to another requires more than
Re: [Puppet Users] New to hiera
I'm curious to your opinion on point # 3, are you talking about OS
packages or your organizations app version? If the latter, I was thinking
of using hiera, maybe with a backend other than yaml such as redis, to
store the version of the app, that way like you said it could be used in
deploy pipeline. Why wouldn't you want to do this?
Drew
Hi Drew,
I'm using a yaml backend so I can't speak about other options. So here's my
disertation on why I (mostly) don't want version numbers in hiera. In my
world view, our puppet code is just another component of our system. When
I talk about our pipeline, I'm refering to deploying a version of our
service into an environment. At the highest level, the service version
defines the version of all components - including puppet code. BTW, it's
not a big flat table, just the tip of the dependency tree.
I get my religion from Continuous Delivery (Jez Humble, David Farley) and I
can only hope that I'm not misreading it. Paraphrasing: You can break your
service much faster with a configuration change than with a code change in
one applications. So configuration should be version controlled.
The direction I'm headed is to manage top level version control outside of
our puppet code - with one caveate. We also have {%fqdn} as the first
entry in our hiera tree and the ability to override the version setting via
hiera. But it comes with the following warning:
# FQDN should only be used for temporary overrides.
Cheers,
Larry
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] New to hiera
Hi Josh,
Here's what we found. I would love to hear about other approaches because
we're still debating and building out our service.
1. Yes it's a great way to escape from parameterized classes. If you ever
needed to 'include' a class in two places, parameterized classes makes this
difficult or impossible.
2. I don't recommend using automatic hiera lookup. There are some subtle
gotchas in it especially when using hashes in hiera. In my office our
recommended practice is as almost as you wrote it, class bar ($foo =
hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
an easier transition from parameterized classes so it may still be a good
choice for you. As Ashley points out, read the latest docs!
3. If you explicitly use the hiera() call, you'll get nice variable not
found messages when you make a mistake. Clear error messages are rare in
puppet so don't underestimate their value.
4. Don't over complicate your hiera tree. It's not the place for
everything. Pick your largest collection of system-to-system variation and
put that in hiera.
In our office we identified three variations that we *wanted* to put in
hiera:
1. Variation between servers *within* an instance of our service
2. Variation between similar servers across different instances of our
service (eg. Identity of the zookeeper in production, staging QA)
3. Version differences between instances of our service. IE. use hiera to
manage the pipeline of releases.
What we *actually* put into hiera was only #2. When #1 started creeping
into hiera things got very messy. #1 is better handled by managing server
Roles. #3 should never go into Hiera - but I'm still arguing this point
with my co-workers.
Caveate Emptor: We're still hemming and hawwing about all this.
Cheers, Larry Fast
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] New to hiera
Hi Larry,
I'm curious to your opinion on point # 3, are you talking about OS packages
or your organizations app version? If the latter, I was thinking of using
hiera, maybe with a backend other than yaml such as redis, to store the
version of the app, that way like you said it could be used in deploy
pipeline. Why wouldn't you want to do this?
In addition to what everyone else has said, one of the bigger win's of
using hiera, is the flexibility it offers you with overriding variables.
With the below hierarchy you can override based off an individual server,
environment or datacenter. This could be powerful if you wanted to test
some new JVM settings on just one server in production or an entire
datacenter. You can use any facter fact you want to define your hierarchy.
:hierarchy:
- %{fqdn}
- env/%{env}/common
- datacenter/%{datacenter}
- common
Drew
On Sunday, May 5, 2013 2:04:19 PM UTC-4, Larry Fast wrote:
Hi Josh,
Here's what we found. I would love to hear about other approaches because
we're still debating and building out our service.
1. Yes it's a great way to escape from parameterized classes. If you ever
needed to 'include' a class in two places, parameterized classes makes this
difficult or impossible.
2. I don't recommend using automatic hiera lookup. There are some subtle
gotchas in it especially when using hashes in hiera. In my office our
recommended practice is as almost as you wrote it, class bar ($foo =
hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
an easier transition from parameterized classes so it may still be a good
choice for you. As Ashley points out, read the latest docs!
3. If you explicitly use the hiera() call, you'll get nice variable not
found messages when you make a mistake. Clear error messages are rare in
puppet so don't underestimate their value.
4. Don't over complicate your hiera tree. It's not the place for
everything. Pick your largest collection of system-to-system variation and
put that in hiera.
In our office we identified three variations that we *wanted* to put in
hiera:
1. Variation between servers *within* an instance of our service
2. Variation between similar servers across different instances of our
service (eg. Identity of the zookeeper in production, staging QA)
3. Version differences between instances of our service. IE. use hiera to
manage the pipeline of releases.
What we *actually* put into hiera was only #2. When #1 started creeping
into hiera things got very messy. #1 is better handled by managing server
Roles. #3 should never go into Hiera - but I'm still arguing this point
with my co-workers.
Caveate Emptor: We're still hemming and hawwing about all this.
Cheers, Larry Fast
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] New to hiera
Thanks for the replies!
While I don't like the idea of reaching in to a class to set its
variables, given the design flaws in Puppet with parameterized classes, I
agree that this is a strength of Heira (so far, the only one, IMHO). Maybe
this irks me because I'm also a developer who started with C++.
I looked for docs and didn't find much beyond random blog posts, there are
no docs, really... I did read that article. I'm obviously a stalker. My
(senior) co-worker that I'm training on Puppet is very much a pouncer. On
Friday I had to say no you actually need to read what it says when trying
to get him to use source instead of content (file type). He couldn't copy
and paste, oh noes! I understand, though.
I read a couple of the articles linked from that one, too, and there are
lots of problems with them, and with the example Ashley gave (which the
articles mention also). The NTP argument is pretty weak, IMHO. I don't
think it's a great NTP module, I guess (I don't use it). =P No offense. =)
I think the premise of supporting a new distro or OS (there is more than
Linux out there, you know) with just variable flips almost silly. It
practically requires modifying everything that will use that new OS. If all
you do is flip a variable value, the module probably isn't very
comprehensive. I almost always have functional differences for different
OSes, like deploying an option file for a port, using .d files for some
OSes and not for others, different templates, etc., not just a different
package name.
I completely agree with your finding #4. I keep reading about using Heira
to include classes and it makes me cringe. That's such a terrible idea, and
totally unnecessary, IMHO. It's trivial to include classes based on facts
with Puppet itself, where all the rest of your structure is declared...
Introducing a second area of declaring that stuff is just...ugh.
I don't totally understand your variations...but I like the thought process
and abstraction. It sounds like what I do all day long (on my Puppet
days). I could see version numbers being useful in Heira: having a default
and overriding it for dev or test. Instead of including instance::dev or
instance::test classes, I guess, that do the same thing. We don't use
version numbers at all in Puppet, though. =) This, however, is so far the
only good use for Heira that I've found--overriding variables.
I organize things into classes within a single module. I make high-level
functional classes that group things together based on mostly
non-technical stuff, like workflow or purpose, kind of like roles, and
include those in nodes. These grouping are usually pretty well-defined,
with limited scope. E.g. I have a few different support-group sub-classes
(e.g.: support-group::research) that create different user accounts and set
or override some parameters, or high level role classes like research-group
sub-classes. I almost never include a class in a node manifest directly
(usually just when testing). I have no node variables or top scope
variables. We have lots of HPC clusters, from 1 or 2 to entire racks of
machines, so putting everything into classes as a policy and habit really
helps us. This eliminates almost all problems with overriding variables so
far. I guess if we need to split something up (like different types of
NTP server), I would create classes for them and include those where
necessary. I'm not sure if/how Heira can help with this yet.
The only real problem I have is user accounts integrating with various
other things (like NFS/no NFS/root squash/pam_mount *and* AutoFS on the
same box). We have a very big, very flexible user define with a layer on
top of it for some corner cases. It's a lot of functional differences. I'm
not yet sure if Heira could help with this, either, but it's something I'll
be looking into...
I might try to track my journey and make blog posts to help codify what I
learn. Maybe someone will find it interesting or useful if they get into
the same situation...
GOodnight,
Josh
On Sun, May 5, 2013 at 2:04 PM, Larry Fast lfast1...@gmail.com wrote:
Hi Josh,
Here's what we found. I would love to hear about other approaches because
we're still debating and building out our service.
1. Yes it's a great way to escape from parameterized classes. If you ever
needed to 'include' a class in two places, parameterized classes makes this
difficult or impossible.
2. I don't recommend using automatic hiera lookup. There are some subtle
gotchas in it especially when using hashes in hiera. In my office our
recommended practice is as almost as you wrote it, class bar ($foo =
hiera('bar::foo')) { ... }. Hiera automatic lookup is a good way to allow
an easier transition from parameterized classes so it may still be a good
choice for you. As Ashley points out, read the latest docs!
3. If you explicitly use the hiera() call, you'll get nice variable not
found messages when you make a mistake. Clear error messages are rare in
[Puppet Users] New to hiera
Hello,
I've never looked at heira before. I'm looking into upgrading from 2.7 to
3, and it seems that heira is the recommended way to do things now. From
what I've read so far, it looks like one professed reason to use it is to
go from parameterized classes to hiera overrides, because I guess
parameterized classes are bad. So instead of:
class bar ($foo = 'blah') { ... }
One would have something like:
class bar { $foo = heira('bar::foo', 'blah') }
Is that correct?
I'm not yet sold on heira; so far it seems to just shift complexity outside
of the classes and add a little more in the process, with hierarchies and
stuff, and now I have data in multiple places... I still need case
statements to handle different OSes and stuff. Eh, we'll see. Maybe I just
haven't read something that explains the benefits well yet.
Josh
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] New to hiera
On May 4, 2013, at 9:50 PM, Josh j...@endries.org wrote:
So instead of:
class bar ($foo = 'blah') { ... }
One would have something like:
class bar { $foo = heira('bar::foo', 'blah') }
Is that correct?
That's the hard way to do it. The easy way is to do
class bar ($foo) { … }
and Hiera will look up the key bar::foo from the configured data sources.
--
Brian Lalor
bla...@bravo5.org
--
You received this message because you are subscribed to the Google Groups
Puppet Users group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-users@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: [Puppet Users] New to hiera
On Sat, May 4, 2013 at 9:50 PM, Josh j...@endries.org wrote: I'm not yet sold on heira; so far it seems to just shift complexity outside of the classes and add a little more in the process, with hierarchies and stuff, and now I have data in multiple places... I still need case statements to handle different OSes and stuff. Eh, we'll see. Maybe I just haven't read something that explains the benefits well yet. You should check out https://puppetlabs.com/blog/hiera-for-pouncers-and-stalkers/ as it's a great guide to the benefits of hiera. You really don't need case statements if you use heira, because you can make a hiera structure based on a fact like $::osfamily and then have: osfamily/Redhat.yaml osfamily/Debian.yaml This gives you the benefit that if you drop in a new .yaml file supporting a new distribution you only have to drop it into heira, and not modify multiple manifests. The article I linked probably does a better job of explaining it, but the goal is to remove all the data from your manifests leaving just variables behind, and then hiera fills in all the blanks. It takes some getting used to but I think you'll find it leads to better, cleaner, manifests in the long run. -- You received this message because you are subscribed to the Google Groups Puppet Users group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
