[Puppet Users] Re: Parametrized classes, definitions and structure
Thanks John for yet another very useful reply, it is much appreciated.
I'm going to do some more reading on Hiera and look at reworking my
modules.
And thank you Ramin for some examples of Hiera in practice, looks very
interesting! Going to have a play around with it next week :)
Thanks!
On Apr 25, 7:01 pm, Ramin K wrote:
> On 4/24/2012 10:31 AM,AndyTaylorwrote:
>
>
>
>
>
>
>
>
>
> > Thank you very much for your detailed reply John. I hadn't looked at
> > Hiera before, it looks very interesting. On the point of it's not good
> > to include node/site data in manifests, it's kind of essential in my
> > eyes to the setup I am trying to implement.
>
> > At my company, one server may serve a site which has very different
> > requirements to another, so I really needed something that was very
> > flexible. To expand on the structure I outlined in my previous post:
> > each server has it's own node.pp file. If you need to change something
> > (e.g. MySQL settings, which will probably the most frequent one that
> > needs changing), you can simply edit that server's node manifest file.
> > The great advantage here is that each site's node manifest serves as
> > documentation of what changes have been made to it. It's also very
> > easy to read and understand for someone new to Puppet.
>
> > From some quick reading of the Hiera posts on the Puppet Labs blog, I
> > guess I could simply have a yaml file for each server with overrides
> > instead of doing it via the node manifest. However, I don't really see
> > the advantage of this approach, except for it would cut out some of
> > the clutter in my Puppet modules. The idea of a hierarchical set of
> > data being considered in line with Facter facts is very interesting
> > though, especially if I combined it with some custom facts of my own;
> > I will do some more reading on that.
>
> > With the class/definition point, I am wherever possible using classes,
> > as usually I'm dealing with the nature of a node (e.g. you are a
> > webserver, database server etc.) The only case I have used definitions
> > is for these configuration file overrides. So if I'm understanding
> > your comments on class/definition use properly, I think I'm sort of
> > taking the right approach to it...
>
> > The key concern for me at the moment is flexibility and scalability...
> > none of my stuff is live yet, and I want to make sure I'm using best
> > practices before I put it all in place. :)
>
> Mr Bollinger did his usual excellent job of thoroughly explaining the
> problem space in other parts of this thread. I'd simply add that using a
> richer data store, such as Hiera, can keep you from doing complicated
> gymnastics in your module code.
>
> I'm using a hiera.yaml file similar to the following with fqdn at the
> top and common at the bottom. I've added a role fact which is reading
> /etc/role on the client. You can use any Facter fact such as %{domain}
> or write your own. %{location} or %{datacenter} seems to be a common one
> for admins with multiple sites.
>
> :hierarchy:
> - %{fqdn}
> - %{role}_%{environment}
> - %{role}
> - %{environment}
> - common
>
> In my mysql::data class I add a variable and give it a default of 256M.
>
> $innodb_buffer_pool_size = hiera('mysql_innodb_buffer_pool_size', '256M')
>
> I can then set the following to deal with the specifics of my environment.
> 512M in my prod environment, production.yaml
> 4G in role db, db.yaml
> 1.5G on role db in environment stage, db_stage.yaml
> 8G on server db01.main.sfo, db01.main.sfo.mydomain.com.yaml
>
> Going one step further you can use hiera_array to merge data. For
> example you might allow a set of servers to monitor all servers and then
> allow the local monitors as well. Assuming a location fact and the
> following data files
>
> common.yaml
> monitoring_hosts:
> - 'mon01.ord'
> - 'mon02.ord'
>
> lax.yaml
> monitoring_hosts:
> - 'mon01.lax'
>
> class someclass::data {
> $hosts = hiera_array('monitoring_hosts')
>
> }
>
> someclass::data::hosts = ['mon01.lax','mon01.ord','mon02.ord']
>
> Hopefully this should give you some ideas of how to use Hiera in your
> new system.
>
> Ramin
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Re: Parametrized classes, definitions and structure
On 4/24/2012 10:31 AM, Andy Taylor wrote:
Thank you very much for your detailed reply John. I hadn't looked at
Hiera before, it looks very interesting. On the point of it's not good
to include node/site data in manifests, it's kind of essential in my
eyes to the setup I am trying to implement.
At my company, one server may serve a site which has very different
requirements to another, so I really needed something that was very
flexible. To expand on the structure I outlined in my previous post:
each server has it's own node.pp file. If you need to change something
(e.g. MySQL settings, which will probably the most frequent one that
needs changing), you can simply edit that server's node manifest file.
The great advantage here is that each site's node manifest serves as
documentation of what changes have been made to it. It's also very
easy to read and understand for someone new to Puppet.
From some quick reading of the Hiera posts on the Puppet Labs blog, I
guess I could simply have a yaml file for each server with overrides
instead of doing it via the node manifest. However, I don't really see
the advantage of this approach, except for it would cut out some of
the clutter in my Puppet modules. The idea of a hierarchical set of
data being considered in line with Facter facts is very interesting
though, especially if I combined it with some custom facts of my own;
I will do some more reading on that.
With the class/definition point, I am wherever possible using classes,
as usually I'm dealing with the nature of a node (e.g. you are a
webserver, database server etc.) The only case I have used definitions
is for these configuration file overrides. So if I'm understanding
your comments on class/definition use properly, I think I'm sort of
taking the right approach to it...
The key concern for me at the moment is flexibility and scalability...
none of my stuff is live yet, and I want to make sure I'm using best
practices before I put it all in place. :)
Mr Bollinger did his usual excellent job of thoroughly explaining the
problem space in other parts of this thread. I'd simply add that using a
richer data store, such as Hiera, can keep you from doing complicated
gymnastics in your module code.
I'm using a hiera.yaml file similar to the following with fqdn at the
top and common at the bottom. I've added a role fact which is reading
/etc/role on the client. You can use any Facter fact such as %{domain}
or write your own. %{location} or %{datacenter} seems to be a common one
for admins with multiple sites.
:hierarchy:
- %{fqdn}
- %{role}_%{environment}
- %{role}
- %{environment}
- common
In my mysql::data class I add a variable and give it a default of 256M.
$innodb_buffer_pool_size = hiera('mysql_innodb_buffer_pool_size', '256M')
I can then set the following to deal with the specifics of my environment.
512M in my prod environment, production.yaml
4G in role db, db.yaml
1.5G on role db in environment stage, db_stage.yaml
8G on server db01.main.sfo, db01.main.sfo.mydomain.com.yaml
Going one step further you can use hiera_array to merge data. For
example you might allow a set of servers to monitor all servers and then
allow the local monitors as well. Assuming a location fact and the
following data files
common.yaml
monitoring_hosts:
- 'mon01.ord'
- 'mon02.ord'
lax.yaml
monitoring_hosts:
- 'mon01.lax'
class someclass::data {
$hosts = hiera_array('monitoring_hosts')
}
someclass::data::hosts = ['mon01.lax','mon01.ord','mon02.ord']
Hopefully this should give you some ideas of how to use Hiera in your
new system.
Ramin
--
You received this message because you are subscribed to the Google Groups "Puppet
Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Re: Parametrized classes, definitions and structure
On Apr 24, 12:31 pm, Andy Taylor wrote: > Thank you very much for your detailed reply John. I hadn't looked at > Hiera before, it looks very interesting. On the point of it's not good > to include node/site data in manifests, it's kind of essential in my > eyes to the setup I am trying to implement. Unless the setup you're trying to implement is "one with node-specific data in its manifests", you are mistaken. Or perhaps you misunderstand me. I am not at all saying that your manifests should not rely on node- or site-specific data, only that those data do not belong in the manifest files themselves. Instead, put them in an external data source, such as one accessed via Hiera. That's why I mentioned the principle in the first place. > At my company, one server may serve a site which has very different > requirements to another, so I really needed something that was very > flexible. To expand on the structure I outlined in my previous post: > each server has it's own node.pp file. If you need to change something > (e.g. MySQL settings, which will probably the most frequent one that > needs changing), you can simply edit that server's node manifest file. > The great advantage here is that each site's node manifest serves as > documentation of what changes have been made to it. It's also very > easy to read and understand for someone new to Puppet. It is somewhat better for all node-specific data to be associated with nodes' declarations, and for all site-specific data to be in site.pp, than for such data to be spread throughout your manifests. It still presents problems, however. In particular, it's brittle to put the node-specific data into global variable declarations. The only way that can work is to use conditional 'import'ing of the appropriate node file (else different nodes' declarations will clash). 'Import' itself is best avoided as much as possible, though you'd be on the borderline of one of the few reasonable use cases. On the other hand, putting node-specific data into node-scoped variables limits you to either relying on dynamic scoping (which is being phased out) or using those variables only directly inside your node declaration. Using literal node-specific values directly as function, class, or defined type parameters is a bit better, but it requires you to forward those values throughout the rest of your manifests wherever they are needed. That complicates everything, requiring extra parameters, and perhaps even preventing you from using non- parameterized classes in some places. > From some quick reading of the Hiera posts on the Puppet Labs blog, I > guess I could simply have a yaml file for each server with overrides > instead of doing it via the node manifest. Pretty much, yes. > However, I don't really see > the advantage of this approach, except for it would cut out some of > the clutter in my Puppet modules. More than that: 1) It could greatly simplify and clarify some of your manifests. Your node declarations especially would benefit, but reducing parameterization elsewhere clarifies your manifests by focusing attention on the big picture -- roles, features, capabilities -- and away from the details. 2) It can help you obtain the benefits of using classes (where they are appropriate) without requiring class parameterization (which would lose you some of those same benefits). 3) From a change management perspective, separating the data allows you to more easily recognize the nature of any given change. > The idea of a hierarchical set of > data being considered in line with Facter facts is very interesting > though, especially if I combined it with some custom facts of my own; > I will do some more reading on that. > > With the class/definition point, I am wherever possible using classes, > as usually I'm dealing with the nature of a node (e.g. you are a > webserver, database server etc.) The only case I have used definitions > is for these configuration file overrides. So if I'm understanding > your comments on class/definition use properly, I think I'm sort of > taking the right approach to it... It sounds like you are generally doing the right thing, but you did ask about those definitions, and I don't think they're good style. As I said before, however, the most important criterion is that it works well. > The key concern for me at the moment is flexibility and scalability... > none of my stuff is live yet, and I want to make sure I'm using best > practices before I put it all in place. :) I think making your manifests as generic as possible, separating data from manifests, and using a flexible external data source such as Hiera is a win on both flexibility and scalability. I doubt you'll find a consensus on best practices, but I am confident that you will find others who also consider those to be Puppet best-practices principles. John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. T
[Puppet Users] Re: Parametrized classes, definitions and structure
Thank you very much for your detailed reply John. I hadn't looked at
Hiera before, it looks very interesting. On the point of it's not good
to include node/site data in manifests, it's kind of essential in my
eyes to the setup I am trying to implement.
At my company, one server may serve a site which has very different
requirements to another, so I really needed something that was very
flexible. To expand on the structure I outlined in my previous post:
each server has it's own node.pp file. If you need to change something
(e.g. MySQL settings, which will probably the most frequent one that
needs changing), you can simply edit that server's node manifest file.
The great advantage here is that each site's node manifest serves as
documentation of what changes have been made to it. It's also very
easy to read and understand for someone new to Puppet.
>From some quick reading of the Hiera posts on the Puppet Labs blog, I
guess I could simply have a yaml file for each server with overrides
instead of doing it via the node manifest. However, I don't really see
the advantage of this approach, except for it would cut out some of
the clutter in my Puppet modules. The idea of a hierarchical set of
data being considered in line with Facter facts is very interesting
though, especially if I combined it with some custom facts of my own;
I will do some more reading on that.
With the class/definition point, I am wherever possible using classes,
as usually I'm dealing with the nature of a node (e.g. you are a
webserver, database server etc.) The only case I have used definitions
is for these configuration file overrides. So if I'm understanding
your comments on class/definition use properly, I think I'm sort of
taking the right approach to it...
The key concern for me at the moment is flexibility and scalability...
none of my stuff is live yet, and I want to make sure I'm using best
practices before I put it all in place. :)
On Apr 24, 5:13 pm, jcbollinger wrote:
> On Apr 24, 9:13 am, Andy Taylor wrote:
>
>
>
>
>
>
>
>
>
> > Hi,
>
> > I've written a set of Puppet modules to manage servers, but am
> > slightly concerned about the structure I've used as it seems to differ
> > significantly from anything else I've seen in Puppet Forge/elsewhere
> > on the internet.
>
> > I've made extensive use of definitions so that defaults for modules
> > can be overridden in the node manifest for a server. For example:
>
> > node examplenode.example.com {
>
> > include apache
> > include mysql
>
> > define apache::config {
> > listenport => 8080
>
> > }
>
> > define mysql::config
>
> > }
>
> > So in each module, I have a config definition which must be called in
> > each node manifest, even if you don't want to override any defaults
> > (as in the case of MySQL above). I haven't used parametrized classes
> > at all.
>
> > It all works fine,
>
> That it works well is the most important criterion. With that
> said, ...
>
> > but are there any disadvantages to the sort of
> > structure I'm using? Should I be using more parametrized classes
> > instead of definitions?
>
> One problem with using definitions, especially definitions that are
> required to be instantiated for multiple classes to work, is that each
> instantiation of a definition is a separate resource. Each must
> therefore have a unique name among the resources of its type, and each
> will be applied separately. This presents either a management problem
> (who is responsible for instantiating the definition in any given set
> of circumstances?) or an efficiency problem (equivalent definition
> instances are redundantly applied) or both.
>
> Parameterized classes do not present that efficiency problem because
> all classes are singletons. That intensifies the management problem,
> however, because unlike ordinary classes, each parameterized class can
> be declared only once for any given node. In general, few problems
> are best solved by parameterized classes.
>
> You should use defined types only when you need *resources* for which
> there is no native type available. In that case you should know that
> you want resources rather than classes. That might be because nodes
> can have more than one instance of your 'thing', because you want
> support for virtual or exported 'things', or (best) simply because
> your 'thing' is better characterized as something nodes *have* than as
> something that they *are*.
>
> On the other hand, you should use (ordinary) classes where you want
> their singleton nature -- especially idempotency, which is a far more
> useful property than many appreciate. You must also use classes where
> you need your 'thing' to be declared via an ENC (the C is for
> CLASSifier, after all). You should generally use classes where they
> represent an aspect of nodes' type / kind / role / nature, as opposed
> to a tangible thing you can actually manage.
>
> As for default values and value overriding, if your 'things' are a
> better fit for classes th
[Puppet Users] Re: Parametrized classes, definitions and structure
On Apr 24, 9:13 am, Andy Taylor wrote:
> Hi,
>
> I've written a set of Puppet modules to manage servers, but am
> slightly concerned about the structure I've used as it seems to differ
> significantly from anything else I've seen in Puppet Forge/elsewhere
> on the internet.
>
> I've made extensive use of definitions so that defaults for modules
> can be overridden in the node manifest for a server. For example:
>
> node examplenode.example.com {
>
> include apache
> include mysql
>
> define apache::config {
> listenport => 8080
>
> }
>
> define mysql::config
>
> }
>
> So in each module, I have a config definition which must be called in
> each node manifest, even if you don't want to override any defaults
> (as in the case of MySQL above). I haven't used parametrized classes
> at all.
>
> It all works fine,
That it works well is the most important criterion. With that
said, ...
> but are there any disadvantages to the sort of
> structure I'm using? Should I be using more parametrized classes
> instead of definitions?
One problem with using definitions, especially definitions that are
required to be instantiated for multiple classes to work, is that each
instantiation of a definition is a separate resource. Each must
therefore have a unique name among the resources of its type, and each
will be applied separately. This presents either a management problem
(who is responsible for instantiating the definition in any given set
of circumstances?) or an efficiency problem (equivalent definition
instances are redundantly applied) or both.
Parameterized classes do not present that efficiency problem because
all classes are singletons. That intensifies the management problem,
however, because unlike ordinary classes, each parameterized class can
be declared only once for any given node. In general, few problems
are best solved by parameterized classes.
You should use defined types only when you need *resources* for which
there is no native type available. In that case you should know that
you want resources rather than classes. That might be because nodes
can have more than one instance of your 'thing', because you want
support for virtual or exported 'things', or (best) simply because
your 'thing' is better characterized as something nodes *have* than as
something that they *are*.
On the other hand, you should use (ordinary) classes where you want
their singleton nature -- especially idempotency, which is a far more
useful property than many appreciate. You must also use classes where
you need your 'thing' to be declared via an ENC (the C is for
CLASSifier, after all). You should generally use classes where they
represent an aspect of nodes' type / kind / role / nature, as opposed
to a tangible thing you can actually manage.
As for default values and value overriding, if your 'things' are a
better fit for classes than for definitions, then you should be
looking at external data and Hiera instead of parameterization. Even
for any definitions you retain, it might be to your advantage to
define certain default values via external data than to code them into
your manifests, or even to remove parameters in favor of external
data. Really, it is best if your manifests contain no node- or site-
specific data.
John
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
