When it comes to split responsibilities, such as that between the Ops teams
and Dev teams, we usually do a couple things:
1. Use hiera to store key/value pairs that can be visible worldwide (like
application versions). This allows developers to be able to test their own
stuff, and when ready, submit pull requests into the production environment
branches
2. Use node scoped variables for Ops-only information
3. Make Ops the keeper of facts and secrets in the node definitions
4. Structure our hierarchy to include applications, datacenters and/or
business units (ours is actually more complex, but this is simplified)
5. Parameterize all classes we wish to use hiera for.
5.1 In some cases, explicitly call hiera based on availability of node
scoped variables (this is more for pre-hiera backwards compatibility)
6. Use folder based environments (though it seems Puppet suggests moving
away from these, they give us more flexibility). We use a script we found
on the github called branch2env.py (slightly modified to fit our
environment better). I haven't used r10k, but it sounds like it does
something similar.
The actual node definition might look something like:
node vars-dc1-finance {
# This is the "folder" for puppet/hiera code to look at
$environment = 'production'
# These are "facts" that aid in hiera lookups
$datacenter = 'dc1'
$biz = 'finance'
# This is a node scoped secret
$order_service::webapp::password = 'foofoo'
}
node finance-webapp.dc1.com inherits vars-dc1-finance {
include base
include order_service::webapp
}
A basic hierarchy might be:
- nodes/%{::fqdn}
- dc/%{datacenter}
- biz/%{biz}
- common
Some example hiera yamls:
dc/dc1.yaml:
order_service::webapp::endpoint1: 'https://endpoint1.dc1.com'
base::yum_repo_server: 'repo.dc1.com'
biz/finance.yaml:
java::version: '1.7.0_60'
order_service::webapp::version: '1.2.3'
biz/hr.yaml:
java::version: '1.6.0_30'
order_service::webapp::version: '1.0.0'
common.yaml:
#This pulls from a node scoped variable
order_service::webapp::password: %{order_service::webapp::password}
# These are defaults in the event something higher in the hierarchy does
define them.
java::version: '1.6.0_18'
order_service::webapp::version: '1.0.0'
Now for some basic class definitions:
class order_service::webapp ($version, $password, $endpoint1) {
include java
package { "order_service-${version}" :
ensure => installed
}
file { "/etc/creds/order_service" :
content => "${password}"
}
# This template uses the $endpoint1 variable via <%= @endpoint1 %>
file { "/etc/overrides/order_service":
content => template("order_service/webapp/overrides.xml.erb")
}
}
class java ($version) {
package { "jdk-${version}" :
ensure => installed
}
}
Our hiera and modules are environment aware. And environment is stored in
puppet.conf via an erb.
hiera.yaml:
:datadir: "/etc/puppet/hiera/%{environment}"
puppet.conf:
environment = production (This is added via an erb template that reads the
node scoped $environment variable)
modulepath = /etc/puppet/modules/$environment
Our nodes are included in site.pp from a predefined location. This allows
us to do a couple things:
1. Keep tighter control of production assets
2. Allow developers to add in node definitions in cases this is necessary
(mainly due to old policies).
e.g.
import /etc/puppet/nodes/prod/* (points to a for-Ops-only repo)
import /etc/puppet/nodes/dev/* (points to a developer accessed repo)
We went through a few iterations of getting this right and figuring out how
to reduce duplication of data as much as possible. Just try to remember a
few things:
1. Try to keep your node definitions as simple as possible. When possible
only add facts not derived from facter to node definitions.
2. Keep configuration data out of your classes. Even defaults. Use
common.yaml or similar to add in defaults.
3. Unless absolutely necessary DO NOT use resource style declarations
(e.g. class { "java": version => '1.7.0_60'}). Hiera handles parameter
lookups quite well and you won't get duplicate declaration issues.
HTH,
Jake
On Saturday, August 2, 2014 2:44:05 AM UTC+7, Devminded wrote:
>
> Hi Pete.
>
> I ended up doing just that, class variables for versions of applications
> backed by Hiera. I am trying to avoid putting %{environment} in Hiera to
> keep different jurisdictions from interfering with each other. I just use
> directory environments for both Hiera data and puppet modules.
>
> Now I did end up with the issue I was afraid of: A messy mix of
> infrastructure data owned by Ops (IP-addresses, nodes, clustering,
> firewall, etc) with all the application data that is owned by delivery
> (what things are tested and integrated together).
>
> I'm thinking that my hierarchy is wrong and will try to separate the
> application versions from infrastructure data and retrieve them from
> separate repos. I'm fairly new to Hiera so I don't know if that will even
> work... will have a look into r10k as well.
>
> Something like this:
> ---
> :hierarchy:
> - "%{::role}_infrastructure"
> - "%{::role}_versions"
> - "..."
> - "common"
>
> Wish I learned all this "Ops" stuff years ago instead of fumbling in the
> dark...
>
>
> Den fredagen den 1:e augusti 2014 kl. 00:26:29 UTC+2 skrev Pete:
>>
>> Hi,
>>
>> I think the best way to achieve this is to use class variables for all
>> the versions of packages you want to manage and use hiera as the
>> backend.
>> You can actually use the $::environment fact in your hiera.yaml file
>> when defining the datadir.
>>
>> I tend to put my hiera tree in a separate repository but you could
>> just as easily put it in your environment repository,
>>
>> I would also recommend using r10k to manage your environment and hiera
>> checkouts.
>>
>> Does that all make sense?
>>
>> Pete.
>>
>> On 29 July 2014 09:10, Devminded <d.d.sa...@gmail.com> wrote:
>> > Hello everybody.
>> >
>> > I'm a developer new to puppet and are working with a complex system
>> made up
>> > of several subsystems. We have regulatory requirements which forces us
>> to
>> > have several production environments (at-least one per jurisdiction).
>> This
>> > is causing us some pains.
>> >
>> > The main problem is that the different jurisdictions "requires"
>> different
>> > versions (and sometimes content) of the deployed applications. Due to a
>> > drawn out certification process some environments lags behind in
>> versions by
>> > up to six month (that's how long it can take to get changes certified).
>> > Given that different environments require different versions of both
>> puppet
>> > modules and application versions how do we handle this while still
>> > guaranteeing system integrity?
>> >
>> > I'm in the process of trying out some other things, like the roles and
>> > profiles pattern. Looking at the 'version' properties in my example
>> below;
>> > is it a good idea to keep it hardcoded and release a new version of the
>> > puppet module for each version of the software (order_service in this
>> case)
>> > and assign those modules per environment or should I make the
>> application
>> > versions a hiera variable as well? And if I do decide to keep versions
>> in
>> > hiera how do I ensure that no-one (in ops) deploys a version that has
>> not
>> > been integration-tested with the rest of the system?
>> >
>> > Keeping versions in hiera would make everything much more dynamic but
>> also
>> > so much more difficult to ensure consistency, especially in a
>> Continuous
>> > Delivery workflow...
>> >
>> > Any takers? What obvious thing did I miss?
>> >
>> >
>> > *Example*
>> > class role::app_server {
>> > include profile::base
>> > include profile::linux
>> > include profile::jboss
>> > include profile::order_service::webapp
>> > }
>> >
>> > class profile::order_service::webapp {
>> > class { 'order_webapp'
>> > version => '3.1.2',
>> > db_address => hiera('db_address'),
>> > etc...
>> > }
>> > }
>> >
>> > class profile::jboss {
>> > class { 'jbosseap':
>> > version => 6.3,
>> > port => hiera('port'),
>> > broadcast_address => hiera('broadcast_address'),
>> > user_roles => hiera('user_roles'),
>> > user_groups => hiera('user_groups')
>> > }
>> > class { 'java'
>> > version => '>=1.7.0'
>> > }
>> > }
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Puppet Users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to puppet-users...@googlegroups.com.
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/puppet-users/06c4d567-6785-45ec-ae4a-8a41c236d2fe%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/cb30e200-7741-4c09-afb8-0b4f5122ba72%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
