I am reproducing the role of the puppetmaster, but I have more overall control
and I'm pushing the work of compiling the catalogue off to the nodes.
Currently I use a series of XML files until I'm ready to move them to a
database. My directory structure looks like this:
/configuration.xml
/os/OS/configuration.xml
/sites/SITECODE/configuration.xml
/sites/SITECODE/hosts/HOST/configuration.xml
When the puppeteer determines it's time to do a run on a host, it reads in what
site code the host is in from a database. It then takes the site code and
passes it to the manifest generator function to parse the appropriate XML
files. As I read in a host, I convert the XML entries into a hash and merge
them with right side precedence in the following order:
global->os->site->host
So a host only gets the appropriate data for that specific host, which
maintains security levels. Also, any global resource that's defined (virtual
resource, resource defaults, variables) can be overwritten at the os, site and
host levels. I can also disable virtual resources and resource defaults at a
lower level than the level it was defined in. This allows my actual puppet code
to be more generic and contain far less data. This way I feel safe with the
code living on the hosts instead of the puppetmaster. Of course this would
never work in some industries like banking, but it works great for us. It's
going to allow us to role out changes and puppetize existing boxes much faster.
On Nov 17, 2009, at 12:16 AM, Ohad Levy wrote:
> that sounds interesting, how does it work? in a way, you are reproducing the
> role of the puppetmaser...
>
>
> On Tue, Nov 17, 2009 at 1:20 PM, Carl Caum <carl.c...@gmail.com> wrote:
> No, the manifest is generated per host on the fly. So each manifest only has
> access to the data for it specifically. That's one advantage of my approach.
> I maintain the security model of using puppetmasters, but I distribute the
> work load to the nodes themselves.
>
> The manifest that's delivered to the host looks like this:
>
> variables
> resource defaults
> virtual resources
> node host {
> classes
> }
>
> On Nov 16, 2009, at 10:58 PM, Ohad Levy wrote:
>
>> Hi Carl,
>>
>> One thing that I didn't understand, does it means that the nodes you have
>> get only their part of the manifest? or do they get all manifest and apply
>> the right parts?
>>
>> the main issue I have with your approach is that a lot of information is
>> available on the client side (even if there is no node specific information).
>>
>> as far for features for foreman, let me know what you had in mind...
>>
>> cheers,
>> Ohad
>>
>> On Tue, Nov 17, 2009 at 12:30 PM, Carl Caum <carl.c...@gmail.com> wrote:
>> I have series of module servers that serve modules to the host. I don't use
>> puppetmasters though. In a nutshell, I have a manifest server called the
>> 'puppeteer' that generates manifests based on meta data stored in a
>> database. Well, it will be in database soon. For now it's in XML files.
>> The meta data contains everything from modules and what version of the
>> modules a host should have as well as things like variables, virtual
>> resources, resource defaults and classes. When it's time for a host to do a
>> puppet run, the puppeteer generates a manifest and puts an entry in to a
>> queue for that host. A service on the host, called 'puppetstring' polls the
>> queue every few seconds for work. When there's an entry in the queue for
>> it, it pulls down its data (the manifest and modules it needs). It syncs up
>> the modules and the appropriate version from the local module servers
>> (puppetcache servers) and executes the manifest.
>>
>> This awards me much flexibility. Basically my modules rarely contain data,
>> just structure. The contents of files, and even entire resources themselves
>> can be controlled through a web interface and be deployed on the host, site
>> or global level. This also allows me to do automated rollouts where I have
>> a sequence of hosts do runs and based on their success or failure, I can
>> determine whether or not to move on to the next step. This also allows me
>> to allow certain control over specific machines and resources to end users.
>> My database engineers can, for instance, control the contents of the
>> tnsnames.ora files on all Oracle clients through a web interface. I can
>> also put logic in to my scheduling of runs based on external criteria. For
>> instance, if the cluster has a lot of work to do for the next two hours,
>> don't do puppet runs every thirty minutes but instead every hour to reduce
>> the CPU load. Or maybe don't do a run on a cluster node if more than 40% of
>> the cluster nodes are currently doing runs.
>>
>> The web interface has yet to actually be built and I'm really sure whether
>> or not I should extend on The Foreman or role out my own, but that's another
>> topic.
>>
>> I've promised several people a blog post on all this and one day I'll
>> deliver on that promise :)
>>
>> On Nov 16, 2009, at 9:43 PM, Ohad Levy wrote:
>>
>>>
>>>
>>> On Tue, Nov 17, 2009 at 11:33 AM, Scott Smith <sc...@ohlol.net> wrote:
>>>
>>>
>>> Oh, no. We have some pretty cool deployment tools that I use to deploy
>>> puppetmasters. I can also use them to update manifests as if they were
>>> config updates to any other app.
>>> you are not using puppet to manage your puppetmasters?
>>>
>>>
>>> We have something like 4k or so hosts in prod (and another 1k or so in
>>> qa/dev) so it's pretty much a necessity =)
>>> I know the feeling
>>>
>>> It also allows me to enforce a true development lifecycle model for
>>> Puppet. The manifests, configs, and everything else really do look like
>>> any other app we develop/maintain.
>>> This is the main reason why we've started using environments, this was one
>>> of the only ways we could ensure that a host manifest will change only if
>>> it was planned to release.
>>> the testing process was also simplified a lot, as you can just create a
>>> test env with the same links (actually puppet does it all for you) but you
>>> can just play with the links to point to newer versions of your modules.
>>>
>>> on a side note, if you are willing, I'm interested in the performance of
>>> Foreman (http://theforeman.org) in a large infrastructure (for the Puppet
>>> reports, inventory etc) , I'll appreciate it if you are willing to provide
>>> feedback
>>>
>>> cheers,
>>> Ohad
>>>
>>>
>>
>>
>>
>>
>>
>>
>
>
>
>
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en
-~----------~----~----~----~------~----~------~--~---
