I actually wrote up a post about using multiple environments for this purpose. It includes the process flow chart as well.
http://cwebber.ucr.edu/2011/03/multiple-environments-in-puppet/ --cwebber On 3/22/11 2:10 PM, "Thomas Bellman" <bell...@nsc.liu.se> wrote: >Giovanni Bordello wrote: > >> But what if I need to do a web-server specific change? And then >> Mail-server specific change? If I had only one test client machine I >> would have to reinstall it every time I needed to do verify a change >>for >> a different group of servers. That's hardly a way to go. There must be >>a >> better approach. >> >> How do you guys test puppet changes before they go live? > >First of all, we do have multiple environments, so we can test >new versions of the manifests before taking them into production. >There is of course the an environment named "production" which >clients are usually running from. Then each sysadmin has their >own personal environment, and some of us have multiple personal >environments. We do our changes in our personal environment, >test them, and when we feel they are ready, we push them to the >central Git repository, and then do a 'git pull' in the production >environment. I highly recommend this. > >Our approach to test machines vary a bit depending on how extensive, >invasive and risky changes we are making. If I'm doing something >small and simple, like adding a member to a mail alias on the email >server, I just do that, commit, push, pull and let puppetd on the >email server do its job. (Since I only run puppetd every fourth >hour, not every thirty minutes, I might do a manual run of puppetd >to apply my change quicker.) > >For somewhat larger changes, but changes where I feel the risk of >actually harming the server is very low, I would first run puppetd >manually in no-op mode on the live client against my environment. >If that looks okay, I will then run puppetd again and let it do its >changes, and then I test that my changes work as I intended. If >they don't, then I revert them (usually manually), and go back to >fixing my manifests. > >When the risk gets higher, or if I think something will take me >some time to implement, I install a test server. That will usually >be a Xen guest. I make a new node definition in my manifests that >is a copy of the real server I want to change, except that it has >different hostname, IP address, and MAC address. Then I install >CentOS on that with kickstart (ca 5 minutes), and run Puppet on >it (less than 15 minutes). I have Puppet generate Xen config >files and a kickstart file for the test machine on the Xen host, >so it is fairly painless. I do need to manually create the LVM >volumes for the virtual disks, but all in all I can easily have >a clone of server up and running in 30 minutes, including adding >the test machine to DNS and DHCP, installing OS on it and running >Puppet on it. > >When I have my test server up and running, I do all my testing >on that. When I'm close to finished with extensive changes, I >often re-install my test server from scratch to check that >everything really works. Often I find that they don't (typically >some missed dependencies) and have to fix that, and then I do >a new re-install. > >Since virtual servers are fairly cheap (and I can often give >them less CPU, memory and disk than the real server needs), I >sometimes have several such test servers running, if I am doing >work on several different features at the same time. (That would >typically be because I started on doing something, then some >other change with higher priority came up and I had to put my >original work aside for a few days or weeks.) > > >Where the limit is for something you dare test directly on live >production servers, would vary between organisations. We can >usually tolerate the occasional unplanned downtime if they are >short enough, so I probably have a higher threshold than many >others before I install a separate test machine. > >But regardless of how high risks you can take with your production >machines, I heartily recommend that you make it easy to create and >install test machines. Virtualization is really nice for that. > > > /Bellman > >-- >You received this message because you are subscribed to the Google Groups >"Puppet Users" group. >To post to this group, send email to puppet-users@googlegroups.com. >To unsubscribe from this group, send email to >puppet-users+unsubscr...@googlegroups.com. >For more options, visit this group at >http://groups.google.com/group/puppet-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
