On 31/01/11 19:11, Udo Waechter wrote: > Hi. > > I am just reading this thread, and it strikes me that we have the > same problems with 2.6.3. > > Since upgrading from 2.6.2 to .3 puppetmaster shows the behabiour > described in this thread. We have about 160 clients, and puppetmaster > (is now) 8 core, 8Gb RAM kvm instance. Had this with 4 cores and 4 > gigs RAM, "doublesizing" the VM did not change a thing! > > We use passenger 2.2.11debian-2 and apache 2.2.16-3, ruby1.8 from > squeeze.
I see a pattern here. It seems Micah (see a couple of mails above in this thread) has about the same setup, except he's using mongrels. It would be great to try a non-debian ruby (hint: Ruby Enterprise Edition for instance) to see if that's any better. Do you use storeconfigs? > Puppetmaster works fine after restart, then after about 2-3 hours it > becomes pretty unresponsive, catalog runs go upt do 120 seconds and > more (the baseline being something about 10 seconds). With 160 hosts, a 30 min sleeptime, and a compilation of 10s, that means you need 1600 cpu seconds to build catalogs for all your fleet. With a concurrency of 8 core (assuming you use a pool of 8 passenger app), that's 200s per core, which way less than the max of 1800s you can accomodate in a 30 min time-frame. Of course this assumes an evenly distributed load an perfect concurrency, but still you have plenty of available resources. So I conclude this is not normal. > I need to restart apache/puppetmaster about once a day. When I do > that I need to: > > * stop apache * kill (still running) pupppetmasters (with SIGKILL!), > some are always left running with "CPU 100%" * start apache Does stracing/ltracing the process show something useful? > Something is very weird there, and there were no fundamental changes > to the manifests/modules. > > The only thing that really changed is the VM itself. It was XEN (for > years), we switched to KVM with kernel 2.6.35 > > Another strange thing: > > puppet-clients do run a lot longer nowadays. A machine usually took > about 40-50 seconds for one run. When puppetmaster goes crazy it now > takes ages (500 seconds and even more). If your master are busy, there are great chances your clients have to wait more to get served either catalogs or sourced files (or file metadata). This can dramatically increase the run time. > Something is weird there... --udo, Indeed. -- Brice Figureau My Blog: http://www.masterzen.fr/ -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
