On Wed, Mar 10, 2010 at 1:34 PM, Douglas Garstang <doug.garst...@gmail.com> wrote: > On Wed, Mar 10, 2010 at 1:17 PM, Brice Figureau > <brice-pup...@daysofwonder.com> wrote: >> On 10/03/10 22:06, Douglas Garstang wrote: >>> So, it became apparent to me, after emailing someone off list, that >>> managing a lot of files in deep directory structures might be part of >>> the cause. >>> >>> We are running 10 instances of JBOSS and 10 instances of tomcat on >>> each of these servers. Don't ask me why, it's just the way it was done >>> before I arrived and changing it is not trivial. >>> >>> On disk, each instance of JBOSS starts at >>> /opt/jboss/current/server/tfelN (where N is the instance number) >>> >>> and each instance of tomcat starts at: >>> /opt/tomcat/tfelN/starterkit/current (where N is the instance number) >> >> Do you source the whole hierarchy? >> Or do you only manage it? >> >>> I manually looked through the puppet config and counted 25 unique >>> files that are being managed for jboss and tomcat within these paths. >>> If you do the math, 25 x 10 x 2 = 500. That's therefore (currently) >>> 500 unique files that are being managed in these deep directory >>> structures. Could that potentially be the reason behind puppets crap >>> performance? >> >> What do you manage for those files? >> But no, 500 doesn't seem like a high number to me. >> >> You mentioned in another e-mail in this thread that the problem is more >> the 20 minutes run than the CPU. >> Could it be possible you have many "slow" execs? >> Or you manage many packages? >> >> This also reminds me Ohad's bug: >> http://projects.reductivelabs.com/issues/1719 >> >> At this stage you should probably run puppetd on the console in --debug >> to see what happens (and run with --summarize too) and if it stalls. > > I just ran puppet in debug mode and it was obvious that most of the > puppet run time was spent in checksumming files. > > Eg: > > debug: > //Node[app01.fr.xxx.com]/Jboss::Instance[tfel8]/File[/opt/jboss/current/server/tfel8/conf/jboss.web/localhost/rewrite.properties]: > Creating checksum {md5}f5d16bcc20b92631eb59514018fd34e5 > > ... takes a long time to run. Multiple that by several hundred files... > > However, when I run this on the command line: > md5sum > /opt/jboss/current/server/tfel8/conf/jboss.web/localhost/rewrite.properties > > ... the result is instananeous... So... is puppet using a ruby library > for performing md5 checksums? Is that where the performance bottle > neck could be? > > Doug >
Also... I just grabbed an example online of performing an md5 checksum on a file in ruby. Ran it on the same file above. Result was instananeous... the question remains... what is puppet doing??? Doug -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-us...@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
