Excellent work, Nick - thank you. -- Luke Kanies | +1-615-594-8199 | http://about.me/lak
On May 10, 2012, at 11:44 AM, Nick Lewis <[email protected]> wrote: On Thursday, May 10, 2012 at 10:30 AM, Ashley Penney wrote: On Thu, May 10, 2012 at 1:42 AM, Jeff Weiss <[email protected]>wrote: Fourth, Peter, Émile, and Trevor (or anyone else experiencing the problem), would you be willing to be pre-release testers of improvements? Our ops team is seeing the problem too, but that's only a single real-world data point. We need to make sure we don't self-optimize. We need your help to make sure the performance fixes address *your* problems not just ours. I just wanted to chime in to offer to test any performance patches because I'm definitely interested in helping get to the bottom of any performance drops. We also see the massive memory use during run issues other people have seen which has resulted in us stopping puppet runs on certain over-commited boxes as it can tip them over the edge. Anything that targets either of this areas is something I'll happily hack in by hand for testing purposes. (I have 4 puppetmasters currently so it's easy to target just a few and not the rest for comparisons). On that note: After spending a few hours hunting yesterday, we identified the cause of a significant performance regression in compilation when using modules with metadata. I've attached a patch for anyone who would like to test it out. We're seeing approximately 5-10x faster compilation. The basic gist of the issue is that every time Puppet looks up a class (via include/import/autoload), it constructs a list of possible modules the class could be in (based on its name). For each module, it will create a Puppet::Module instance, so that it can ask which manifests in that module may contain the class. Unfortunately, creating the Puppet::Module instance will cause the metadata.json file to be read and parsed. Given that this happens several times *per include*, it gets expensive quickly. Fortunately, as we already have a list of Puppet::Module objects for all the modules, the simple solution is to just use one of those. Once again, a patch is attached. Please test this and provide feedback. Note that this patch should not be applied to Puppet 2.6, or to 2.7.3 or earlier. Due to a bug in caching, it will cause significantly *worse* performance. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en. -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en. <typeloader-compilation-performance.patch> -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
