On Tue, Jul 3, 2012 at 5:00 AM, Brice Figureau <brice-pup...@daysofwonder.com> wrote: > Hi, > > I'm proud to release a set of unofficial and certainly unstable (in > various ways) patches against both 2.7.x and 3.0.x to improve the > performance of the puppet compiler (and only the compiler). > > Warning: I never tried to use those patches in production. Also those > patches are almost not covered by tests (except those already in place). > > Those patches are not ready for general consumption and upstream merging > (that's why I didn't publish a pull request), but I'd appreciate any > feedback and tests on other sets of manifests to see the potential gains. > My idea was to help PL dev team to produce a more performant puppet > compiler in the future, and I hope those patches could be a good start :)
You're awesome Brice. :) > > If a pull request is a better format for commenting, please let me know. > > Those patches can be found here: > https://github.com/masterzen/puppet/tree/performance-fixes-2.7 > and > https://github.com/masterzen/puppet/tree/performance-fixes-3.0 > > The methodology I used was using the perftools ruby profiler as Daniel > suggested in a previous post, with the set of manifests (back from 0.24 > era) I use in production. > > Those manifests have the following particularity: they are old and > ancient and use almost no new Puppet feature, like paramterized class, > but abuse templates, definitions. Those manifests contains around 100 > modules. > I also have some manually imported definitions that are heavily used (ie > for concatenation for instance). I also have tons of classes not > following the autoloader path scheme. > > So, for sure my optimizations are heavily tailored to my use case and > might not give any improvement to any other cases (although I'd be > surprised if compilation time could get worst with those patches). > > The biggest saver for me was to prevent touching the disk during the > compilation (ie not try to autoload constantly or not try to autoload > something that can't be autoloaded). > > For my tests, I had storeconfigs off (even though the manifests are > collecting exported resources) as this was more convenient for me. > > Here are the results on my manifests (note, the first run includes > manifest parsing which is something I didn't yet profile/optimize, the > other run are without manifests modifications): > > 3.0 > --- > before: > Compiled catalog for corp1 in environment production in 5.38 seconds > Compiled catalog for corp1 in environment production in 3.01 seconds > Compiled catalog for corp1 in environment production in 2.96 seconds > Compiled catalog for corp1 in environment production in 2.93 seconds > Compiled catalog for corp1 in environment production in 2.89 seconds > > after: > Compiled catalog for corp1 in environment production in 4.12 seconds > Compiled catalog for corp1 in environment production in 1.54 seconds > Compiled catalog for corp1 in environment production in 1.51 seconds > Compiled catalog for corp1 in environment production in 1.47 seconds > Compiled catalog for corp1 in environment production in 1.46 seconds > > 2.7 > --- > before: > Compiled catalog for corp1 in environment production in 5.10 seconds > Compiled catalog for corp1 in environment production in 2.72 seconds > Compiled catalog for corp1 in environment production in 2.71 seconds > Compiled catalog for corp1 in environment production in 2.69 seconds > Compiled catalog for corp1 in environment production in 2.68 seconds > > after: > Compiled catalog for corp1 in environment production in 4.62 seconds > Compiled catalog for corp1 in environment production in 1.46 seconds > Compiled catalog for corp1 in environment production in 1.49 seconds > Compiled catalog for corp1 in environment production in 1.42 seconds > Compiled catalog for corp1 in environment production in 1.41 seconds > > As you can see, the speed gain is about 50%, which is not (that) bad. > > On a side note: I had a lot of test failure on 3.0.x with > spec/unit/provider_spec.rb that looks like this: > 223) Puppet::Provider features supports_parameter? should support > two_features with provider two > Failure/Error: type.feature :numeric, '', :methods => [:one, :two] > NoMethodError: > undefined method `feature' for nil:NilClass > # ./spec/unit/provider_spec.rb:518 > > Running the provider_spec by itself doesn't exhibit those failures. > > Thanks for listening :) > Please comment :) > -- > Brice Figureau > My Blog: http://www.masterzen.fr/ > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To post to this group, send email to puppet-dev@googlegroups.com. > To unsubscribe from this group, send email to > puppet-dev+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-dev?hl=en. > -- Nigel Kersten | http://puppetlabs.com | @nigelkersten Schedule Meetings at: http://tungle.me/nigelkersten -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to puppet-dev@googlegroups.com. To unsubscribe from this group, send email to puppet-dev+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
