Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-19 Thread Dominic Cleal
On 18/06/12 18:22, Jo Rhett wrote: > On Jun 17, 2012, at 4:22 AM, Dominic Cleal wrote: >> Yes, prior to 3.0 it is a lot slower. The reason's simple, it loads and >> parses every file unless you explicitly tell it (via lens/incl params) >> which file you intend to edit. >> >> 3.0 has an optimisatio

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-18 Thread Jo Rhett
On Jun 17, 2012, at 4:22 AM, Dominic Cleal wrote: > Yes, prior to 3.0 it is a lot slower. The reason's simple, it loads and > parses every file unless you explicitly tell it (via lens/incl params) > which file you intend to edit. > > 3.0 has an optimisation (#7285) so if you use the context param

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-17 Thread Brice Figureau
On 16/06/12 20:42, Patrick Carlisle wrote: > > > On Sat, Jun 16, 2012 at 9:11 AM, Brice Figureau > mailto:brice-pup...@daysofwonder.com>> > wrote: > > On 13/06/12 23:46, Daniel Pittman wrote: > > On Wed, Jun 13, 2012 at 1:32 PM, Peter Meier > mailto:peter.me...@immerda.ch>> wrote: >

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-17 Thread Dominic Cleal
On 12/06/12 11:47, Trevor Vaughan wrote: > 3) Augeas is slower that the native concat type that we posted to > Github (though that needs some polish) Yes, prior to 3.0 it is a lot slower. The reason's simple, it loads and parses every file unless you explicitly tell it (via lens/incl params) whic

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-16 Thread Patrick Carlisle
On Sat, Jun 16, 2012 at 9:11 AM, Brice Figureau < brice-pup...@daysofwonder.com> wrote: > On 13/06/12 23:46, Daniel Pittman wrote: > > On Wed, Jun 13, 2012 at 1:32 PM, Peter Meier > wrote: > >> > >>> Actually, this is a group where y'all probably *can* deliver on > >>> this request: > >>> > >>> I

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-16 Thread Brice Figureau
On 13/06/12 23:46, Daniel Pittman wrote: > On Wed, Jun 13, 2012 at 1:32 PM, Peter Meier wrote: >> >>> Actually, this is a group where y'all probably *can* deliver on >>> this request: >>> >>> If you are experiencing poor performance, it would be awesome if >>> you could generate some profiling dat

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-14 Thread Brice Figureau
On Wed, 2012-06-13 at 13:52 -0700, Luke Kanies wrote: > On Jun 13, 2012, at 1:22 PM, Peter Meier wrote: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > >> I would really appreciate that. I recently moved a larger > >> installation to 2.7 (latest) and I'm probably going back, as the

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-14 Thread Jo Rhett
This would be a great wiki page ;-) On Jun 13, 2012, at 2:46 PM, Daniel Pittman wrote: > On Wed, Jun 13, 2012 at 1:32 PM, Peter Meier wrote: >> >>> Actually, this is a group where y'all probably *can* deliver on >>> this request: >>> >>> If you are experiencing poor performance, it would be awe

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-14 Thread Jo Rhett
On Jun 13, 2012, at 8:52 AM, Luke Kanies wrote: > Would you be willing to share your configs with us so we can do some > performance analysis on them? Starting with known-slow configs would > make a big difference. I am no longer an employee at the shop, but a long time ago I sent our entire ca

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> Although, I do a lot of hiera lookups now, but also switched >> from full storeconfigs to thin storeconfigs, I think the >> difference is huge and I'm wondering what might cause it. > > fwiw hiera isn't caching - there was a caching yaml backend >

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread R.I.Pienaar
- Original Message - > From: "Peter Meier" > To: puppet-dev@googlegroups.com > Sent: Wednesday, June 13, 2012 11:21:47 PM > Subject: Re: [Puppet-dev] why I don't feel that ENCs are very functional > right now (performance) > > -BEGIN PGP SIGNE

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > I committed the caching YAML code to master, and it should be > merged. Just boarding a plane, but can link when I land :) I thought that have already be included in one of the 1.0.0 hiera rc-releases? Or are you talking about a different cache? I'

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Daniel Pittman
On Wed, Jun 13, 2012 at 1:32 PM, Peter Meier wrote: > >> Actually, this is a group where y'all probably *can* deliver on >> this request: >> >> If you are experiencing poor performance, it would be awesome if >> you could generate some profiling data for us. >> >> perftools.rb is the nicest profil

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Gary Larizza
On Wednesday, June 13, 2012, Peter Meier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > Actually, this is a group where y'all probably *can* deliver on > > this request: > > > > If you are experiencing poor performance, it would be awesome if > > you could generate some profiling d

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Luke Kanies
On Jun 13, 2012, at 1:22 PM, Peter Meier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > >> I would really appreciate that. I recently moved a larger >> installation to 2.7 (latest) and I'm probably going back, as the >> slow down + extended memory usage is just not worth it. >> >> Co

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > Actually, this is a group where y'all probably *can* deliver on > this request: > > If you are experiencing poor performance, it would be awesome if > you could generate some profiling data for us. > > perftools.rb is the nicest profiler for Ruby I

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread R.I.Pienaar
- Original Message - > From: "Peter Meier" > To: puppet-dev@googlegroups.com > Sent: Wednesday, June 13, 2012 9:22:30 PM > Subject: Re: [Puppet-dev] why I don't feel that ENCs are very functional > right now (performance) > > -BEGIN PGP SIGNE

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > I would really appreciate that. I recently moved a larger > installation to 2.7 (latest) and I'm probably going back, as the > slow down + extended memory usage is just not worth it. > > Compile time doubled in my case (~60s compared to 27s). We > m

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Daniel Pittman
On Wed, Jun 13, 2012 at 8:52 AM, Luke Kanies wrote: > On Jun 12, 2012, at 8:55 PM, Peter Meier wrote: >> I'm hoping that 3.0 brings some significant performance improvements. >>> >>> We discovered some functionality regressions that we are working >>> on right now and hope to have out o

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Andrew Parker
On Jun 12, 2012, at 4:55 PM, Trevor Vaughan wrote: > The ENC interface appears to be slow in general. > > I can't say much about Dashboard and Foreman given their use of > storedconfigs, so I'm not sure where the bottlenecks are there > exactly. > > It *seems* to be that the base ENC gets read

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-13 Thread Luke Kanies
On Jun 12, 2012, at 8:55 PM, Peter Meier wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > >>> I'm hoping that 3.0 brings some significant performance >>> improvements. >>> >> >> We discovered some functionality regressions that we are working >> on right now and hope to have out of the

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> I'm hoping that 3.0 brings some significant performance >> improvements. >> > > We discovered some functionality regressions that we are working > on right now and hope to have out of the way soon. We've also made > the decision to hold off on re

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Trevor Vaughan
The ENC interface appears to be slow in general. I can't say much about Dashboard and Foreman given their use of storedconfigs, so I'm not sure where the bottlenecks are there exactly. It *seems* to be that the base ENC gets read multiple times instead of cached the first time, but that might jus

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Andrew Parker
On Jun 12, 2012, at 3:47 AM, Trevor Vaughan wrote: > Hi Jo, > > I've experienced some of the same issues that you're highlighting > while using Puppet internally at my company. > > My experience has shown: > > 1) 2.7 does slow things down significantly Unfortunate, but true. If I understand co

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Trevor Vaughan
Oh, I meant writing your own native types to replace the execs. Look at the puppet-stdlib 'file_line' type for an easy to follow example. If you can do this, you'll probably cut your run time dramatically. Trevor On Tue, Jun 12, 2012 at 3:48 PM, Jo Rhett wrote: > On Jun 12, 2012, at 3:47 AM, T

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Jo Rhett
On Jun 12, 2012, at 3:47 AM, Trevor Vaughan wrote: > 4) Calling a large number of Exec's and Service's will cause some sort > of undetermined queueing backup on your clients Yeah, that was our diagnosis of the problem. At the former site, it was necessary to use Execs to test package update issu

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Joshua Lifton
Trevor, Jo, all, Thanks for your thoughts on the state of ENCs. We agree there are numerous problems and are in fact in the process of building a new ENC we hope will solve many if not most of these problems. Certainly, it's still in the prototyping phase and is not nearly ready for production use

Re: [Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Trevor Vaughan
Hi Jo, I've experienced some of the same issues that you're highlighting while using Puppet internally at my company. My experience has shown: 1) 2.7 does slow things down significantly 2) ENC's are too slow to scale except in the case of LDAP where the performance seems relatively reasonable as

[Puppet-dev] why I don't feel that ENCs are very functional right now (performance)

2012-06-12 Thread Jo Rhett
I'm sure this thread is going to take some heat. Obviously what I post here is very YMMV, and I guess that's my point -- MMDV and it just doesn't seem equal just yet. That said, I'd be very happy to be enlightened about things I may have overlooked. So please reply to this thread with a chance t