Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-31 Thread Tristan Smith
I've plenty of possible mechanisms to experiment with for lessening the load, but I honestly don't _have_ to now that I've fixed the caching problem. 20% higher load == the system peaks out at 40%idle, now. I've got some headroom. With the 5s cache, it was bringing the server to its knees. That's n

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-31 Thread Brian Wilkins
What about staggering your runs? It seems trivial but at least it would reduce your load I think. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+u

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-31 Thread Tristan Smith
A followup for closure; I've now turned environment_timeout to 'unlimited' and am doing a graceful restart of httpd to reset passenger. This seems to work. The resultant performance is _much_ more sane; I'm still seeing mebbe 20% increase in server CPU requirements, but that's quite manageable. O

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-30 Thread Konrad Scherer
On 05/27/2014 05:39 PM, Andy Parker wrote: On Tuesday, May 27, 2014 5:53:29 AM UTC-7, Konrad Scherer wrote: On 14-05-22 03:21 PM, Daniele Sluijters wrote: > The environment caching is already there, use the environment_timeout > setting. Mine is set to unlimited and I reload at dep

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-27 Thread Andy Parker
On Tuesday, May 27, 2014 5:53:29 AM UTC-7, Konrad Scherer wrote: > > On 14-05-22 03:21 PM, Daniele Sluijters wrote: > > The environment caching is already there, use the environment_timeout > > setting. Mine is set to unlimited and I reload at deploy time by > > touching tmp/restart.txt. This so

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-27 Thread Konrad Scherer
On 14-05-22 03:21 PM, Daniele Sluijters wrote: The environment caching is already there, use the environment_timeout setting. Mine is set to unlimited and I reload at deploy time by touching tmp/restart.txt. This so far seems to work really well. Thanks for the suggestion. I have also been deal

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-25 Thread Tristan Smith
I've been using the default (which if I read correctly is 5s?); seems likely that a good answer here for me would be to push it out to $something_long and use restart.txt or somesuch to uptake changes. I may try that this week. My MaxRequests was already 10k. I'm guessing that 5 seconds of cache w

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-23 Thread Chuck
I had it set to 3 minutes for this test. I am going to work on trying some different settings. So far the best result has been with the following settings PassengerMaxRequests 5000 environment_timeout 15 minutes "service httpd graceful" whenever my svn update script detects a change. I will be

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-23 Thread Josh Partlow
Hi Tristan, Chuck, What environment_timeout do you have set currently? On Friday, May 23, 2014 12:20:33 PM UTC-7, Tristan Smith wrote: > > Whuf. I'm already at 1, so that's not it for me. > > Seems like the doubled run time was just murdering me in the face. Trying > to run 1000 clients i

Re: [Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-23 Thread Tristan Smith
Whuf. I'm already at 1, so that's not it for me. Seems like the doubled run time was just murdering me in the face. Trying to run 1000 clients in 30m was just too much. Guess I could make it an hour splay, but the overall resource requirement change is sufficiently large I'm probably just go

[Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-23 Thread Chuck
I am noticing increase CPU and Memory requirements from Puppet 3.4.3 I had to set the following passenger config so that my server would not run out of memory: PassengerMaxRequets 1 CPU idle 3.4.3 = 70% - 80% Memory utilization = 52% CPU idle 3.6.1 = 60% - 73% Memory utilization = 85% --

[Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-22 Thread Ellison Marks
Ah, whoops. Shame on *me* for not remembering what version they were talking about in that thread. On Thursday, May 22, 2014 12:21:37 PM UTC-7, Daniele Sluijters wrote: > > The environment caching is already there, use the environment_timeout > setting. Mine is set to unlimited and I reload at d

[Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-22 Thread Daniele Sluijters
The environment caching is already there, use the environment_timeout setting. Mine is set to unlimited and I reload at deploy time by touching tmp/restart.txt. This so far seems to work really well. On Thursday, 22 May 2014 19:26:47 UTC+2, Tristan Smith wrote: > > Dang. That does look an awful

[Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-22 Thread Tristan Smith
Dang. That does look an awful lot like my issue. I am in fact using directory environments (mostly because of the screaming deprecation warnings telling me I was a bad man if I didn't). :/ Shame on me for using a .0 release. On Thursday, May 22, 2014 10:17:24 AM UTC-7, Ellison Marks wrote: > >

[Puppet Users] Re: Puppet 3.6.0... and scaling?

2014-05-22 Thread Ellison Marks
Hey, I think I remember another thread that mentioned that there were some performance issues with directory environments. Basically, the next 3.6 release will add a caching option that mostly alleviated the problem for the OP from that thread. https://groups.google.com/forum/#!topic/puppet-use