Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-25 Thread Daniel
On Thursday, January 24, 2013 2:41:53 PM UTC, jcbollinger wrote: On Thursday, January 24, 2013 5:24:58 AM UTC-6, Daniel wrote: Seems that chaining exported resources might not be too efficient and produces lots of data that could be the reason for puppetdb crashing. The culprits being

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-24 Thread Daniel
Seems that chaining exported resources might not be too efficient and produces lots of data that could be the reason for puppetdb crashing. The culprits being these two lines in two manifest files: ./nsca/server.pp: #File | tag == $get_tag | - Nagios_host | tag == $get_tag | ./nrpe/server.pp:

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-24 Thread jcbollinger
On Thursday, January 24, 2013 5:24:58 AM UTC-6, Daniel wrote: Seems that chaining exported resources might not be too efficient and produces lots of data that could be the reason for puppetdb crashing. The culprits being these two lines in two manifest files: ./nsca/server.pp: #File | tag

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-24 Thread Ken Barber
I think John is on to something here, Daniel - I haven't seen your full content yet, but are you creating a file for each nagios_host, so that you can use that as the 'target'? Thus creating a single file for each nagios_host entry? If this is the case, the John is spot-on ... and you're creating

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-23 Thread Daniel
This is now reported here: http://projects.puppetlabs.com/issues/18804 -- You received this message because you are subscribed to the Google Groups Puppet Users group. To view this discussion on the web visit https://groups.google.com/d/msg/puppet-users/-/ZpyFiFkYjawJ. To post to this group,

[Puppet Users] Re: Terrible exported resources performance

2013-01-22 Thread Daniel
On Monday, January 21, 2013 1:05:26 PM UTC, Daniel wrote: In the larger env it takes about 70 minutes, if it manages to finish at all. Initially, as a quick test, I was running puppetdb without postgres and had to give it 5GB to get it to finish at all (70 mins). With postgres 8.4, load on

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-22 Thread Ken Barber
This sounds like a sensible workaround, I will definitely have a look. I haven't yet had enough time to look at the issue properly, but it seems that this very long time is indeed consumed by catalog construction. Puppetdb fails after this is finished, so it seems that it dies when nagios host

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-22 Thread Daniel Siechniewicz
On Tue, Jan 22, 2013 at 3:04 PM, Ken Barber k...@puppetlabs.com wrote: This sounds like a sensible workaround, I will definitely have a look. I haven't yet had enough time to look at the issue properly, but it seems that this very long time is indeed consumed by catalog construction. Puppetdb

Re: [Puppet Users] Re: Terrible exported resources performance

2013-01-22 Thread Ken Barber
How many resources are we talking for the 100 nodes? If you're running puppetdb one way to do this is to get a report on the nagios server that does the collection: curl -G -H 'Accept: application/json' 'http://puppetdb:8080/resources' --data-urlencode 'query=[=, [node, name], nagiosserver]' | p

[Puppet Users] Re: Terrible exported resources performance

2013-01-21 Thread Luke Bigum
Hi Daniel, On Monday, January 21, 2013 1:05:26 PM UTC, Daniel wrote: In the larger env it takes about 70 minutes, if it manages to finish at all. Initially, as a quick test, I was running puppetdb without postgres and had to give it 5GB to get it to finish at all (70 mins). With postgres