Re: [Puppet Users] Re: Catalog compile times 40s - Puppet 2.7.18
On Dec 22, 2012 11:04 PM, Michael Stahnke stah...@puppetlabs.com wrote:
Puppet 3 has lots of performance improvements. In many cases it's
faster than 2.6.x was. Some of those fixes were back-ported to 2.7.20,
but not all of them could be.
For best performance, try out something in the Puppet 3 series.
On Wed, Dec 19, 2012 at 8:30 AM, shurik a.grus...@gmail.com wrote:
Hi
We are tried to update puppet master from 2.6.6 to 2.7.18 and saw
similar
picture. Catalog
compile time took ~60-120 seconds. Before this update compilation took
about
~1-5 second.
After upgrade average CPU usage increased from 10-20% up to 20-40%,
during
some activity
(compilation?) ruby process puppetmasterd take 100% too.
All, is there are some magic to solve or this is expected behaviour?
Thanks!
---
Alexander
On Saturday, December 15, 2012 10:23:49 PM UTC+4, treydock wrote:
On Sunday, December 9, 2012 1:41:34 PM UTC-6, treydock wrote:
I am finding that the puppetlabs-apache module is somehow adding 30-60
seconds onto a host's catalog compile time when the puppetmaster has
no
other hosts contacting or generating catalogs. The Puppetmaster is
setup to
use Puppet-2.7.18 - Apache Passenger. RIght now only 2 hosts are
even
configured to use this new PM, the PM itself and a Foreman host. With
neither hosts contacting the PM, and disabling the include apache
and
require apache::mod::dev in puppetlabs-passenger, I get Config
retrieval:
2.45.
Catalogs had hostnames replaced, puppetmaster.tld is the CNAME for
Puppet
master and puppetmaster-host.tld is the actual hostname.
Catalog: http://pastebin.com/V29JmB8J
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:05:35 2012: notice: Finished catalog run in 2.43 seconds
Sun Dec 9 13:05:35 2012: Changes:
Sun Dec 9 13:05:35 2012: Events:
Sun Dec 9 13:05:35 2012: Total: 1
Sun Dec 9 13:05:35 2012: Noop: 1
Sun Dec 9 13:05:35 2012: Resources:
Sun Dec 9 13:05:35 2012: Out of sync: 1
Sun Dec 9 13:05:35 2012: Total: 22
Sun Dec 9 13:05:35 2012: Skipped: 6
Sun Dec 9 13:05:35 2012: Time:
Sun Dec 9 13:05:35 2012: Resources: 0.00
Sun Dec 9 13:05:35 2012:Filebucket: 0.00
Sun Dec 9 13:05:35 2012: Package: 0.00
Sun Dec 9 13:05:35 2012: File: 0.00
Sun Dec 9 13:05:35 2012: Firewall: 0.01
Sun Dec 9 13:05:35 2012: Exec: 0.14
Sun Dec 9 13:05:35 2012: Last run: 1355079935
Sun Dec 9 13:05:35 2012:Config retrieval: 2.45
Sun Dec 9 13:05:35 2012: Total: 2.61
Sun Dec 9 13:05:35 2012: Version:
Sun Dec 9 13:05:35 2012:Config: 1355077701
Sun Dec 9 13:05:35 2012:Puppet: 2.7.18
Uncommenting include apache and require apache::mod::dev in
puppetlabs-passenger about 40 seconds are added to the compile time.
I went
through disabled all modules and parameters, and found that enabling
the
passenger module alone with Apache included caused the huge delay.
Catalog: http://pastebin.com/wwcKgX5b
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:13:08 2012: notice: Finished catalog run in 2.23 seconds
Sun Dec 9 13:13:08 2012: Changes:
Sun Dec 9 13:13:08 2012: Events:
Sun Dec 9 13:13:08 2012: Total: 4
Sun Dec 9 13:13:08 2012: Noop: 4
Sun Dec 9 13:13:08 2012: Resources:
Sun Dec 9 13:13:08 2012: Out of sync: 4
Sun Dec 9 13:13:08 2012: Skipped: 6
Sun Dec 9 13:13:08 2012: Total: 80
Sun Dec 9 13:13:08 2012: Time:
Sun Dec 9 13:13:08 2012:Filebucket: 0.00
Sun Dec 9 13:13:08 2012: Resources: 0.00
Sun Dec 9 13:13:08 2012: Package: 0.00
Sun Dec 9 13:13:08 2012: Firewall: 0.01
Sun Dec 9 13:13:08 2012: File: 0.01
Sun Dec 9 13:13:08 2012: A2mod: 0.03
Sun Dec 9 13:13:08 2012: Service: 0.07
Sun Dec 9 13:13:08 2012: Exec: 0.14
Sun Dec 9 13:13:08 2012: Last run: 1355080388
Sun Dec 9 13:13:08 2012:Config retrieval: 42.79
Sun Dec 9 13:13:08 2012: Total: 43.06
Sun Dec 9 13:13:08 2012: Version:
Sun Dec 9 13:13:08 2012:Config: 1355080261
Sun Dec 9 13:13:08 2012:Puppet: 2.7.18
The install is via RPM. I'm using the Fedora SRPMs rebuilt for
CentOS 6.
The Puppet master has two environments, development and production
with this
as the puppet.conf
# cat /etc/puppet/puppet.conf
---
[main]
server= puppetmaster.tld
logdir= /var/log/puppet
vardir= /var/lib/puppet
ssldir= /var/lib/puppet/ssl
rundir= /var/run/puppet
factpath =
Re: [Puppet Users] Re: Catalog compile times 40s - Puppet 2.7.18
Puppet 3 has lots of performance improvements. In many cases it's
faster than 2.6.x was. Some of those fixes were back-ported to 2.7.20,
but not all of them could be.
For best performance, try out something in the Puppet 3 series.
On Wed, Dec 19, 2012 at 8:30 AM, shurik a.grus...@gmail.com wrote:
Hi
We are tried to update puppet master from 2.6.6 to 2.7.18 and saw similar
picture. Catalog
compile time took ~60-120 seconds. Before this update compilation took about
~1-5 second.
After upgrade average CPU usage increased from 10-20% up to 20-40%, during
some activity
(compilation?) ruby process puppetmasterd take 100% too.
All, is there are some magic to solve or this is expected behaviour?
Thanks!
---
Alexander
On Saturday, December 15, 2012 10:23:49 PM UTC+4, treydock wrote:
On Sunday, December 9, 2012 1:41:34 PM UTC-6, treydock wrote:
I am finding that the puppetlabs-apache module is somehow adding 30-60
seconds onto a host's catalog compile time when the puppetmaster has no
other hosts contacting or generating catalogs. The Puppetmaster is setup to
use Puppet-2.7.18 - Apache Passenger. RIght now only 2 hosts are even
configured to use this new PM, the PM itself and a Foreman host. With
neither hosts contacting the PM, and disabling the include apache and
require apache::mod::dev in puppetlabs-passenger, I get Config retrieval:
2.45.
Catalogs had hostnames replaced, puppetmaster.tld is the CNAME for Puppet
master and puppetmaster-host.tld is the actual hostname.
Catalog: http://pastebin.com/V29JmB8J
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:05:35 2012: notice: Finished catalog run in 2.43 seconds
Sun Dec 9 13:05:35 2012: Changes:
Sun Dec 9 13:05:35 2012: Events:
Sun Dec 9 13:05:35 2012: Total: 1
Sun Dec 9 13:05:35 2012: Noop: 1
Sun Dec 9 13:05:35 2012: Resources:
Sun Dec 9 13:05:35 2012: Out of sync: 1
Sun Dec 9 13:05:35 2012: Total: 22
Sun Dec 9 13:05:35 2012: Skipped: 6
Sun Dec 9 13:05:35 2012: Time:
Sun Dec 9 13:05:35 2012: Resources: 0.00
Sun Dec 9 13:05:35 2012:Filebucket: 0.00
Sun Dec 9 13:05:35 2012: Package: 0.00
Sun Dec 9 13:05:35 2012: File: 0.00
Sun Dec 9 13:05:35 2012: Firewall: 0.01
Sun Dec 9 13:05:35 2012: Exec: 0.14
Sun Dec 9 13:05:35 2012: Last run: 1355079935
Sun Dec 9 13:05:35 2012:Config retrieval: 2.45
Sun Dec 9 13:05:35 2012: Total: 2.61
Sun Dec 9 13:05:35 2012: Version:
Sun Dec 9 13:05:35 2012:Config: 1355077701
Sun Dec 9 13:05:35 2012:Puppet: 2.7.18
Uncommenting include apache and require apache::mod::dev in
puppetlabs-passenger about 40 seconds are added to the compile time. I went
through disabled all modules and parameters, and found that enabling the
passenger module alone with Apache included caused the huge delay.
Catalog: http://pastebin.com/wwcKgX5b
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:13:08 2012: notice: Finished catalog run in 2.23 seconds
Sun Dec 9 13:13:08 2012: Changes:
Sun Dec 9 13:13:08 2012: Events:
Sun Dec 9 13:13:08 2012: Total: 4
Sun Dec 9 13:13:08 2012: Noop: 4
Sun Dec 9 13:13:08 2012: Resources:
Sun Dec 9 13:13:08 2012: Out of sync: 4
Sun Dec 9 13:13:08 2012: Skipped: 6
Sun Dec 9 13:13:08 2012: Total: 80
Sun Dec 9 13:13:08 2012: Time:
Sun Dec 9 13:13:08 2012:Filebucket: 0.00
Sun Dec 9 13:13:08 2012: Resources: 0.00
Sun Dec 9 13:13:08 2012: Package: 0.00
Sun Dec 9 13:13:08 2012: Firewall: 0.01
Sun Dec 9 13:13:08 2012: File: 0.01
Sun Dec 9 13:13:08 2012: A2mod: 0.03
Sun Dec 9 13:13:08 2012: Service: 0.07
Sun Dec 9 13:13:08 2012: Exec: 0.14
Sun Dec 9 13:13:08 2012: Last run: 1355080388
Sun Dec 9 13:13:08 2012:Config retrieval: 42.79
Sun Dec 9 13:13:08 2012: Total: 43.06
Sun Dec 9 13:13:08 2012: Version:
Sun Dec 9 13:13:08 2012:Config: 1355080261
Sun Dec 9 13:13:08 2012:Puppet: 2.7.18
The install is via RPM. I'm using the Fedora SRPMs rebuilt for CentOS 6.
The Puppet master has two environments, development and production with this
as the puppet.conf
# cat /etc/puppet/puppet.conf
---
[main]
server= puppetmaster.tld
logdir= /var/log/puppet
vardir= /var/lib/puppet
ssldir= /var/lib/puppet/ssl
rundir= /var/run/puppet
factpath = $vardir/lib/facter
pluginsync= true
templatedir = $confdir/templates
[agent]
report = true
environment =
[Puppet Users] Re: Catalog compile times 40s - Puppet 2.7.18
Hi
We are tried to update puppet master from 2.6.6 to 2.7.18 and saw similar
picture. Catalog
compile time took ~60-120 seconds. Before this update compilation took
about ~1-5 second.
After upgrade average CPU usage increased from 10-20% up to 20-40%, during
some activity
(compilation?) ruby process puppetmasterd take 100% too.
All, is there are some magic to solve or this is expected behaviour?
Thanks!
---
Alexander
On Saturday, December 15, 2012 10:23:49 PM UTC+4, treydock wrote:
On Sunday, December 9, 2012 1:41:34 PM UTC-6, treydock wrote:
I am finding that the puppetlabs-apache module is somehow adding 30-60
seconds onto a host's catalog compile time when the puppetmaster has no
other hosts contacting or generating catalogs. The Puppetmaster is setup
to use Puppet-2.7.18 - Apache Passenger. RIght now only 2 hosts are even
configured to use this new PM, the PM itself and a Foreman host. With
neither hosts contacting the PM, and disabling the include apache and
require apache::mod::dev in puppetlabs-passenger, I get Config
retrieval: 2.45.
Catalogs had hostnames replaced, puppetmaster.tld is the CNAME for Puppet
master and puppetmaster-host.tld is the actual hostname.
Catalog: http://pastebin.com/V29JmB8J
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:05:35 2012: notice: Finished catalog run in 2.43 seconds
Sun Dec 9 13:05:35 2012: Changes:
Sun Dec 9 13:05:35 2012: Events:
Sun Dec 9 13:05:35 2012: Total: 1
Sun Dec 9 13:05:35 2012: Noop: 1
Sun Dec 9 13:05:35 2012: Resources:
Sun Dec 9 13:05:35 2012: Out of sync: 1
Sun Dec 9 13:05:35 2012: Total: 22
Sun Dec 9 13:05:35 2012: Skipped: 6
Sun Dec 9 13:05:35 2012: Time:
Sun Dec 9 13:05:35 2012: Resources: 0.00
Sun Dec 9 13:05:35 2012:Filebucket: 0.00
Sun Dec 9 13:05:35 2012: Package: 0.00
Sun Dec 9 13:05:35 2012: File: 0.00
Sun Dec 9 13:05:35 2012: Firewall: 0.01
Sun Dec 9 13:05:35 2012: Exec: 0.14
Sun Dec 9 13:05:35 2012: Last run: 1355079935
Sun Dec 9 13:05:35 2012:Config retrieval: 2.45
Sun Dec 9 13:05:35 2012: Total: 2.61
Sun Dec 9 13:05:35 2012: Version:
Sun Dec 9 13:05:35 2012:Config: 1355077701
Sun Dec 9 13:05:35 2012:Puppet: 2.7.18
Uncommenting include apache and require apache::mod::dev in
puppetlabs-passenger about 40 seconds are added to the compile time. I
went through disabled all modules and parameters, and found that enabling
the passenger module alone with Apache included caused the huge delay.
Catalog: http://pastebin.com/wwcKgX5b
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:13:08 2012: notice: Finished catalog run in 2.23 seconds
Sun Dec 9 13:13:08 2012: Changes:
Sun Dec 9 13:13:08 2012: Events:
Sun Dec 9 13:13:08 2012: Total: 4
Sun Dec 9 13:13:08 2012: Noop: 4
Sun Dec 9 13:13:08 2012: Resources:
Sun Dec 9 13:13:08 2012: Out of sync: 4
Sun Dec 9 13:13:08 2012: Skipped: 6
Sun Dec 9 13:13:08 2012: Total: 80
Sun Dec 9 13:13:08 2012: Time:
Sun Dec 9 13:13:08 2012:Filebucket: 0.00
Sun Dec 9 13:13:08 2012: Resources: 0.00
Sun Dec 9 13:13:08 2012: Package: 0.00
Sun Dec 9 13:13:08 2012: Firewall: 0.01
Sun Dec 9 13:13:08 2012: File: 0.01
Sun Dec 9 13:13:08 2012: A2mod: 0.03
Sun Dec 9 13:13:08 2012: Service: 0.07
Sun Dec 9 13:13:08 2012: Exec: 0.14
Sun Dec 9 13:13:08 2012: Last run: 1355080388
Sun Dec 9 13:13:08 2012:Config retrieval: 42.79
Sun Dec 9 13:13:08 2012: Total: 43.06
Sun Dec 9 13:13:08 2012: Version:
Sun Dec 9 13:13:08 2012:Config: 1355080261
Sun Dec 9 13:13:08 2012:Puppet: 2.7.18
The install is via RPM. I'm using the Fedora SRPMs rebuilt for CentOS 6.
The Puppet master has two environments, development and production with
this as the puppet.conf
# cat /etc/puppet/puppet.conf
---
[main]
server= puppetmaster.tld
logdir= /var/log/puppet
vardir= /var/lib/puppet
ssldir= /var/lib/puppet/ssl
rundir= /var/run/puppet
factpath = $vardir/lib/facter
pluginsync= true
templatedir = $confdir/templates
[agent]
report = true
environment = production
preferred_serialization_format = yaml
[master]
modulepath=
/etc/puppet/environments/$environment/modules:/etc/puppet/modules
manifest = /etc/puppet/manifests/site.pp
environment = production
autosign = false
[Puppet Users] Re: Catalog compile times 40s - Puppet 2.7.18
On Sunday, December 9, 2012 1:41:34 PM UTC-6, treydock wrote:
I am finding that the puppetlabs-apache module is somehow adding 30-60
seconds onto a host's catalog compile time when the puppetmaster has no
other hosts contacting or generating catalogs. The Puppetmaster is setup
to use Puppet-2.7.18 - Apache Passenger. RIght now only 2 hosts are even
configured to use this new PM, the PM itself and a Foreman host. With
neither hosts contacting the PM, and disabling the include apache and
require apache::mod::dev in puppetlabs-passenger, I get Config
retrieval: 2.45.
Catalogs had hostnames replaced, puppetmaster.tld is the CNAME for Puppet
master and puppetmaster-host.tld is the actual hostname.
Catalog: http://pastebin.com/V29JmB8J
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:05:35 2012: notice: Finished catalog run in 2.43 seconds
Sun Dec 9 13:05:35 2012: Changes:
Sun Dec 9 13:05:35 2012: Events:
Sun Dec 9 13:05:35 2012: Total: 1
Sun Dec 9 13:05:35 2012: Noop: 1
Sun Dec 9 13:05:35 2012: Resources:
Sun Dec 9 13:05:35 2012: Out of sync: 1
Sun Dec 9 13:05:35 2012: Total: 22
Sun Dec 9 13:05:35 2012: Skipped: 6
Sun Dec 9 13:05:35 2012: Time:
Sun Dec 9 13:05:35 2012: Resources: 0.00
Sun Dec 9 13:05:35 2012:Filebucket: 0.00
Sun Dec 9 13:05:35 2012: Package: 0.00
Sun Dec 9 13:05:35 2012: File: 0.00
Sun Dec 9 13:05:35 2012: Firewall: 0.01
Sun Dec 9 13:05:35 2012: Exec: 0.14
Sun Dec 9 13:05:35 2012: Last run: 1355079935
Sun Dec 9 13:05:35 2012:Config retrieval: 2.45
Sun Dec 9 13:05:35 2012: Total: 2.61
Sun Dec 9 13:05:35 2012: Version:
Sun Dec 9 13:05:35 2012:Config: 1355077701
Sun Dec 9 13:05:35 2012:Puppet: 2.7.18
Uncommenting include apache and require apache::mod::dev in
puppetlabs-passenger about 40 seconds are added to the compile time. I
went through disabled all modules and parameters, and found that enabling
the passenger module alone with Apache included caused the huge delay.
Catalog: http://pastebin.com/wwcKgX5b
# puppet agent --{summarize,test,debug,evaltrace,noop} | perl -pe
's/^/localtime().: /e'
snip
Sun Dec 9 13:13:08 2012: notice: Finished catalog run in 2.23 seconds
Sun Dec 9 13:13:08 2012: Changes:
Sun Dec 9 13:13:08 2012: Events:
Sun Dec 9 13:13:08 2012: Total: 4
Sun Dec 9 13:13:08 2012: Noop: 4
Sun Dec 9 13:13:08 2012: Resources:
Sun Dec 9 13:13:08 2012: Out of sync: 4
Sun Dec 9 13:13:08 2012: Skipped: 6
Sun Dec 9 13:13:08 2012: Total: 80
Sun Dec 9 13:13:08 2012: Time:
Sun Dec 9 13:13:08 2012:Filebucket: 0.00
Sun Dec 9 13:13:08 2012: Resources: 0.00
Sun Dec 9 13:13:08 2012: Package: 0.00
Sun Dec 9 13:13:08 2012: Firewall: 0.01
Sun Dec 9 13:13:08 2012: File: 0.01
Sun Dec 9 13:13:08 2012: A2mod: 0.03
Sun Dec 9 13:13:08 2012: Service: 0.07
Sun Dec 9 13:13:08 2012: Exec: 0.14
Sun Dec 9 13:13:08 2012: Last run: 1355080388
Sun Dec 9 13:13:08 2012:Config retrieval: 42.79
Sun Dec 9 13:13:08 2012: Total: 43.06
Sun Dec 9 13:13:08 2012: Version:
Sun Dec 9 13:13:08 2012:Config: 1355080261
Sun Dec 9 13:13:08 2012:Puppet: 2.7.18
The install is via RPM. I'm using the Fedora SRPMs rebuilt for CentOS 6.
The Puppet master has two environments, development and production with
this as the puppet.conf
# cat /etc/puppet/puppet.conf
---
[main]
server= puppetmaster.tld
logdir= /var/log/puppet
vardir= /var/lib/puppet
ssldir= /var/lib/puppet/ssl
rundir= /var/run/puppet
factpath = $vardir/lib/facter
pluginsync= true
templatedir = $confdir/templates
[agent]
report = true
environment = production
preferred_serialization_format = yaml
[master]
modulepath=
/etc/puppet/environments/$environment/modules:/etc/puppet/modules
manifest = /etc/puppet/manifests/site.pp
environment = production
autosign = false
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
certname = puppetmaster.tld
dns_alt_names = puppetmaster.tld,puppetmaster-host.tld
report= true
reports = store
#reports = store, foreman
[production]
manifest = /etc/puppet/manifests/site.pp
Has anyone run into this issue with puppetlabs-apache or know why my
compile times are so long? I have a
