Re: [Puppet Users] Why agent report is so slow?
Sometimes we use puppet agent --test --environment xxx to debug puppet configuration. The 2 minutes and more execution time is unacceptable. As agent output is enough, we do not need the report that time. This is why i want to disable the reports. On Mon, Sep 17, 2012 at 11:44 PM, Daniel Pittman wrote: > The agent builds up an in-memory report no matter what; you can't avoid > that. > > You could write a "null" report terminus that responds to save by > simply discarding the data, though. That would remove the cost of > YAML serialization. > > I guess the key question, though, is why you feel you need this? > > On Mon, Sep 17, 2012 at 12:14 AM, flex wrote: > > Thank you, Daniel. > > Then how can we make puppet not generate the reports? I try to add > > 'report=false', but seems no effect. > > > > On Sat, Sep 15, 2012 at 2:46 AM, Daniel Pittman > > wrote: > >> > >> On Fri, Sep 14, 2012 at 3:16 AM, flex wrote: > >> > Thanks, Daniel. We do have some recursive file copies and hundreds of > >> > other > >> > resources. So we'd better decrease the resources numbers? > >> > >> Well, that would be the only short term path to making report output > >> faster - but, in general, if you are managing it you probably need to > >> be managing it. :) > >> > >> -- > >> Daniel Pittman > >> ⎋ Puppet Labs Developer – http://puppetlabs.com > >> ♲ Made with 100 percent post-consumer electrons > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Puppet Users" group. > >> To post to this group, send email to puppet-users@googlegroups.com. > >> To unsubscribe from this group, send email to > >> puppet-users+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >> http://groups.google.com/group/puppet-users?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Puppet Users" group. > > To post to this group, send email to puppet-users@googlegroups.com. > > To unsubscribe from this group, send email to > > puppet-users+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/puppet-users?hl=en. > > > > -- > Daniel Pittman > ⎋ Puppet Labs Developer – http://puppetlabs.com > ♲ Made with 100 percent post-consumer electrons > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Why agent report is so slow?
The agent builds up an in-memory report no matter what; you can't avoid that. You could write a "null" report terminus that responds to save by simply discarding the data, though. That would remove the cost of YAML serialization. I guess the key question, though, is why you feel you need this? On Mon, Sep 17, 2012 at 12:14 AM, flex wrote: > Thank you, Daniel. > Then how can we make puppet not generate the reports? I try to add > 'report=false', but seems no effect. > > On Sat, Sep 15, 2012 at 2:46 AM, Daniel Pittman > wrote: >> >> On Fri, Sep 14, 2012 at 3:16 AM, flex wrote: >> > Thanks, Daniel. We do have some recursive file copies and hundreds of >> > other >> > resources. So we'd better decrease the resources numbers? >> >> Well, that would be the only short term path to making report output >> faster - but, in general, if you are managing it you probably need to >> be managing it. :) >> >> -- >> Daniel Pittman >> ⎋ Puppet Labs Developer – http://puppetlabs.com >> ♲ Made with 100 percent post-consumer electrons >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To post to this group, send email to puppet-users@googlegroups.com. >> To unsubscribe from this group, send email to >> puppet-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/puppet-users?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. -- Daniel Pittman ⎋ Puppet Labs Developer – http://puppetlabs.com ♲ Made with 100 percent post-consumer electrons -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Why agent report is so slow?
Thank you, Daniel. Then how can we make puppet not generate the reports? I try to add 'report=false', but seems no effect. On Sat, Sep 15, 2012 at 2:46 AM, Daniel Pittman wrote: > On Fri, Sep 14, 2012 at 3:16 AM, flex wrote: > > Thanks, Daniel. We do have some recursive file copies and hundreds of > other > > resources. So we'd better decrease the resources numbers? > > Well, that would be the only short term path to making report output > faster - but, in general, if you are managing it you probably need to > be managing it. :) > > -- > Daniel Pittman > ⎋ Puppet Labs Developer – http://puppetlabs.com > ♲ Made with 100 percent post-consumer electrons > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Why agent report is so slow?
On Fri, Sep 14, 2012 at 3:16 AM, flex wrote: > Thanks, Daniel. We do have some recursive file copies and hundreds of other > resources. So we'd better decrease the resources numbers? Well, that would be the only short term path to making report output faster - but, in general, if you are managing it you probably need to be managing it. :) -- Daniel Pittman ⎋ Puppet Labs Developer – http://puppetlabs.com ♲ Made with 100 percent post-consumer electrons -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Why agent report is so slow?
Thanks, Daniel. We do have some recursive file copies and hundreds of other resources. So we'd better decrease the resources numbers? On Fri, Sep 14, 2012 at 2:44 AM, Daniel Pittman wrote: > On Thu, Sep 13, 2012 at 12:31 AM, flex wrote: > > > > I use cron to run puppet agent, but recently i found the script takes too > > long time. After using '--debug', i got this log: > > > > 2012-09-13 15:19:53.928744500 debug: Finishing transaction 69941650538440 > > 2012-09-13 15:19:53.928988500 debug: Storing state > > 2012-09-13 15:19:54.930995500 debug: Stored state in 1.00 seconds > > 2012-09-13 15:19:54.931145500 notice: Finished catalog run in 48.21 > seconds > > 2012-09-13 15:20:03.721411500 debug: Value of > > 'preferred_serialization_format' (pson) is invalid for report, using > default > > (b64_zlib_yaml) > > 2012-09-13 15:20:03.721578500 debug: report supports formats: > b64_zlib_yaml > > raw yaml; using b64_zlib_yaml > > 2012-09-13 15:21:10.805932500 Puppet run done. > > > > The last 'Puppet run done' is a bash echo after puppet agent. So we can > see > > it took 1 minute and more before the echo line. But if i set 'report = > > false' in puppet.conf, there is no such long time. Does puppet use this > time > > to generate and send reports? > > In essence, yes: we build the report structure as we go, but > transforming it for submission, and sending it over the wire, > certainly take time. That is proportional to the number of resources > you have in your catalog, including things like recursive file copies > or tidies. > > >How can i tune this? > > There really isn't any way to improve performance: there isn't (much) > you can do to improve YAML output performance - although Telly, Puppet > 3.0.0, will be a bit faster at building YAML when it is released. > > You could check to see if the performance problem comes from network > transmission problems, but really there isn't much you can do to > improve a single HTTPS request, which is how we submit the report back > to the the master. > > -- > Daniel Pittman > ⎋ Puppet Labs Developer – http://puppetlabs.com > ♲ Made with 100 percent post-consumer electrons > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To post to this group, send email to puppet-users@googlegroups.com. > To unsubscribe from this group, send email to > puppet-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/puppet-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Why agent report is so slow?
On Thu, Sep 13, 2012 at 12:31 AM, flex wrote: > > I use cron to run puppet agent, but recently i found the script takes too > long time. After using '--debug', i got this log: > > 2012-09-13 15:19:53.928744500 debug: Finishing transaction 69941650538440 > 2012-09-13 15:19:53.928988500 debug: Storing state > 2012-09-13 15:19:54.930995500 debug: Stored state in 1.00 seconds > 2012-09-13 15:19:54.931145500 notice: Finished catalog run in 48.21 seconds > 2012-09-13 15:20:03.721411500 debug: Value of > 'preferred_serialization_format' (pson) is invalid for report, using default > (b64_zlib_yaml) > 2012-09-13 15:20:03.721578500 debug: report supports formats: b64_zlib_yaml > raw yaml; using b64_zlib_yaml > 2012-09-13 15:21:10.805932500 Puppet run done. > > The last 'Puppet run done' is a bash echo after puppet agent. So we can see > it took 1 minute and more before the echo line. But if i set 'report = > false' in puppet.conf, there is no such long time. Does puppet use this time > to generate and send reports? In essence, yes: we build the report structure as we go, but transforming it for submission, and sending it over the wire, certainly take time. That is proportional to the number of resources you have in your catalog, including things like recursive file copies or tidies. >How can i tune this? There really isn't any way to improve performance: there isn't (much) you can do to improve YAML output performance - although Telly, Puppet 3.0.0, will be a bit faster at building YAML when it is released. You could check to see if the performance problem comes from network transmission problems, but really there isn't much you can do to improve a single HTTPS request, which is how we submit the report back to the the master. -- Daniel Pittman ⎋ Puppet Labs Developer – http://puppetlabs.com ♲ Made with 100 percent post-consumer electrons -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
[Puppet Users] Why agent report is so slow?
Hi, all: I use cron to run puppet agent, but recently i found the script takes too long time. After using '--debug', i got this log: 2012-09-13 15:19:53.928744500 debug: Finishing transaction 69941650538440 2012-09-13 15:19:53.928988500 debug: Storing state 2012-09-13 15:19:54.930995500 debug: Stored state in 1.00 seconds 2012-09-13 15:19:54.931145500 notice: Finished catalog run in 48.21 seconds 2012-09-13 15:20:03.721411500 debug: Value of 'preferred_serialization_format' (pson) is invalid for report, using default (b64_zlib_yaml) 2012-09-13 15:20:03.721578500 debug: report supports formats: b64_zlib_yaml raw yaml; using b64_zlib_yaml 2012-09-13 15:21:10.805932500 Puppet run done. The last 'Puppet run done' is a bash echo after puppet agent. So we can see it took 1 minute and more before the echo line. But if i set 'report = false' in puppet.conf, there is no such long time. Does puppet use this time to generate and send reports? How can i tune this? -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
