On 13 September 2013 18:52, Henrik Lindberg <henrik.lindb...@cloudsmith.com>wrote:
> Hi, > Ideas regarding a potential performance boost that can be gained by > performing batch processing of package installs/operations has been > floating around in the Puppet echo system for quite some time. > > There is a discussion (and a somewhat dated implementation/proposal) in > http://projects.puppetlabs.**com/issues/2198<http://projects.puppetlabs.com/issues/2198>which > is good background reading for this topic. > > In issue #2198 (if you skipped reading it ;-)), the idea is that Puppet > should have the feature to install a list of packages given by the user. > > It seems doable to generalize this idea and let puppet automatically > optimize package installs under certain conditions. Performing individual > package installs is quite expensive and even if the optimization > opportunities may not be extensive (e.g. say that 20% (number completely > made up) of packages could at least be paired with one other package) this > is still a worth while activity. > > To kick this off, we need to do some research and design. So, here is an > attempt to get this started by asking a bunch of questions. > > Under what conditions can two (or more) packages operations be batched? ------------------------------**------------------------------**----------- > In the relationship graph traversal code (puppet/graph/relationship_graph#traverse) resources that are ready to run are put on the queue. I think that it should be possible to batch every package resource currently on the queue when a package resource is encountered. It shouldn't matter which class it comes from or anything. > > What needs to be done to providers? > ------------------------------**----- > Clearly the capability to handle multiple requests must be implemented for > package managers that support this. What should the API look like? > > What needs to be done to the Package type? > ------------------------------**------------ > Is it all an ordering issue and handing off resources to the provider, or > do we need to do things to the Package type as well? > > Are there situations were it is of value to veto batching per resource? > (depending on how much optimization than can be deduced by looking at > resource-dependencies). > I would hope this is some sort of generic feature other types & providers can opt in to. For example database_user could create several users in a single connection without tearing it down between each one. -- Erik Dalén -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To post to this group, send email to puppet-dev@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-dev. For more options, visit https://groups.google.com/groups/opt_out.
