Issue #4499 has been updated by Markus Roberts. Status changed from Needs more information to Needs design decision
There are tricky design issues here (thread and environment safety, for one thing). It's also the case that there's no such thing as a free lunch in these matters. If, for example, the data is read in once per catalog and stored in a puppet variable, caching wouldn't help (it would actually hurt slightly). Do we have example use cases where the performance impact is being felt, that we might use to think about possible solutions? ---------------------------------------- Refactor #4499: extlookup parses files multiple times http://projects.puppetlabs.com/issues/4499 Author: Alan Barrett Status: Needs design decision Priority: Normal Assigned to: James Turnbull Category: functions Target version: queued Affected version: 0.25.5 Branch: According to "ruby -r profile", more than 50% of the runtime for "puppetmasterd --compile someclient" is in CSV#parse_body. In a typical example, extlookup is called 204 times, CSV::Reader#each is called 225 times (only a little more than the number of calls to extlookup, because almost all variables are found in the first file), CSV#parse_row is called 42486 times (which is not much less than the product of {number of lines in the file} * {number of calls to extlookup}), and CSV#parse_body is called 84747 times (which is approximately double the number of calls to CSV#parse_body). I The code in extlookup.rb appears to read and parse the data files on every call. It would be better if it kept a cache of {file name, variable name, raw value} tuples, invalidated the cache when the file timestamp changed, and otherwise avoided unnecessary re-reading of unchanged files. The decision about which files to search should probably still be done on every call, in case extlookup_datadir or extlookup_precedence has changed. Expansion of "#{variable}" embedded in the raw value should also be done on every call. -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
