On Wednesday, May 8, 2013 11:38:06 AM UTC-5, Aaron Mills wrote: > > Thought I'd drag this topic back to life rather than open a new one for > the same issue. I'm seeing pretty much the exact same behavior on my > catalog compile times. With puppet 3.1.0 and hiera-gpg 1.1.0 I'm seeing > compile times usually in the 60-90-second range. This is causing a lot of > agent runs to timeout or get an "end of file" error. Consolidating > GPG-encrypted data into a single file doesn't seem to have any bearing on > compile times. > > Running the master in debug mode doesn't seem to surface any obvious > issues. Has anyone made any headway on this issue? > >
I assumed the first time around that the increased compile times were a function of the relatively large computational cost of cryptography. Nevertheless, there might be some inefficiencies in the way hiera-gpg works internally, in the way it works together with the hiera framework, and in the behavior your manifests provoke from it. Hiera-gpg decrypts each target file it consults, in its entirety, whenever it is queried for a key. How expensive that is depends on the number and size of the files, on the position in the hierarchy where target keys are typically found, and on which hiera access function you actually use. If you use many parameterized classes, then Puppet 3's automatic class parameter binding will tend to aggravate that problem, especially if you typically allow class parameters to take default values or DSL-specified values (so that hiera has lots of complete misses on the parameter names). There are several things you could do to try to mitigate, among them: - Use the :gpg: back-end only for data that really need to be encrypted. Use the plain :yaml: backend for everything else, and give it higher priority. - Minimize use of parameterized classes, or else ensure that all class parameters are recorded in your hiera data files, even if they take default values. - If you use hiera for data other than class parameters, then avoid looking up the same key multiple times. Instead, read the data into some class's variables, and have everyone else get the data from those variables. - Avoid need for hiera_hash() and hiera_array(), each of which will decrypt every one of the :gpg: backend's files on every call. (And do be sure you know the difference between using those functions and using the plain hiera() function to retrieve hashes and arrays.) - If you're willing to get a bit intrusive, then restructure your data and parameters so that fewer overall lookups are required. For instance, combine multiple individual values into hashes, so that you can perform a single lookup for the hash instead of a separate lookup for each component. Good luck, John -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To post to this group, send email to puppet-users@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
