Issue #14759 has been updated by Rich Rauenzahn.
Anyone looking for a workaround, instead of passing a list of dicts to iterate
over a list of data in order create resources like this...
define a_resource {
$foo = $name['key1']
$bar = $name['key2']
}
a_resource { $my_list_of_dicts: }
Write a custom function that transforms your dict into something like:
$enumed = [ $list_of_keys, $dict ]
Where $list_of_keys is something like [ 1,2,3,4 ] and $dict is { 1: { ... }, 2:
{...} }
Then change a_resource to
define a_resource($key=$name, $dict) {
$foo = $dict[$key]['key1']
$bar = $dict[$key]['key2']
}
a_resource { $enumed[0]: dict => $enumed[1] }
Which is really the better way to be doing this anyway rather than relying on
puppet allowing $name to be a dict.
----------------------------------------
Bug #14759: state.yaml balooning when using dictionary as resource $name
https://projects.puppetlabs.com/issues/14759#change-64106
Author: Rich Rauenzahn
Status: Unreviewed
Priority: Normal
Assignee:
Category:
Target version:
Affected Puppet version:
Keywords:
Branch:
We had some puppet processes ballooning and I spent some time trying to figure
out why. Turns out our state.yaml is something like 500MB after the run.
Examining the state.yaml shows a particular resource ballooning that we use for
iterating over a moderately large list of dictionaries.
We use a custom function to generate a list of dictionaries, and then we pass
the dictionary to the resource, the resource picks apart the dictionary,
filters based on some keys, etc., and creates other resources.
I verified the custom function is behaving appropriately -- it creates
something around 180 items. The state.yaml balloons it up to about 400,000 -
kind of a cartesian product I think.
oversimplified example:
$list = generate_list()
define process_list {
if $list['foo'] == 'xxxx' {
file {
...
}
}
}
What I found in the state.yaml was each element (a dict) of $list stringified
and listed as Process_List[] multiple times, but the stringification listed the
dictionary in different orders, causing the same entry to be duplicated, but
represented as a different strinfication of the dict.
My guess is that internally Puppet is stringifying the dictionary in multiple
locations and this causes duplicate resources to be defined when creating the
state to be saved in the state.yaml (I don't think it is doing that for
determining resources to be created; otherwise we'd be seeing duplication
errors).
Anyway, I know it is kind of an odd case we're using here, but the ballooning
is so severe (~4GB) I wanted to report it.
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
