I think John is on to something here, Daniel - I haven't seen your
full content yet, but are you creating a file for each nagios_host, so
that you can use that as the 'target'? Thus creating a single file for
each nagios_host entry?
If this is the case, the John is spot-on ... and you're creating more
relationships than necessary. You probably want one file to one
nagios_* resource, and thus one 'edge' for each, not this many-to-many
case you seem to be creating at collection time.
Of course I'm just guessing, not having seen your content.
On Thu, Jan 24, 2013 at 2:41 PM, jcbollinger <john.bollin...@stjude.org> wrote:
> On Thursday, January 24, 2013 5:24:58 AM UTC-6, Daniel wrote:
>>
>> Seems that chaining exported resources might not be too efficient and
>> produces lots of data that could be the reason for puppetdb crashing.
>>
>> The culprits being these two lines in two manifest files:
>>
>> ./nsca/server.pp: #File <<| tag == $get_tag |>> -> Nagios_host <<| tag ==
>> $get_tag |>>
>> ./nrpe/server.pp: #File <<| tag == $get_tag |>> -> Nagios_host <<| tag ==
>> $get_tag |>>
>
>
>
> And there may be your combinatorial problem. Supposing that the tags are
> not specific to the exporting node, the numbers of Files and Nagios_hosts
> increase linearly with the number of nodes, but the number of relationships
> among them increases with the square of the number of nodes.
>
>>
>> replacing them with unchained:
>>
>> File <<| tag == $get_tag |>>
>> Nagios_host <<| tag == $get_tag |>>
>>
>> causes it to run even with 1GB for puppetdb (still a 16GB vm) in under 10
>> mins, which is acceptable.
>
>
> Do you in fact need the order of application constraints that the chaining
> declared? All of them?
>
> This is an area that seems to give people problems. On one hand, Puppet
> provides a couple of mechanisms that allow people to compactly declare large
> numbers of ordering relationships. On the other hand, people sometimes lose
> sight of the distinction between functional dependency and application-order
> dependency. Either one of these can cause trouble, as unneeded declarations
> of any kind slow Puppet and make it consume more resources. They also make
> you manifests more susceptible to dependency cycles. The combination is
> worse, however, because the number of unneeded relationships produced can be
> multiplicative.
>
> Depending on what relationships you actually need, you have various options:
>
> 1) If it doesn't actually matter whether Puppet syncs given Files before or
> after it syncs any particular Nagios_host, then you never needed any
> relationships at all, and simply removing the chaining as you did is the
> best solution.
>
> 2) If specific Files must be synced before specific Nagios_hosts, then you
> should express those relationships and not any others. In particular, if
> you only need relationships among Files and Nagios_hosts exported by the
> same node, then you should declare the relationships on the exporting side
> instead of on the collecting side. (And if they always come in such pairs
> then perhaps you should wrap then in a defined type and export that
> instead).
>
> 3) If you really need something along the lines that the chaining expressed
> -- all Files with a given tag synced before all Nagios_hosts bearing that
> tag -- then you can break the combinatorial problem by introducing an
> artificial barrier resource for each tag:
>
> define mymodule::barrier() {
> # no content
> }
>
> mymodule::barrier { $get_tag: }
>
> File<<| tag == $get_tag |>> -> Mymodule::Barrier[ $get_tag ] -> Nagios_host
> <<| tag == $get_tag |>>
>
> That gives you (number(Files) + number(Nagios_host)) relationships for each
> tag instead of (number(Files) * number(Nagios_host)) relationships.
>
> Of course, you can use any single resource as a barrier; it doesn't need to
> be of a defined type. In some cases there might be a real resource that
> naturally fills the role. If you have Puppet's stdlib module then the
> Anchor resource would be a good fit; it is intended for a similar purpose
> anyway.
>
>
> John
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/2_AvVt-Qm4oJ.
>
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
