On 2014-12-02 9:43, Brice Figureau wrote:
On Wed, 2014-02-12 at 00:13 +0100, Henrik Lindberg wrote:
On 2014-11-02 17:25, Brice Figureau wrote:
I'm currently working on marking all resources with a kind attribute
that can be either node, class, definition or builtin. This property
gets sent over the catalog (which requires a schema modification, so
might not be mergeable in a 3.x release). This allows the client to not
try to guess a given resource type.
I'm marking those resources at creation time (so the settings resource
for instance get marked as builtin), and try to lookup the type of the
resource accordingly (defaulting to the previous way of doing things if
the resource is not marked).
Unfortunately, I discovered that Puppet::Resource was so central in
Puppet that everyone creates new instances of those for various
completely different task, and most of the time it's not possible to
know in advance the resource kind.
The last one I found is when looking up a resource in a given catalog,
we create a blank resource of the given title to be able to get a
properly munged title back to perform the look up. I find that
particularly bad performance-wise, creating so many transient objects
(might be better with latest ruby GC though).
I hope to have a PR ready by the end of the next week end, but at least
you know what I'm trying to do :)
Discussed this with zaphond42 today and I want to try an approach that
is different - since we will be visiting all directories at least once
when looking for resource type implementations in ruby, we could just as
well just cache their names up front. That way, subsequent searches are
simply a hash lookup, and we can continue to lookup ruby plugin resource
types before user defined.
This works when we change the semantics to perform a stale check only
before each request is processed. I am counting on that there are far
fewer plugin types than there are definitions.
Since the plan is to not load all of them upfront, but only record their
existence the entire operation should be fast enough even if it will map
unused types.
That sounds great, if it solves the agent case too. IMHO we are focusing
a bit too much on the master performance :)
I think it will.
Lookup of user defined definitions could also benefit from similar
caching of directory entries - with the goal of only visiting each
directory once, but it is a bit more complicated to implement.
The current way of loading definition works fine, because it is done
during parsing, so the resource lookup is just a hash lookup, it doesn't
involve reparsing everything. In the agent, the known_resource_types is
empty and that means we constantly hit the type loader. If the type
loading scheme you propose also works in the agent, then we'll be mostly
fine except we won't be able to cache the definitions resources types
(and thus will hit the type loader every time). That was the root of my
ideas of marking resources, because once we know what kind a resource
is, everything becomes simpler, as you don't have to guess what kind of
type your resource is, and it can be looked up more easily.
Not sure what you are saying about known_resource_type - are you saying
it is always empty? What I am proposing will result in 1 scan of all
plugin resource types - then when a lookup of a type name is performed
it first checks if this could be a plugin type (one hash lookup), if not
then it must be a user defined type. Is there something in that you
suspect will not work on the agent side?
- henrik
--
You received this message because you are subscribed to the Google Groups "Puppet
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/ldfvmj%24qmb%241%40ger.gmane.org.
For more options, visit https://groups.google.com/groups/opt_out.
