Got it. Just thought of it while reading the e-mail. I'll head over there tomorrow to get it in.
Thanks, Trevor On Thu, Mar 13, 2014 at 10:01 PM, Eric Sorenson < [email protected]> wrote: > Hi Trevor, this is tracked under PUP-523, please feel free to chime in on > that ticket if you have additional use cases/requirements it doesn't > address. > > https://tickets.puppetlabs.com/browse/PUP-523 > > > --eric0 > > On Mar 13, 2014, at 5:59 PM, Trevor Vaughan <[email protected]> > wrote: > > > Henrik, > > > > All of this looks great to me. However, I was asked by someone recently > if the language had the concept of a private class scope. > > > > We're seeing more patterns in the wild where people are creating classes > that are only meant to be used internally to the class and not exposed to > the rest of the world. > > > > Is there some way that the new scoping system could account for private > classes? > > > > The best we could come up with right now is the idea of having a > 'private' directory just to make it clear that they are not meant for > public consumption but a 'private' keyword would be great so that the > language itself could enforce the restriction. > > > > Thanks, and looking forward to the performance gains (but a bit worried > about my custom types that use cross-resource variables). > > > > Thanks, > > > > Trevor > > > > > > On Thu, Mar 13, 2014 at 8:19 PM, Henrik Lindberg < > [email protected]> wrote: > > Hi, > > we are just started to get more concrete on how to implement things for > 4x and breaking it down into actionable items. If you have looked in Jira, > there are currently 5 big issues in the epic "Biff the Catalog Builder" [1] > - which is the goal (a new, better performing catalog builder (what is > currently known as the "compiler") where we can fix many known issues that > today are just to hard to implement. > > > > This time, I want to talk about the implementation of Scope, which is > > part of "(PUP-1832) Implement the Puppet 4.0 Runtime" [2]. > > > > Currently scope has many responsibilities (too many): > > > > * it is classic computer language scope (what is visible "here") > > * for a class it also represents one aspect of "an instance of a class" > > (the attributes of the class are variables in that scope). > > * Inheritance is achieved by looking up and continuing the search for a > > variable in another "scope". > > > > Coming up with a new implementation is important to make scope perform > well. Thus it is important to know: > > > > - write vs read ratio > > - unqualified vs. qualified lookup (i.e. reading $a:.b::x from within > $a::b vs from other scopes) > > - typical nesting levels of named scopes > > > > We also have to decide if any of the relative name-space functionality > should remain (i.e. reference to x::y is relative to potentially a series of > > other name spaces ("dynamic scoping"), or if it is always a global > reference when it is qualified. > > > > The implementation idea we have in mind is that there is one global > scope where all "qualified variables" are found/can be resolved, and that > all other variables are in local scopes that nest. (Local scopes include > ephemeral scopes for match variables). > > > > Given the numbers from measuring the read ratio, we (sort of already > know, but still need to measure) need a fast route from any scope to the > global - we know that a qualified variable is never resolved by any local > scope so we can go straight to the global scope. (This way > > we do not have to traverse the chain up to the "parent most" scope (the > global one). Local scopes are always local, there is no way to address the > local variables from some other non-nested scope - essentially how the > regular CPU stack works, or how variables in a language like C work). > > > > i.e. we have something like this in Scope > > > > Scope > > attr_reader :global_scope > > attr_reader :parent_scope > > # ... > > end > > > > > > The global scope keeps an index designed to be as fast as possible to > resolve a qualified name to a value. The design of this index depends on > the frequency of different types of lookup. If all qualified lookups are > absolute it would simply be a hash of all absolute names to values (it > really cannot be faster than that). > > > > The logic for lookup then becomes: > > - for un-qualified name, search up the parent chain (this chain does not > reach the global scope), if still unresolved, look in global scope. > > - for qualified name, look up in global scope directly > > > > If we need to also consider relative namespaces (i.e. x::y could mean > z::x::y, or a::b::c::x::y etc. we can then either probe in turn with each > name (which is fine if the number of things to probe is low), or provide a > reverse index where y is first looked up to get the next level of names, > etc. (the idea being that this requires fewer operations to find the right > one). > > > > IF we can completely remove the notion of relative namespacing we gain > performance! > > > > The global scope, in addition to having the qualified names also needs > to separate the names by "kind" since we can have the same name for > different "kinds". We can now keep keep all named things in the global > scope - functions, types, variables, etc. Global scope and loading are > > associated (more about loading in a later post) but it is worth noting > that it may be of value to be able to record that there has already been an > attempt of loading a particular name, and that there was nothing there to > load... > > > > We are going to need the following kinds of scopes: > > > > * Global Scope - holding map from kind, to fully qualified name to value > > * Local Scope - holding variables that shadow parent scope > > * Ephemeral / Match Scope (read only) - when a match is made > > * Class Scope - the topmost scope for a class - needed because variable > lookup in it, and its nested scope needs to lookup all class attributes > (and defined them) via reading/setting variables. > > * Resource Scope - the topmost scope for a user defined resource type - > needed because its parameters are available as read only variables. > > > > The resource scope simply makes the resource parameters available. It > behaves as a local scope otherwise. > > > > The class scope looks up unqualified variables in the class itself, if > not found there, it continues up the parent chain of scopes. If the class > inherits from another, then, the parent scope is one that represents its > super class. > > > > In class scope, setting a variable also means that it is set in global > scope with the fully qualified name. This is where the logic around class > private variables comes in. If it is private, it cannot be accessed from > the outside (i.e. with a qualified name), and thus it > > is only set in the class / class-scope. This in turn brings up the issue > of also supporting "protected" variables; only visible from within the > class logic, and the logic in sub classes, and if subclasses should see > private inherited variables or not (probably not). > > > > The above could probably do with some picture :-) > > > > Now, some questions... > > > > - Are there any particular performance concerns you think we need to be > aware of? > > - Do you have concerns about things we missed? Something important scope > needs to do? > > - Do you have metrics from your environment? (number of lookups of > various kinds, etc) > > - What is your reaction to getting rid of dynamic/relative name > resolution? (Breakage vs. sanity...) > > > > Regards > > - henrik > > > > Links > > --- > > > > [1]: https://tickets.puppetlabs.com/browse/PUP-1789 > > [2]: https://tickets.puppetlabs.com/browse/PUP-1832 > > > > -- > > You received this message because you are subscribed to the Google > Groups "Puppet Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/lfthtr%24vnh%241%40ger.gmane.org > . > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > -- > > Trevor Vaughan > > Vice President, Onyx Point, Inc > > (410) 541-6699 > > [email protected] > > > > -- This account not approved for unencrypted proprietary information -- > > > > -- > > You received this message because you are subscribed to the Google > Groups "Puppet Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXKr6-okx%2B_VoyLEpf_-MkxdVwJ4vw7XJRPzV860V3ueQ%40mail.gmail.com > . > > For more options, visit https://groups.google.com/d/optout. > > Eric Sorenson - [email protected] - freenode #puppet: eric0 > puppet platform // coffee // techno // bicycles > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/81045344-9198-4167-B6D6-F1E712A35886%40puppetlabs.com > . > For more options, visit https://groups.google.com/d/optout. > -- Trevor Vaughan Vice President, Onyx Point, Inc (410) 541-6699 [email protected] -- This account not approved for unencrypted proprietary information -- -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoVXS%2BryGtPNeX7UQRMQM2ipRvJRAjP3Ds30qFL3BsQ%3DHg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
