On Thu, Mar 13, 2014 at 5:59 PM, Trevor Vaughan <tvaug...@onyxpoint.com>wrote:
> Henrik, > > All of this looks great to me. However, I was asked by someone recently if > the language had the concept of a private class scope. > > Ask and ye shall find: https://tickets.puppetlabs.com/browse/PUP-523 We are planning on reserving the private keyword in puppet 4 and implementing private variables and classes in the puppet 4 series. > We're seeing more patterns in the wild where people are creating classes > that are only meant to be used internally to the class and not exposed to > the rest of the world. > > Is there some way that the new scoping system could account for private > classes? > > The best we could come up with right now is the idea of having a 'private' > directory just to make it clear that they are not meant for public > consumption but a 'private' keyword would be great so that the language > itself could enforce the restriction. > > Thanks, and looking forward to the performance gains (but a bit worried > about my custom types that use cross-resource variables). > > Thanks, > > Trevor > > > On Thu, Mar 13, 2014 at 8:19 PM, Henrik Lindberg < > henrik.lindb...@cloudsmith.com> wrote: > >> Hi, >> we are just started to get more concrete on how to implement things for >> 4x and breaking it down into actionable items. If you have looked in Jira, >> there are currently 5 big issues in the epic "Biff the Catalog Builder" [1] >> - which is the goal (a new, better performing catalog builder (what is >> currently known as the "compiler") where we can fix many known issues that >> today are just to hard to implement. >> >> This time, I want to talk about the implementation of Scope, which is >> part of "(PUP-1832) Implement the Puppet 4.0 Runtime" [2]. >> >> Currently scope has many responsibilities (too many): >> >> * it is classic computer language scope (what is visible "here") >> * for a class it also represents one aspect of "an instance of a class" >> (the attributes of the class are variables in that scope). >> * Inheritance is achieved by looking up and continuing the search for a >> variable in another "scope". >> >> Coming up with a new implementation is important to make scope perform >> well. Thus it is important to know: >> >> - write vs read ratio >> - unqualified vs. qualified lookup (i.e. reading $a:.b::x from within >> $a::b vs from other scopes) >> - typical nesting levels of named scopes >> >> We also have to decide if any of the relative name-space functionality >> should remain (i.e. reference to x::y is relative to potentially a series of >> other name spaces ("dynamic scoping"), or if it is always a global >> reference when it is qualified. >> >> The implementation idea we have in mind is that there is one global scope >> where all "qualified variables" are found/can be resolved, and that all >> other variables are in local scopes that nest. (Local scopes include >> ephemeral scopes for match variables). >> >> Given the numbers from measuring the read ratio, we (sort of already >> know, but still need to measure) need a fast route from any scope to the >> global - we know that a qualified variable is never resolved by any local >> scope so we can go straight to the global scope. (This way >> we do not have to traverse the chain up to the "parent most" scope (the >> global one). Local scopes are always local, there is no way to address the >> local variables from some other non-nested scope - essentially how the >> regular CPU stack works, or how variables in a language like C work). >> >> i.e. we have something like this in Scope >> >> Scope >> attr_reader :global_scope >> attr_reader :parent_scope >> # ... >> end >> >> >> The global scope keeps an index designed to be as fast as possible to >> resolve a qualified name to a value. The design of this index depends on >> the frequency of different types of lookup. If all qualified lookups are >> absolute it would simply be a hash of all absolute names to values (it >> really cannot be faster than that). >> >> The logic for lookup then becomes: >> - for un-qualified name, search up the parent chain (this chain does not >> reach the global scope), if still unresolved, look in global scope. >> - for qualified name, look up in global scope directly >> >> If we need to also consider relative namespaces (i.e. x::y could mean >> z::x::y, or a::b::c::x::y etc. we can then either probe in turn with each >> name (which is fine if the number of things to probe is low), or provide a >> reverse index where y is first looked up to get the next level of names, >> etc. (the idea being that this requires fewer operations to find the right >> one). >> >> IF we can completely remove the notion of relative namespacing we gain >> performance! >> >> The global scope, in addition to having the qualified names also needs to >> separate the names by "kind" since we can have the same name for different >> "kinds". We can now keep keep all named things in the global scope - >> functions, types, variables, etc. Global scope and loading are >> associated (more about loading in a later post) but it is worth noting >> that it may be of value to be able to record that there has already been an >> attempt of loading a particular name, and that there was nothing there to >> load... >> >> We are going to need the following kinds of scopes: >> >> * Global Scope - holding map from kind, to fully qualified name to value >> * Local Scope - holding variables that shadow parent scope >> * Ephemeral / Match Scope (read only) - when a match is made >> * Class Scope - the topmost scope for a class - needed because variable >> lookup in it, and its nested scope needs to lookup all class attributes >> (and defined them) via reading/setting variables. >> * Resource Scope - the topmost scope for a user defined resource type - >> needed because its parameters are available as read only variables. >> >> The resource scope simply makes the resource parameters available. It >> behaves as a local scope otherwise. >> >> The class scope looks up unqualified variables in the class itself, if >> not found there, it continues up the parent chain of scopes. If the class >> inherits from another, then, the parent scope is one that represents its >> super class. >> >> In class scope, setting a variable also means that it is set in global >> scope with the fully qualified name. This is where the logic around class >> private variables comes in. If it is private, it cannot be accessed from >> the outside (i.e. with a qualified name), and thus it >> is only set in the class / class-scope. This in turn brings up the issue >> of also supporting "protected" variables; only visible from within the >> class logic, and the logic in sub classes, and if subclasses should see >> private inherited variables or not (probably not). >> >> The above could probably do with some picture :-) >> >> Now, some questions... >> >> - Are there any particular performance concerns you think we need to be >> aware of? >> - Do you have concerns about things we missed? Something important scope >> needs to do? >> - Do you have metrics from your environment? (number of lookups of >> various kinds, etc) >> - What is your reaction to getting rid of dynamic/relative name >> resolution? (Breakage vs. sanity...) >> >> Regards >> - henrik >> >> Links >> --- >> >> [1]: https://tickets.puppetlabs.com/browse/PUP-1789 >> [2]: https://tickets.puppetlabs.com/browse/PUP-1832 >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-dev+unsubscr...@googlegroups.com. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/puppet-dev/lfthtr%24vnh%241%40ger.gmane.org. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > tvaug...@onyxpoint.com > > -- This account not approved for unencrypted proprietary information -- > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXKr6-okx%2B_VoyLEpf_-MkxdVwJ4vw7XJRPzV860V3ueQ%40mail.gmail.com<https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoXKr6-okx%2B_VoyLEpf_-MkxdVwJ4vw7XJRPzV860V3ueQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Andrew Parker a...@puppetlabs.com Freenode: zaphod42 Twitter: @aparker42 Software Developer *Join us at PuppetConf 2014, September 23-24 in San Francisco - * http://bit.ly/pupconf14 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANhgQXt3N1gFS%2BbQ2c2B3Ai%2BpSTBaO6k6HDdk0jNHkGOJ66NiQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
