On Thu, Jul 24, 2014 at 5:32 PM, Andy Parker <a...@puppetlabs.com> wrote:
> Howdy, > > Henrik, David, Erik, John, and others have been having some pretty epic > conversations around resource expressions, precedence, order of evaluation, > and several other topics. What kicked all of that off was us looking for > some feedback on decisions we were making for the Puppet 4 language about > how resource overrides, defaults, and so on actually work (or don't in some > cases). I think we've finally reached some decisions! > > Henrik took all of the ideas and started trying to work out what we could > do and what we couldn't. Those are in a writeup at > https://docs.google.com/a/puppetlabs.com/document/d/1mlwyaEeZqCfbF2oI1F-95cochxfe9gubjfc_BXjkOjA/edit# > > > Lots of information in there, so here is the summary. > > The principles behind the decisions: > 1. Only make changes that have a high likelihood of *not* needing to be > backed out later. > 2. Strict or explicit is better than lax or implicit. It uncovers issues > and keeps you from lying to yourself. > 3. Puppet 3 has already stacked up a lot of changes. Do not break > manifests unless we really have to. > 4. Let's not break manifests and then have to break them in almost the > same way once we start working on a new catalog system. > > There are three kinds of resource expression that we have to deal with: > > 1. Resource instantiation > 2. Resource defaults > 3. Resource overrides > > Looking forward, I think it is highly likely that the catalog system that > we'll be working on during puppet 4 will be some sort of production (rules) > system. In that kind of a world, resource instantiation likely remains as > is, but defaults and overrides will end up having to change quite a bit, if > not in syntax, at least in semantics. > > Replying to myself...tsk tsk. Oh well. This is to provide an update on what's going on. Henrik put together a bunch of changes to the implementation to carry out these decisions. I worked on the specification to get it to align with these decisions The change to the specification happened in this PR https://github.com/puppetlabs/puppet-specifications/pull/14 We've already identified one problem in the specification, it doesn't allow nested arrays of strings in the title. I have a note to fix that. The implementation changes are currently in this PR https://github.com/puppetlabs/puppet/pull/2914 I'm working through adding tests based on the specification and to explore what kinds of changes to the current semantics we have got. Joshua and I have been working on putting together some tests to not only test the implementation against the specification, but also to see how different it really ends up being from the old implementation. The tests check several different scenarios, but basically, the first item is used as the title of a resource, the second is whether we expect to create a resource or get an error. The third element is either the regexp for the error message or the titles of the resources that actually get created. This is for the current system: ["thing" ,"resource", ["thing"]], ["[thing]" ,"resource", ["thing"]], ["[[nested, array]]" ,"resource", ["nested", "array"]], ["1" ,"resource", ["1"]], # deprecate? ["3.0" ,"resource", ["3.0"]], # deprecate? ["[1]" ,"resource", ["1"]], # deprecate? ["[3.0]" ,"resource", ["3.0"]], # deprecate? #["true" ,"resource", ["true"]], literal true doesn't parse. true as a variable parses and creates a resource #["false" ,"resource", ["false"]], similar problems to [false] below. literal false doesn't parse ["[true]" ,"resource", ["true"]], # this makes no sense given the next case ["[false]" ,"error", /No title provided and :notify is not a valid resource reference/], # *sigh* #["undef" ,"resource", ["undef"]], # works for variable value, not for literal. deprecate? ["[undef]" ,"resource", ["undef"]], # deprecate? #["{nested => hash}" ,"resource", ["{nested => hash}"]], doesn't work for a literal hash (syntax error). In a variable it is created, but isn't possible to reference the resource. deprecate? #["[{nested => hash}]" ,"resource", ["{nested => hash}"]], works as both literal and variable, but isn't possible to reference the resource. deprecate? ["/regexp/" ,"error", /Syntax error/], ["[/regexp/]" ,"error", /Syntax error/], ["default" ,"error", /Syntax error/], ["[default]" ,"error", /Syntax error/], The current system has a lot of odd outcomes in it. Here are the tests for the future parser (as changed by the PR) ["thing" ,"resource", ["thing"]], ["[thing]" ,"resource", ["thing"]], ["[[nested, array]]" ,"resource", ["nested", "array"]], ["1" ,"error", /Illegal title type.*Expected String, got Integer/], ["3.0" ,"error", /Illegal title type.*Expected String, got Float/], ["[1]" ,"error", /Illegal title type.*Expected String, got Integer/], ["[3.0]" ,"error", /Illegal title type.*Expected String, got Float/], ["true" ,"error", /Illegal title type.*Expected String, got Boolean/], ["false" ,"error", /Illegal title type.*Expected String, got Boolean/], ["[true]" ,"error", /Illegal title type.*Expected String, got Boolean/], ["[false]" ,"error", /Illegal title type.*Expected String, got Boolean/], ["undef" ,"error", /Missing title.*undef/], ["[undef]" ,"error", /Missing title.*undef/], ["{nested => hash}" ,"error", /Illegal title type.*Expected String, got Hash/], ["[{nested => hash}]" ,"error", /Illegal title type.*Expected String, got Hash/], ["/regexp/" ,"error", /Illegal title type.*Expected String, got Regexp/], ["[/regexp/]" ,"error", /Illegal title type.*Expected String, got Regexp/], ["default" ,"resource", []], # nothing created because this is just a local default ["[default]" ,"resource", []], I'm a little worried that these changes might be too extreme. Thoughts? The specification still needs a lot of work. It is really hard to piece together right now, but we are working on changing how it is laid out. > DECISION ONE > > Resource defaults and Resource overrides will be left untouched. > > Decision one follows from principles 3 and 4. In the discussions it became > clear that changing when defaults or overrides are applied, the scope of > defaults, or anything else about them was going to cause a lot of problems. > Puppet's master branch changed resource defaults to follow the same scoping > rules as variables. That change will be reverted. > > DECISION TWO > > Resource instantiations are value producing expressions > > The expression based grammar that puppet 4 will be based on changed almost > everything into a general expression, which allowed a lot of composition > that wasn't possible before. This didn't change resource expressions. > Resource expressions could not be assigned ($a = notify {hi:}). That is > being changed. This removes several odd corners in the grammar and makes it > all more consistent. It is also highly unlikely that it would be removed > later (principle 1). The value of a resource expression is a reference to > the created resource, or an array of references if there is more than one. > > QUESTION: should the value always be an array of references? That would > make it much more predictable. > > DECISION THREE > > Resource instantiation expressions will not be allowed in dangerous > locations > > Once resource expressions can be placed anywhere there are a few places > where they would actually just do more harm than good (principle 2). One > example is as a parameter default (define a($b = notify {hi:}) {}). > > DECISION FOUR > > The LHS of a resource *instantiation* expression can be an expression > > What?!? This means you can do: > > $a = notify > $a { hi: } > > Once again, in clearing up odd cases in the grammar this is opened up to > us. This is a very powerful feature to have available. Since this is very > useful and fits well into the grammar I don't see this being a temporary > thing that would then have to go away later (principle 1). > > DECISION FIVE (how many of these are there?) > > A resource with a title of default provides the default parameter values > for other resources in the same instantiation expression. > > Thanks to David Schmitt for this idea! > > Since we aren't going to change the behavior of resource default > expressions (Notify { ... }) it seems like there needs to be something done > to provide a better, safer way of specifying defaults. This will allow: > > notify { > default: message => hi; > bye: } > > The result will be a resource of type Notify with title bye and message > hi. It is highly unlikely that this will go away (principle 1) as it is > syntactic sugar for specifying the parameters for every resource. > > DECISION SIX > > There will be a splat operator for resource instantiation expressions > > To make the default resources (decision five) really useful there needs to > be a way to reuse the same values across multiple defaults. The current, > dangerous, semantics of resource default expressions skirt this issue by > making defaults part of the (dynamic) evaluation scope. In order to make > the default resources nearly as useful but much safer, we need to add a way > to allow reuse of defaults across multiple resource instantiation > expressions explicitly (principle 2). > > $owner_mode = { owner => andy, mode => '777' } # gotta make it secure > file { default: *=> $owner_mode; > '/home/andy/.bashrc': ; > '/home/andy/.ssh/id_rsa': ; > } > > file { '/etc/passwd': *=> $owner_mode } > > As a side note, do you see what can now be done? > > $a = notify > $b = hi > $c = { message => bye } > $a { $b: *=> $c } > > DECISION SEVEN > > undef is not allowed as a title > > Not much to say here. notify { undef: } fails (or anything that evaluates > to undef) > > DECISION EIGHT > > An array as a title expands to individual resource instantiation > expressions with titles of the elements of the array. > > This isn't really too far off from the current semantics, no real change > here. It is only to call out that we are formalizing that as the semantics. > An empty array ends up being a noop (no resources instantiated). An array > that contains undef will produce an error (see decision seven). The value > default can be an element of the array and will produce the default section > for the resources being instantiated (as pointless as that seems since they > will all have the same body). > > DECISION NINE > > Decisions two through eight do not apply to resource default or resource > override expressions. > > Just to make it clear that decision one still holds. > > CONCLUSION > > I think that covers it all. This will be reflected by a revert to some > code, modifying the grammar, adding some new evaluation capabilities, > including tests, and updating the specification. All of this is falling > under PUP-501, PUP-511, and PUP-2898 in some way shape or form. > > This email was to record the decisions; make them public; double check > that Henrik, Joshua and I all had the same understanding of them; and give > another chance to everyone to weigh in. > > I did have one question that I uncovered as I was writing this up. Some > feedback on that would be great as well. > > -- > Andrew Parker > a...@puppetlabs.com > Freenode: zaphod42 > Twitter: @aparker42 > Software Developer > > *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September > 22-24 in San Francisco* > *Register by May 30th to take advantage of the Early Adopter discount > <http://links.puppetlabs.com/puppetconf-early-adopter> **—**save $349!* > -- Andrew Parker a...@puppetlabs.com Freenode: zaphod42 Twitter: @aparker42 Software Developer *Join us at PuppetConf 2014 <http://www.puppetconf.com/>, September 22-24 in San Francisco* *Register by May 30th to take advantage of the Early Adopter discount <http://links.puppetlabs.com/puppetconf-early-adopter> **—**save $349!* -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CANhgQXvuJtMVyWPVRdGXDMqWWk0NU-r9C%2BZNh6AJvc4_2R-Ayg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
