On 2014-08-07 18:09, John Bollinger wrote:
On Sunday, July 6, 2014 8:26:28 PM UTC-5, henrik lindberg wrote:
Thank you everyone that commented on the first RFC for Resource
Defaults
(and Collection) - some very good ideas came up. I am creating a new
thread because I wanted to take the Collection part in a separate
discussion (honestly I would like to burn the entire implementation -
but that is separate topic), and I want to present some new ideas about
defaults triggered by David Schmitts idea about defaults applicable to
only a resource body.
Let me back up and first describe a problem we have in the grammar.
The egrammar (as well as the current grammar in 3x) tries to be helpful
by recognizing certain combinations as illegal (an override that is
virtual or exported), a resource default or override cannot specify a
title. This unfortunately means that the grammar has to recognize
sequences of tokens that makes this grammar ambiguous and it has to be
solved via operator precedence tricks (that makes the problem show
up as
other corner cases of the grammar).
I agree that that is a problem. Are you saying that the 3.x grammar
also has such ambiguities, or just that it also rejects some token
sequences (as any grammar must do)?
3x grammar does not have this specific problem; instead it has a dozen
other problems because people worked around ambiguities by creating
sub-trees of the expression tree for different kinds of expressions. It
is just horrible.
(This is a classic mistake of trying
to implement too much semantics in the grammar / parser).
So...
What if we simply made the three resource expressions (create resource,
set resource defaults, an resource override) have exactly the same
grammar, and push of validation to the static validation that takes
place and the runtime.
Basically the grammar would be (I am cheating just a little here to
avoid irrelevant details):
ResourceExpression
: At? left_expr = Expression '{' ResourceBodies ';'? '}'
;
ResourceBodies
: ResourceBody (';' ResourceBody)*
;
ResourceBody
: title = Expression ':' AttributeOperations ','?
;
AttributeOperations
: AttributeOperation (',' AttributeOperation)*
;
AttributeOperation
: AttributeName ('=>' | '+>') Expression
AttributeName
: NAME | KeywordsAcceptableAsAttributeName
;
# Details here irrelevant, meaning is: virtual or exported
resource
# AT is the '@' token
At
: AT
| AT AT
| ATAT
;
So, how are the three kinds expressed? Notice that a title is required
for each ResourceBody. So we are basically going to handle different
combinations of left_expr and titles. We simply evaluate the left_expr
at runtime and treat the combinations of the *resulting* type and type
of title:
If left (result) is a String, it must be the name of a Resource Type.
The title works as it does now in 3x. In addition the title (of one
resource body) may be Default, which sets the defaults for this
resource
expression only (all bodies in the same resource expression) - i.e.
"Schmitt style".
notify { hi: message => 'hello there' }
file {
default:
mode => '0331',
owner => 'weird';
'/tmp:foo':
content => 'content 1'
}
If the left is the keyword 'class', it works the same way as for when
the left is a String but defaults can now only set defaults for meta
parameters since there are no other attributes that work safely across
all classes. (Yes you can do this today with Class { } in 3x)
class { 'a::b': param => value }
class { default: audit => true }
If the left is Type[CatalogEntry] (i.e. a resource or class reference),
the meaning changes to either a default or an override.
I'm not fully up to speed on the type system, but surely if the left
side evaluates to a resource or class *reference* then the statement can
only be an override. Right? For it to express resource defaults, the
left side must evaluate to a type -- either Class or a resource type.
The problem is that this happens before anything is evaluated, the
grammar is making decisions on tokens, not what the result evaluates to.
What I want to do is to push the validation off to the runtime (where
it is known what the expressions evaluate to). Doing so requires that
the resource expression is unambiguous - now I have to treat the
following { } body as a kind of operator having a particular precedence.
To be able to do this, I would like all resource expressions to have a
title, as that makes the expr { expr : part unique
and cannot be confused with an expression followed by a hash.
A resource reference is essentially expr [ expr ], it now considers any
such expression as one that should result in a type reference, but it
can not know for sure statically - it must evaluate the expression to
see what it gets (yet, the grammar contains checks that cannot answer
this correctly). (it is the remaining messy part of the egrammar).
An example is
probably easier that lots of words to describe this:
Define the defaults for all instances of the resource type Notify:
Notify { default:
message => 'the default message'
}
Override the message for the notify resource hi.
Notify { hi:
message => 'ciao'
}
If the left type is instance specific it is now an error (since a title
followed by ':' is required to create a less ambiguous grammar) - if we
allowed this, a user could write:
Notify[hi] { bye: message => 'adieu' }
we allow a very odd statement (and the reason why resource overrides
currently does not allow a title in its "resource body"). So, an error
for this case.
I tried hard to not like this, but then I recognized how congruent it is
-- or could be -- with one of my pet ideas: resource constraints. Felix
implemented part of that idea, but as I understand it, he handled only
the diagnostic part (recognizing whether constraints were satisfied) and
not the prescriptive part (updating the catalog, if possible, to ensure
that declared constraints /are/ satisfied).
I don't much like the idea of "overriding" resource declarations as
such, but it's not such a big leap to reposition that as simply
declaring additional requirements (i.e. constraints) on resources that
(may) be declared elsewhere. We get the rest of the way to a limited
form of prescriptive constraints by collapsing this variety of resource
overrides with resource declarations. In other words, consider these
two expressions:
file { '/tmp/hello': ensure => 'file' }
File { '/tmp/hello': ensure => 'file' }
What if they meant exactly the same thing? Specifically, what if they
meant "there must be a resource in the catalog of type File and name
'/tmp/hello', having the value 'file' for its 'ensure' property"?
Either one would then insert such a resource into the catalog if
necessary, and afterward attempt to set the (then) existing resource's
'ensure' parameter.
I say "attempt to set" the existing resource's parameter, because I
think we still need to forbid modifying a value that was set (to a
different value) somewhere else. And that's more complicated than just
checking for undef, because it may be that undef is itself the
intentionally assigned parameter value, so that it would be an error to
change it. It also might need to be more permissive for subclasses and
collectors.
Note that collection currently can modify any already set value on all
kinds of resources (regular, virtual and exported) at any point
throughout the evaluation. How is it that these "rules" are given such
mighty powers when a rule such as "File['tmp/foo'] { owner => x }" is
not allowed to override a set mode of the same file? (I understand the
need to guard against typos and unintentional changes). Basically I see
File[id] { x => y } as the same expression as File <| title == id |> { x
=> y }.
We could go farther with that. If it seems wasteful to have two
different forms of the same statement, then we could apply additional
semantics to one or the other. For example, perhaps the lowercase form
could implicitly declare all unmentioned parameters as undef, with the
effect that they could not be overridden to anything different.. I
don't know whether that would be useful, or whether there is some other
behavior that would be more useful. Perhaps it would be best to just
let the two forms be equivalent, or maybe even to deprecate one.
Anyway, a few corner cases exist. A +> is only allowed if it is a
default or override expression.
+> is a bit of a dark horse in the regime of overrides, as modifying a
previously-declared parameter value is inherent in its design. On the
other hand, it is currently useful for overrides only in subclasses and
collectors, where modifying a declared value is considered acceptable.
I don't think it's a major issue that evaluating a statement containing
a plussignment may yield an error, as long as the meaning of the
statement is not itself in question.
I like the direction this is going! You are absolutely right that
basically all these expressions (resource attribute settings,
"overrides", defaults) basically define a set of rules that together
should define the resulting resources and the values of their attributes
(call them constraints or rules).
This would mean that +> would be perfectly valid even in definitions
that does not modify something inherited, instead it basically says "use
both what someone else said, and what I said" (if the parameter can hold
multiple values).
Do you like these ideas? Is it worth trying to make this work to try it
in practice? (it will take 1-2 days for the implementation, and a bit
more to fix all breaking tests).
As I said before, I tried to dislike them, but I like them despite
myself. Especially if you're willing to accept my extension.
Yes, I think your extension was that there is no difference between a
LHS that is a type (e.g. Notify), and a lower case name of a type (e.g.
notify). I agree completely.
My own main issue with the idea is that it makes code backwards
incompatible; you cannot write a manifest that uses defaults and
overrides in a way that works both in 3.x and 4.x. (Or, I have not
figured out a way yet at least).
There is one way that works across all versions; calling functions -
e.g. something like set_defaults(Type[CatalogEntry] type, Hash[String,
Any] values), and override(CatalogEntry resource_ref, Hash[String, Any]
values). This obviously works because it does not rely on a particular
syntax.
That's the main reason I tried to dislike the idea, and it's not
negligible. It seems like the best alternative for that would be to
select an alternative syntax for type expressions that cannot be
confused with resource references. Doing so would remove some of the
reason for the above idea, but would not render it moot.
Even with functions added, it still means a change, and since it from
the point of view of a user does not provide immediate value (just
translation (a.k.a pain)) the entire idea may be a hard sell. I do think
we can put this and other ideas regarding queries (collection) together
in such a way that it justifies the required changes. Not just sure yet
on the entire package...
Regards
- henrik
--
Visit my Blog "Puppet on the Edge"
http://puppet-on-the-edge.blogspot.se/
--
You received this message because you are subscribed to the Google Groups "Puppet
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-dev/lphjit%24fvn%241%40ger.gmane.org.
For more options, visit https://groups.google.com/d/optout.
