On 11/20/2015 07:05 PM, Ben Nemec wrote:
Thinking about this some more makes me wonder if we need a sample config
generator like oslo.config. It would work off something similar to the
capabilities map, where you would say
SSL:
templates:
-puppet/extraconfig/tls/tls-cert-inject.yaml
output:
-environments/enable-ssl.yaml
And the tool would look at that, read all the params from
tls-cert-inject.yaml and generate the sample env file. We'd have to be
able to do a few new things with the params in order for this to work:
-Need to specify whether a param is intended to be set as a top-level
param, parameter_defaults (which we informally do today with the Can be
overridden by parameter_defaults comment), or internal, to define params
that shouldn't be exposed in the sample config and are only intended as
an interface between templates. There wouldn't be any enforcement of
the internal type, but Python relies on convention for its private
members so there's precedent. :-)
There is new functionality in Heat that will let you pass in a series of
templates and environments and it will return:
- The list of top-level parameters, the same way template-validate
always did
- A list of all nested parameters, keyed by resource.
Take a look at
https://github.com/openstack/heat-specs/blob/master/specs/liberty/nested-validation.rst
for details and an example.
That's not entirely what you're getting at, I realize that. I'm glad to
see you suggest a convention-based approach because I think that's the
only way we're going to be able to convey some of this information.
I think at the same time we add a mechanism to distinguish between
internal and external parameters, we need to add something to indicate
required v. optional.
With a nested stack, anything that's not part of the top-level parameter
contract is defaulted. The problem is that it loses information on what
is a valid default v. what's simply defaulted to pass validation.
I've been noticing this more and more on the vendor integrations. They
have parameters that are required (such as a username) and others that
are less likely to be changed (I can't think of an example, but I think
everyone can see where I'm going with this).
So I think there are two sorts of things (at least, I'm also thinking
off the top of my head) we'd like this tool/sample file to convey:
- Parameters a user would want to change, as compared to those used for
internal data shuffling
- Information on if the user must supply a value, as compared to
parameters with an actual default
All that said, I dig this idea of a tool that would generate a skeleton
environment file.
-There would have to be some way to pick out only certain params from a
template, since I think there are almost certainly features that are
configured using a subset of say puppet/controller.yaml which obviously
can't just take the params from an entire file. Although maybe this is
an indication that we could/should refactor the templates to move some
of these optional params into their own separate files (at this point I
think I should take a moment to mention that this is somewhat of a brain
dump, so I haven't thought through all of the implications yet and I'm
not sure it all makes sense).
The nice thing about generating these programmatically is we would
formalize the interface of the templates somewhat, and it would be
easier to keep sample envs in sync with the actual implementation.
You could go so far as to put CI on top of it like we do with the oslo
config stuff, which would be neat.
You'd never have to worry about someone adding a param to a file but
forgetting to update the env (or at least it would be easy to catch and
fix when they did, just run "tox -e genconfig").
I'm not saying this is a simple or short-term solution, but I'm curious
what people think about setting this as a longer-term goal, because as I
think our discussion in Tokyo exposed, we're probably going to have a
bit of an explosion of sample envs soon and we're going to need some way
to keep them sane.
Some more comments inline.
On 11/19/2015 10:16 AM, Steven Hardy wrote:
On Mon, Nov 16, 2015 at 08:15:48PM +0100, Giulio Fidente wrote:
On 11/16/2015 04:25 PM, Steven Hardy wrote:
Hi all,
I wanted to start some discussion re $subject, because it's been apparrent
that we have a lack of clarity on this issue (and have done ever since we
started using parameter_defaults).
[...]
How do people feel about this example, and others like it, where we're
enabling common, but not mandatory functionality?
At first I was thinking about something as simple as: "don't use top-level
params for resources which the registry doesn't enable by default".
It seems to be somewhat what we tried to do with the existing pluggable
resources.
Also, not to hijack the thread but I wanted to add another question related
to a similar issue:
Is there a reason to prefer use of parameters: instead of
parameter_defaults: in the environment files?
It looks to me that by defaulting to parameter_defaults: users won't need to
update their environment files in case the parameter is moved from top-level
into a specific nested stack so I'm inclined to prefer this. Are there
reasons not to?
The main reason is scope - if you use "parameters", you know the data flow
happens via the parent template (e.g overcloud-without-mergepy) and you
never have to worry about naming collisions outside of that template.
But if you use parameter_defaults, all parameters values defined that way
are effectively global, and you then have to be much more careful that you
never shadow a parameter name and get an unexpected value passed in to it.
Here's another example of why we need to decide this btw:
https://review.openstack.org/#/c/229471/
Here, we have some workers parameters, going only into controller.yaml -
this is fine, but the new options are completely invisible to users who
look at the overcloud_without_mergepy parameters schema as their interface
(in particular I'm thinking of any UI here).
My personal preference is to say:
1. Any templates which are included in the default environment (e.g
overcloud-resource-registry-puppet.yaml), must expose their parameters
via overcloud-without-mergepy.yaml
2. Any templates which are included in the default environment, but via a
"noop" implementation *may* expose their parameters provided they are
common and not implementation/vendor specific.
This seems like a reasonable approach, although that "may" still leaves
a lot of room for bikeshedding. ;-)
It might be good to say that in this case it is "preferred" to use a
top-level param, but if there's a reason not to then it's acceptable to
use parameter_defaults. An example for the SSL case would be the
certificate path - I specifically do not want that visibly exposed to
the user at this point, so I wouldn't want it added to the top-level
template. I consider that an implementation detail where if you know
what you're doing you can override it, but otherwise you shouldn't touch it.
3. Any templates exposing vendor specific interfaces (e.g at least anything
related to the OS::TripleO::*ExtraConfig* interfaces) must not expose any
parameters via the top level template.
How does this sound?
This does mean we suffer some template bloat from (1) and (2), but it makes
the job of any UI or other tool requiring user input much easier, I think?
Steve
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
