On 15.05.17 19:01, Zane Bitter wrote:
On 15/05/17 12:10, Steven Hardy wrote:
On Mon, May 15, 2017 at 04:46:28PM +0200, Lance Haig wrote:
Hi Steve,
I am happy to assist in any way to be honest.
It was great to meet you in Boston, and thanks very much for
volunteering to help out.
BTW one issue I'm aware of is that the autoscaling template examples
we have all use OS::Ceilometer::* resources for alarms. We have a
global environment thingy that maps those to OS::Aodh::*, so at least
in theory those templates should continue to work, but there are
actually no examples that I can find of autoscaling templates doing
things the way we want everyone to do them.
I think we can perhaps come up with some standard scenarios that we want
to showcase and then we can work on getting this setup.
I might suggest that you look at the repo that my colleague Florin and I
setup for our library and training material.
https://github.com/heat-extras
In the lib repo we have a test directory that tests each library
template it might be an idea as to how to achieve test coverage of the
different resources.
We currently just run yamllint testing with the script in there but I am
sure we can add other tests as needed.
The backwards compatibility is not always correct as I have seen when
developing our library of templates on Liberty and then trying to
deploy it
on Mitaka for example.
Yeah, I guess it's true that there are sometimes deprecated resource
interfaces that get removed on upgrade to a new OpenStack version,
and that
is independent of the HOT version.
What if instead of a directory per release, we just had a 'deprecated'
directory where we move stuff that is going away (e.g. anything
relying on OS::Glance::Image), and then deleted them when it
disappeared from any supported release (e.g. LBaaSv1 must be close if
it isn't gone already).
I agree in general this would be good. How would we deal with users who
are running older versions of openstack?
Most of the customers I support have Liberty and newer so I would
perhaps like to have these available as tested.
The challenge for us is that the newer the OStack version the more
features are available e.g. conditionals etc..
To support that in a backwards compatible fashion is going to be tough I
think. Unless I am missing something.
As we've proven, maintaining these templates has been a challenge
given the
available resources, so I guess I'm still in favor of not duplicating
a bunch
of templates, e.g perhaps we could focus on a target of CI testing
templates on the current stable release as a first step?
I'd rather do CI against Heat master, I think, but yeah that sounds
like the first step. Note that if we're doing CI on old stuff then
we'd need to do heat-templates stable branches rather than
directory-per-release.
With my suggestion above, we could just not check anything in the
'deprecated' directory maybe?
I agree in part.
If we are using the heat examples to test the functionality of the
master branch then that would be a good idea.
If we want to provide useable templates for users to reference and use
then I would suggest we test against stable.
I am sure we could find a way to do both.
I would suggets that we first get reliable CICD running on the current
templates and fix what we can in there.
Then we can look at what would be a good way forward.
I am just brain dumping so any other ideas would also be good.
As you guys mentioned in our discussions the Networking example I
quoted is
not something you guys can deal with as the source project affects
this.
Unless we can use this exercise to test these and fix them then I am
happier.
My vision would be to have a set of templates and examples that are
tested
regularly against a running OS deployment so that we can make sure the
combinations still run. I am sure we can agree on a way to do this
with CICD
so that we test the fetureset.
Agreed, getting the approach to testing agreed seems like the first
step -
FYI we do already have automated scenario tests in the main heat tree
that
consume templates similar to many of the examples:
https://github.com/openstack/heat/tree/master/heat_integrationtests/scenario
So, in theory, getting a similar test running on heat_templates
should be
fairly simple, but getting all the existing templates working is
likely to
be a bigger challenge.
Even if we just ran the 'template validate' command on them to check
that all of the resource types & properties still exist, that would be
pretty helpful. It'd catch of of the times when we break backwards
compatibility so we can decide to either fix it or deprecate/remove
the obsolete template. (Note that you still need all of the services
installed, or at least endpoints in the catalog, for the validation to
work.)
Actually creating all of the stuff would be nice, but it'll likely be
difficult (just keeping up-to-date OS images to boot from is a giant
pain). And even then that isn't sufficient to test that it actually
_works_. Let's keep that out of scope for now?
I was thinking if it was possible to get a moch API that we can validate
against so that the tests don't have to run against a full OStack
install. I am not sure if that is possible.
It would make it easier to test things locally when developing new
templates.
Regards
Lance
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
