Re: [openstack-dev] [tripleo] validating overcloud config changes on a redeploy

[Alex Schultz]

On Fri, Apr 27, 2018 at 9:49 AM, Ade Lee  wrote:
> Hi,
>
> Recently I starting looking at how we implement password changes in an
> existing deployment, and found that there were issues.  This made me
> wonder whether we needed a test job to confirm that password changes
> (and other config changes) are in fact executed properly.
>
> As far as I understand it, the way to do password changes is to -
> 1) Create a yaml file containing the parameters to be changed and
>their new values
> 2) call openstack overcloud deploy and append -e new_params.yaml
>
> Note that the above steps can really describe the testing of setting
> any config changes (not just passwords).
>
> Of course, if we do change passwords, we'll want to validate that the
> config files have changed, the keystone/dbusers have been modified, the
> mistral plan has been updated, services are still running etc.
>
> After talking with many folks, it seems there is no clear consensus
> where code to do the above tasks should live.  Should it be in tripleo-
> upgrades, or in tripleo-validations or in a separate repo?
>
> Is there anyone already doing something similar?
>
> If we end up creating a role to do this, ideally it should be
> deployment tool agnostic - usable by both infrared or quickstart or
> others.
>
> Whats the best way to do this?
>

So in my mind, this falls under a testing framework validation where
we want to perform a set of $deployment_actions and ensure that
$specific_things have been completed. For the most part we don't have
anything like that in the upstream tripleo project for actions that
aren't covered by tempest tests.  Even tempest tests are only ensuring
that we configured the services so they work but not a state
transition from A to B.  Honestly I don't think tripleo-upgrades or
tripleo-validations is the appropriate place for this type of check.
tripleo-validations might make sense if we expected an end user to do
this after performing a specific action but I don't think there's
enough of these types of actions for that to be warrented.  It's more
likely that we would want to come up with a deployment test suite that
could be run offline where a scenario like 'change all the passwords'
would be executed and verified that it functioned as expected (all the
passwords were changed).  Something like this might work in a periodic
upstream job but it's more like a full validation suite that would
most likely need to be run offline.

Thanks,
-Alex

> Thanks,
> Ade
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] validating overcloud config changes on a redeploy

[Marius Cornea]

Hi Ade,

To the best of my knowledge the closest place where we do similar
sequence of actions (post deployment build and append environment
files to the deploy command and re-run overcloud deploy on top of an
already deployed overcloud) is tripleo-upgrade. As we discussed on IRC
I was reluctant to adding these kind of tests to tripleo-upgrade since
it was initially created to cover only the minor update and major
upgrades use cases. Nevertheless, thinking more about your use case I
realized that configuration changes tests could fit quite well in
tripleo-upgrade for several reasons:

 - we already have a mechanism[1] in place for attaching extra
environment files to the deploy command
 - we already have tests that can be run during the stack update which
applies the config changes; this could be useful to validate that
configuration changes do not break the data plane(e.g to validate that
a neutron config change doesn't not leave instances without networking
during the stack update)
 - we can easily segregate the config changes plays into their own
directory as we do with update/upgrade[2] and add the reusable ones in
the common directory
 - upgrades might benefit from the config changes tests by running
them in a pre/post minor update/major upgrade step and catch potential
parameters changes between releases

I'd like to hear what others think about this and see if there could
be a better place where to host these kind of tests but personally I'm
ok with adding them to tripleo-upgrade.

Best regards,
Marius

[1] 
http://git.openstack.org/cgit/openstack/tripleo-upgrade/tree/tasks/upgrade/step_upgrade.yml
[2] http://git.openstack.org/cgit/openstack/tripleo-upgrade/tree/tasks

On Fri, Apr 27, 2018 at 11:49 AM, Ade Lee  wrote:
> Hi,
>
> Recently I starting looking at how we implement password changes in an
> existing deployment, and found that there were issues.  This made me
> wonder whether we needed a test job to confirm that password changes
> (and other config changes) are in fact executed properly.
>
> As far as I understand it, the way to do password changes is to -
> 1) Create a yaml file containing the parameters to be changed and
>their new values
> 2) call openstack overcloud deploy and append -e new_params.yaml
>
> Note that the above steps can really describe the testing of setting
> any config changes (not just passwords).
>
> Of course, if we do change passwords, we'll want to validate that the
> config files have changed, the keystone/dbusers have been modified, the
> mistral plan has been updated, services are still running etc.
>
> After talking with many folks, it seems there is no clear consensus
> where code to do the above tasks should live.  Should it be in tripleo-
> upgrades, or in tripleo-validations or in a separate repo?
>
> Is there anyone already doing something similar?
>
> If we end up creating a role to do this, ideally it should be
> deployment tool agnostic - usable by both infrared or quickstart or
> others.
>
> Whats the best way to do this?
>
> Thanks,
> Ade
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] validating overcloud config changes on a redeploy

[Ade Lee]

Hi,

Recently I starting looking at how we implement password changes in an
existing deployment, and found that there were issues.  This made me
wonder whether we needed a test job to confirm that password changes
(and other config changes) are in fact executed properly.

As far as I understand it, the way to do password changes is to - 
1) Create a yaml file containing the parameters to be changed and 
   their new values
2) call openstack overcloud deploy and append -e new_params.yaml

Note that the above steps can really describe the testing of setting
any config changes (not just passwords).

Of course, if we do change passwords, we'll want to validate that the
config files have changed, the keystone/dbusers have been modified, the
mistral plan has been updated, services are still running etc.

After talking with many folks, it seems there is no clear consensus
where code to do the above tasks should live.  Should it be in tripleo-
upgrades, or in tripleo-validations or in a separate repo?

Is there anyone already doing something similar?

If we end up creating a role to do this, ideally it should be
deployment tool agnostic - usable by both infrared or quickstart or
others.

Whats the best way to do this?

Thanks,
Ade

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev