> On 21 Nov 2014, at 17:15, Aleksandr Didenko <adide...@mirantis.com> wrote:
>
> Hi,
>
> following our docs/workflow plus writing rspec tests for every new option we
> add/modify in our manifests could help with regressions. For example:
> • we add new keystone config option in openstack::keystone class -
> keystone_config {'cache/backend': value => 'keystone.cache.memcache_pool';}
> • we create new test for openstack::keystone class, something like this:
> • should
> contain_keystone_config("cache/backend").with_value('keystone.cache.memcache_pool')
> So with such test, if for some reason we lose
> keystone_config("cache/backend") option, 'rake spec' would alert us about it
> right away and we'll get "-1" from CI. Of course we should also implement
> 'rake spec' CI gate for this.
>
> But from the other hand, if someone changes option in manifests and updates
> rspec tests accordingly, then such commit will pass 'rake spec' test and we
> can still lose some specific option.
>
> > We should speed up development of some modular testing framework that will
> > check that corresponding change affects only particular pieces.
>
> Such test would not catch this particular regressions with "keystone_config
> {'cache/backend': value => 'keystone.cache.memcache_pool';}", because even
> with regression (i.e. dogpile backend) keystone was working OK. It has passed
> several BVTs and custom system tests, because 'dogpile' cache backend was
> working just fine while all memcached servers are up and running. So it looks
> like we need some kind of tests that will ensure that particular config
> options (or particular puppet resources) have some particular values (like
> "backend = keystone.cache.memcache_pool" in [cache] block of keystone.conf).
>
> So I would go with rspec testing for specific resources but I would write
> them in 'openstack' module. Those tests should check that needed
> (nova/cinder/keystone/glance)_config resources have needed values in the
> puppet catalog. Since we're not going to sync 'openstack' module with the
> upstream, such tests will remain intact until we change them, and they won't
> be affected by other modules sync/merge (keystone, cinder, nova, etc).
I totally agree, but we need to remember to introduce tests in separate
commits, otherwise loosing commit ID we would also lose tests ;)
Also I’m just wondering how do we keep upstream modules in our repo? They are
not submodules, so how is it organized?
Regards,
--
Tomasz 'Zen' Napierala
Sr. OpenStack Engineer
tnapier...@mirantis.com
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
