Summary - In puppet module spec tests, do not use "stubs", which means the method will be called 0 or more times. Instead, use "expects", which means the method must be called exactly 1 time, or some other more fine grained expectation method stubber.

Our puppet unit tests mostly use rspec, but use Mocha http://gofreerange.com/mocha/docs/index.html for object mocking and method stubbing.

I have already run into several cases where the spec test result is misleading because "stubs" was used instead of "expects", and I have spent a lot of time trying to figure out why a method was not called, because adding an expectation like

          provider.class.stubs(:openstack)
.with('endpoint', 'list', '--quiet', '--format', 'csv', []) .returns('"ID","Region","Service Name","Service Type","Enabled","Interface","URL"
"2b38d77363194018b2b9b07d7e6bdc13","RegionOne","keystone","identity",True,"admin","http://127.0.0.1:5002/v3";
"3097d316c19740b7bc866c5cb2d7998b","RegionOne","keystone","identity",True,"internal","http://127.0.0.1:5001/v3";
"3445dddcae1b4357888ee2a606ca1585","RegionOne","keystone","identity",True,"public","http://127.0.0.1:5000/v3";
')

implies that "openstack endpoint list" will be called.

If at all possible, we should use an explicit expectation. For example, in the above case, use "expects" instead:

          provider.class.expects(:openstack)
.with('endpoint', 'list', '--quiet', '--format', 'csv', []) .returns('"ID","Region","Service Name","Service Type","Enabled","Interface","URL"
"2b38d77363194018b2b9b07d7e6bdc13","RegionOne","keystone","identity",True,"admin","http://127.0.0.1:5002/v3";
"3097d316c19740b7bc866c5cb2d7998b","RegionOne","keystone","identity",True,"internal","http://127.0.0.1:5001/v3";
"3445dddcae1b4357888ee2a606ca1585","RegionOne","keystone","identity",True,"public","http://127.0.0.1:5000/v3";
')

This means that "openstack endpoint list" must be called once, and only once. For odd cases where you want a method to be called some certain number of times, or to return different values each time it is called, the Expectation class http://gofreerange.com/mocha/docs/Mocha/Expectation.html should be used to modify the initial expectation.

Unfortunately, I don't think we can just do a blanket "s/stubs/expects/g" in *_spec.rb, without incurring a lot of test failures. So perhaps we don't have to do this right away, but I think future code reviews should -1 any spec file that uses "stubs" without a strong justification.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to