On 7/21/2016 2:03 AM, Bhor, Dinesh wrote:
Hi Nova Devs,
Many times, there are a number of data sets that we have to run the same
tests on.
And, to create a different test for each data set values is
time-consuming and inefficient.
Data Driven Testing [1] overcomes this issue. Data-driven testing (DDT)
is taking a test,
parameterizing it and then running that test with varying data. This
allows you to run the
same test case with many varying inputs, therefore increasing coverage
from a single test,
reduces code duplication and can ease up error tracing as well.
DDT is a third party library needs to be installed separately and invoke the
module when writing the tests. At present DDT is used in cinder and rally.
There are several projects using it:
http://codesearch.openstack.org/?q=ddt%3E%3D1.0.1&i=nope&files=&repos=
I first came across it when working a little in manila.
To start with, I have reported this as a bug [2] and added initial patch
[3] for the same,
but couple of reviewers has suggested to discuss about this on ML as
this is not a real bug.
IMO this is not a feature implementation and it’s just a effort for
simplifying our tests,
so a blueprint will be sufficient to track its progress.
So please let me know whether I can file a new blueprint or nova-specs
to proceed with this.
[1] http://ddt.readthedocs.io/en/latest/index.html
[2] https://bugs.launchpad.net/nova/+bug/1604798
[3] https://review.openstack.org/#/c/344820/
Thank you,
Dinesh Bhor
______________________________________________________________________
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
I agree that it's not a bug. I also agree that it helps in some specific
types of tests which are doing some kind of input validation (like the
patch you've proposed) or are simply iterating over some list of values
(status values on a server instance for example).
Using DDT in Nova has come up before and one of the concerns was hiding
details in how the tests are run with a library, and if there would be a
learning curve. Depending on the usage, I personally don't have a
problem with it. When I used it in manila it took a little getting used
to but I was basically just looking at existing tests and figuring out
what they were doing when adding new ones.
I definitely think DDT is easier to use/understand than something like
testscenarios, which we're already using in Nova.
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
