On 12/20/2013 03:43 PM, John Griffith wrote:
Hey Everyone,
So we merged the super simple driver cert test script in to devstack a
while back. For those that aren't familiar you can check it out here
[1]. First iteration of this is simply a do it yourself config and
run that goes through the same volume-tests that every cinder patch
runs through the gate.
There's definitely room for growth here but this seems like a
reasonable first step. The remaining question here is how do we want
to use this? I've made a coupe of suggestions that I'd like to review
and get some feed-back. To be clear this can obviously evolve over
time, but I'd like to start somewhat simple, try it out and build off
of if depending on how things go. So with that here's a couple of
options I've been considering:
1. File a bug in launchpad:
This bug would be for tracking purposes, it would be something like
"no cert results available for driver-X". This would require that the
driver maintainer download/install devstack, configure their driver
and backend and then run the supplied script.
The next question is what to do with the results, there are some options here:
a. Take the resultant tgz file and post it into the bug report as an
attachment. Assuming everything passes the bug can then be marked as
closed/resolved.
b. Create a repo (or even a directory in the Cinder tree) that
includes results files. That way the bug is logged and a gerrit
commit referencing the bug id is submitted and reviewed very similar
to how we handle source changes.
Option 'a' is path of least resistance, however it becomes a very
manual process and it's somewhat ugly. Option 'b' fits more with how
we operate anyway, and provides some automation, and it also leaves a
record of the cert process in the tree that makes visibility and
tracking much easier.
I tend to like option B here. Then as versions of Cinder ship, the
'certification' files can go
along with it.
2. Create a web/wiki page specifically for this information:
This would basicly be a matrix of the drivers, and the current status
of the cert results for the current iteration. It would be something
like a row for every driver in the tree and a column for "last cycle"
and "current cycle". We'd basically set it up so that the
"current-cycle" entries are all listed as "not submitted" after the
milestone is cut. The current entries in that column would roll back
to the "last-cycle" column. Then the driver maintainer could
run/update the matrix at any time during that cycle.
The only thing with this is again it's very manual in terms of
tracking, might be a bit confusing (this may make perfect sense to me,
but seems like jibberish to others :)), and we'd want to have a
repository to store the results files for people to reference.
I think this is a good idea as a reference point for users, but I tend to
like the automated mechanism of Option 1b. I see option 1 and 2 being
parallel streams of information. Option 1 being for the cinder
developer team and option 2 being for end users as a quick check to see
what drivers have passed the certification.
I think the general idea of certifying every driver on every milestone is a
good idea, especially since there are several vendors who are running
against master. This mechanism could give them some assurances that
at least the drivers in Cinder are good to go, or are pending
"certification".
Walt
I'm open to ideas/suggestions here keeping in mind that the initial
purpose is to provide publicly viewable information as to the
integration status of the drivers in the Cinder project. This would
help people building OpenStack clouds to make sure that the backend
devices they may be choosing actually implement all of the base
features and that they actually work. Vendors can of course choose
not to participate, that just tells consumers "beware, vendor-a
doesn't necessarily care all that much, or doesn't have time to test
this".
Anyway, hopefully this makes sense, if more clarification is needed I
can try and clean up my descriptions a bit.
Thanks,
John
[1]:
https://github.com/openstack-dev/devstack/blob/master/driver_certs/cinder_driver_cert.sh
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
.
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
