Dmitry, If you chose to use Rally framework for testing there are 3 opportunities:
- Keep Rally plugins (tests) in separated tree - Keep Rally plugins (tests) in your project tree - Keep Rally plugins (tests) in Rally repo Rally plugins can be used for all kinds of testing: (perf, scalability, load...) so you are killing two birds with one stone. P.S. I would imho prefer to keep all high quality plugins inside Rally repo to simplify operators life.. Best regards, Boris Pavlovic On Wed, Jun 10, 2015 at 11:57 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> wrote: > 2015-06-10 16:48 GMT+09:00 Dmitry Tantsur <dtant...@redhat.com>: > > On 06/10/2015 09:40 AM, Ken'ichi Ohmichi wrote: > >> To solve it, we have decided the scope of Tempest as the etherpad > >> mentioned. > >> > >>> Are there any hints now on where we can start with our integration > tests? > >> > >> > >> For the other projects, we are migrating the test framework of Tempest > >> to tempest-lib which is a library. > >> So each project can implement their own tests in each repository by > >> using the test framework of tempest-lib. > > > > > > So in my case we can start with putting test code to ironic-inspector > tree > > using tempest-lib, right? > > Yeah, right. > Neutron is already doing that. > maybe neutron/tests/api/ of Neutron repository will be a hint for it. > > > Will it be possible to run tests on Ironic as well using plugin from > > ironic-inspector? > > Yeah, it will be possible. > but I'm guessing ironic-inspector is optional and Ironic should not > depend on the gate test result of ironic-inspector. > So maybe you just need to run Ironic tests on ironic-inspector gate > tests, right? > > >>> After a quick look at devstack-gate I got an impression that it's > >>> expecting > >>> tests as part of tempest: > >>> > >>> > https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L600 > >>> > >>> Our final goal is to have devstack gate test for Ironic and Inspector > >>> projects working together. > >> > >> > >> We have discussed external interfaces of Tempest on the summit, so > >> that Tempest gathers tests from each project repository and runs them > >> at the same time. > >> There is a qa-spec for https://review.openstack.org/#/c/184992/ > > > > > > Cool, thanks! Does it mean that devstack-gate will also be updated to > allow > > something like DEVSTACK_GATE_TEMPEST_PLUGINS="https://github.com/...";? > > Yeah, will be. > The idea of this external interface is based on DevStack's one. > I think we will be able to use it on the gate like that. > > Thanks > Ken'ichi Ohmichi > > --- > > >>> On 06/10/2015 08:07 AM, Yuiko Takada wrote: > >>>> > >>>> > >>>> Hi, Dmitry, > >>>> > >>>> I guess the whole idea of new release models is NOT to tie > projects > >>>> to each other any more except for The Big Release twice a year :) > >>>> So > >>>> I think no, we don't need to. We still can do it, if we have > >>>> something to release by the time Ironic releases, but I suggest > >>>> deciding it on case-by-case basis. > >>>> > >>>> OK, I see. > >>>> > >>>> One more concern, about Tempest integration test which I will > implement > >>>> in V2.1.0, > >>>> it seems like that we cannot add Ironic-inspector's tests into Tempest > >>>> even if integration tests. > >>>> Please see: > >>>> https://etherpad.openstack.org/p/YVR-QA-in-the-big-tent > >>> > >>> > >>> > >>> Good catch. I guess the answer depends on where Ironic integration > tests > >>> are > >>> going to live - we're going to live with them. Let me retarget this > >>> thread > >>> to a wider audience. > >>> > >>>> > >>>> But I heard from you that Devananda thinks we need this in tempest > >>>> itself. [3] > >>>> Do you know something like current situation? > >>>> > >>>> > >>>> Best Regards, > >>>> Yuiko Takada > >>>> > >>>> 2015-06-09 15:59 GMT+09:00 Dmitry Tantsur <dtant...@redhat.com > >>>> <mailto:dtant...@redhat.com>>: > >>>> > >>>> On 06/09/2015 03:49 AM, Yuiko Takada wrote: > >>>> > >>>> Hi, Dmitry, > >>>> > >>>> Thank you for notifying. > >>>> > >>>> I've updated our summit etherpad [3] with whatever > >>>> priorities > >>>> I > >>>> remembered, please have a look. I've also untargeted a > few > >>>> things in > >>>> launchpad [4] (and will probably untarget more later > on). > >>>> Please > >>>> assign yourself, if you want something done in this > >>>> release > >>>> time frame. > >>>> > >>>> I've assigned one item to myself in [3], and also I added one > >>>> BP > >>>> to [4], > >>>> so please take a look. > >>>> > >>>> https://blueprints.launchpad.net/ironic-inspector/+spec/delete-db-api > >>>> > >>>> > >>>> Looks good, though I don't think it's a big priority for 2.0.0. > >>>> Definitely worth doing for 2.1.0. > >>>> > >>>> Thanks for assigning for tempest work, that's definitely a > priority > >>>> right now. > >>>> > >>>> > >>>> BTW, how do you think about Ironic-inspector's release model? > >>>> You wrote "Version released with Ironic Liberty" as > >>>> Ironic-inspector Version 2.1.0 in etherpad [3], > >>>> but as you know, Ironic's release model has changed to > feature > >>>> releases.(right?) > >>>> Should we make our release model same as Ironic? > >>>> > >>>> > >>>> I guess the whole idea of new release models is NOT to tie > projects > >>>> to each other any more except for The Big Release twice a year :) > >>>> So > >>>> I think no, we don't need to. We still can do it, if we have > >>>> something to release by the time Ironic releases, but I suggest > >>>> deciding it on case-by-case basis. > >>>> > >>>> > >>>> > >>>> Best Regards, > >>>> Yuiko Takada(Inspector team member) > >>>> > >>>> 2015-06-08 20:38 GMT+09:00 Dmitry Tantsur < > dtant...@redhat.com > >>>> <mailto:dtant...@redhat.com> > >>>> <mailto:dtant...@redhat.com <mailto:dtant...@redhat.com>>>: > >>>> > >>>> > >>>> Hello, Inspector team! > >>>> > >>>> The renaming process is going pretty well, the last > thing > >>>> we need to > >>>> do is to get Infra approval and actual rename [1][2]. > >>>> > >>>> I'd like to allow people (e.g. myself) to start > packaging > >>>> inspector > >>>> under it's new name, so I'd like to make 2.0.0 release > as > >>>> soon as > >>>> possible (as opposed to scheduling it to particular > date). > >>>> All > >>>> breaking changes should land by this release - I don't > >>>> expect 3.0.0 > >>>> soon :) > >>>> > >>>> I've updated our summit etherpad [3] with whatever > >>>> priorities > >>>> I > >>>> remembered, please have a look. I've also untargeted a > few > >>>> things in > >>>> launchpad [4] (and will probably untarget more later > on). > >>>> Please > >>>> assign yourself, if you want something done in this > >>>> release > >>>> time frame. > >>>> > >>>> I would like 2.1.0 to be released with Ironic Liberty > and > >>>> be > >>>> properly supported. > >>>> > >>>> Let me know what you think. > >>>> > >>>> Cheers, > >>>> Dmitry > >>>> > >>>> [1] https://review.openstack.org/#/c/188030/ > >>>> [2] https://review.openstack.org/#/c/188798/ > >>>> [3] > >>>> https://etherpad.openstack.org/p/liberty-ironic-discoverd > >>>> [4] > >>>> https://bugs.launchpad.net/ironic-inspector/+milestone/2.0.0 > >>>> > >>>> > >>>> > >>>> > >>>> > __________________________________________________________________________ > >>>> OpenStack Development Mailing List (not for usage > >>>> questions) > >>>> Unsubscribe: > >>>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>>> > >>>> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > >>>> > >>>> > >>>> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > >>>> > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > __________________________________________________________________________ > >>>> OpenStack Development Mailing List (not for usage questions) > >>>> Unsubscribe: > >>>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>>> > >>>> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > >>>> > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > __________________________________________________________________________ > >>>> OpenStack Development Mailing List (not for usage questions) > >>>> Unsubscribe: > >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>>> > >>>> <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > __________________________________________________________________________ > >>>> OpenStack Development Mailing List (not for usage questions) > >>>> Unsubscribe: > >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>> > >>> > >>> > >>> > >>> > __________________________________________________________________________ > >>> OpenStack Development Mailing List (not for usage questions) > >>> Unsubscribe: > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > >> > >> > __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
