Re: [openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests

[Zhu Bo]

Hi,

I also collect the tests status for nova v3 api manually. You can find 
the status of v3 tests in this link: 
https://etherpad.openstack.org/p/nova-v3-tests


Because there are some extensions that just extend the attribute of 
specific api reponse, or convert some input before specific api
called. There is no url for these extension, but I think we also need 
check them. I collect the tempest tests according to the nova api code. 
check tests to the action

one by one, list the status file by file.

Due to these patch https://review.openstack.org/#/c/39609/ and 
https://review.openstack.org/#/c/39621/ are still under reviewing. so we 
can't add tempest test for nova v3 api. (almost existing tests have been 
ported into v3, but these patches depend on the 39609 and 39621). The 
status of porting existing tests is also listed in this link. we will 
add the missing tests in v2

firstly, when it can be ported into v3, will port it.
*__*
Best Regards
Ivan Zhu

On 2013?10?15? 14:25, Masayuki Igawa wrote:

Hi,

First, thank you to an anonymous for updating this list!
 -> GET /{project_id}/servers/:server_id/diagnostics

And, I have updated: Nova API List for Missing Tempest Tests.
   
https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc

The summary of this list:
different count from
Tested or not # of APIs ratio   the last time
---
Tested API  124  49.6%  +2
Not Tested API   66  26.4%  -2
Not Need to Test(*1) 60  24.0%   0
---
Total(*2):  250 100.0%   0
(*1) Because they are deprecated APIs such as nova-network and volume.
(*2) not included v3 APIs

The tempest version is:
  commit f55f4e54ceab7c6a4d330f92c8059e46233e3560
  Merge: 86ab238 062e30a
  Author: Jenkins 
  Date:   Mon Oct 14 15:55:59 2013 +

By the way, I saw a design summit proposal related to this topic(*3). I think
this information should be generated automatically. So I'd like to talk about
this topic at the summit session.
(*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171

This information would be useful for creating Tempest tests.
Any comments/questions/suggestions are welcome.

Best Regards,
-- Masayuki Igawa



Hi,

# I'm sorry for this resending because my last mail has unnecessary messages.


I have updated: Nova API List for Missing Tempest Tests.
  
https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc

The summary of this list:
different count from
Tested or not# of APIs  ratio   the last time
---
Tested API  122  48.8%  +5
Not Tested API   68  27.2%  -5
Not Need to Test(*1) 60  24.0%   0
---
Total(*2):  250 100.0%   0

(*1) Because they are deprecated APIs such as nova-network and volume.
(*2) not included v3 APIs

I hope this information would be helpful for creating Tempest tests.
Any comments and questions are welcome.

Best Regards,
-- Masayuki Igawa



Hi, Tempest developers

I have made:
  Nova API List for Missing Tempest Tests.
  
https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc

This list shows what we should test. That is:
  * Nova has 250 APIs(not include v3 APIs).
  * 117 APIs are executed(maybe tested).
  * 73 APIs are not executed.
  * 60 APIs are not executed. But they maybe not need to test.
  -> Because they are deprecated APIs such as nova-network and volume.

So I think we need more tempest test cases.
If this idea is acceptable, can you put your name to 'assignee' at your 
favorites,
and implement tempest tests.

Any comments are welcome.

Additional information:
  I made this API list with modification of nova's code that based on
  https://review.openstack.org/#/c/25882/ (Abandoned).

Best Regards,
-- Masayuki Igawa








___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Duplicated test effort development

[Zhu Bo]

hi, Chris thanks for your work.

It's a good way. And how about  creating a blue-print and putting these 
link on it? Some guys have been familiar with looking for work items 
from blue-print, and writing something in  hacking guide may be better.


On 2013?10?30? 21:10, Christopher Yeoh wrote:
On Wed, Oct 30, 2013 at 11:18 PM, Steven Hardy > wrote:


On Wed, Oct 30, 2013 at 11:00:22PM +1030, Christopher Yeoh wrote:

> I don't think blueprints/bugs work so
> well at this, and I don't think we have anything else setup at
the moment,
> so as a temporary measure I've created an etherpad here:

Can I ask why?  Surely blueprints for new features (in this case the
feature is test coverage for $api) are exactly what the normal
openstack
process dictates, and is what most folks are familiar with?


Just to be clear - Its not that I think that we shouldn't have 
blueprints which covers the work being done (we should!), but they 
don't work so well at allowing people to see a good summary of what 
test coverage for an API we have (some of which may have been done a 
long time ago), what needs to be done and the quite fine grained 
allocation of what people are working on.


For example, see the tempest coverage for the Nova v2 API spreadsheet: 
https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc#gid=0


Anyway, I added the keystone test I'm working on (which has a BP)
to the
etherpad, and definitely +1 on not duplicating effort, by whatever
means ;)

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] disable/enable services and agent tests

[Zhu Bo]

I think we should remove these tests, even though tempest run in serial. 
Because we can't assume the enable service action will be successful.


(But on the other hand, one test is failed, the tempest gate will be 
failed. We need find it out and fix it. :-) )


On 2013?11?13? 10:23, Zhi Kun Liu wrote:

Hi all,

In Tempest, we have some testcases that disable/enable services or 
agents. e.g. 
https://review.openstack.org/#/c/55271/1/tempest/api/compute/admin/test_services.py

test_service_enable_disable
test_disable_service_with_disable_reason

Since Tempest run in parallel for now, I'm afraid of these testcases 
have possible impact on other tests.  Anyone has ideas about it? 
Should we remove such testcases?


Regards,
Zhi Kun


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Network stuff in Nova API v3

[Zhu Bo]

On 2013年08月07日 21:42, Alex Xu wrote:

On 2013年08月07日 17:38, John Garbutt wrote:

multi-nic added an extra virtual interface on a seprate network, like
adding a port:
http://docs.openstack.org/trunk/openstack-compute/admin/content/using-multi-nics.html 

That just describe create instance with multinic, that we will 
support. Still have problem
with action add_fixed_ip and remove_fixed_ip in extension multinic. 
Those action

invoke inject_network_info and reset_network.


I think we need to keep a nova-network focused api extension, and a
separate neutron focused api extension, because we have not yet
removed neutron. It should probably proxy the neutron information
still, so people can more easily transition between nova-network and
neutron.

Sound good, thanks.
Nova v2 api will be saved with v3 for some time, I think.  Why not just 
keep neutron api extension in v3?
I think people can have enough time to understand the difference between 
v2 and v3. If we keep
api for nova-network in v3, we will still face the same problem when 
next api version occur  or when

remove the nova-network.

I agree we should probably slim down the neturon focused api extension.

Howerver, it should probably include network-ids and port-ids for each
port, if we still support both:
 nova boot --image  --flavor  --nic net-id=
--nic net-id= 
and this:
nova boot --image  --flavor  --nic port-id= 


Yes, we still support those. But why we need network-ids?

Longer term, we still need the metadata service to provide networking
information, so there will be a nova-api that has to proxy info from
neutron, but I agree we should reduce where we can.

agree with this. There will be a nova-api that has to proxy info from
neutron, but we should reduce where we can.


John

On 7 August 2013 10:08, Alex Xu  wrote:

Hi, guys,

Currently we have one core and two extensions that related network 
in Nova

API v3.
They are ips, attach_interface and multinic. I have two questions 
for them.


The first question is about ips and attach_interface. The below was the
index's response
of ips and attach_interface:
ips:
{
 "addresses": {
 "net1": [
 {
 "addr": "10.0.0.8",
 "mac_addr": "fa:16:3e:c2:0f:aa",
 "type": "fixed",
 "version": 4
 },
 {
 "addr": "30.0.0.5",
 "mac_addr": "fa:16:3e:c2:0f:aa",
 "type": "floating",
 "version": 4
 }
 ]
 }
}

attach_interface:
{
 "interface_attachments": [
 {
 "fixed_ips": [
 {
 "ip_address": "10.0.0.8",
 "subnet_id": 
"f84f7d51-758c-4a02-a4c9-171ed988a884"

 }
 ],
 "mac_addr": "fa:16:3e:c2:0f:aa",
 "net_id": "b6ba34f1-5504-4aca-825b-04511c104802",
 "port_id": "3660380b-0075-4115-be96-f08b41ccdf5d",
 "port_state": "ACTIVE"
 }
 ]
}

The problem is the responses are similar, but just with different 
view,  and

all the information can
get from Neutron directly. I think we didn't want to proxy Neutron 
through

Nova. So how about
we merge ips and attach_interface into an new extension. The new 
extension

will be include the
things as below:
1. Extend the detail of servers to list the uuid of port. User can 
get more

information from Neutron
by port uuid.
2. Attach and detach interface that move from extension 
attach_interface.
3. Extend the creation of servers to support network (The patch 
already here

https://review.openstack.org/#/c/36615/)

The second question is about multinic. Looking into the code, 
multinic just

add fixed_ip for server's port.
That can be done by Neutron API directly too. But there are
inject_network_info and reset_network
in the code. Only xen and vmware's driver implement that function. 
I'm not

familiar with xen and
vmware, I guess it use guest agent to update the guest network. If I am
right, I think we didn't
encourage using that way to update guest network.There are api for
inject_network_info and reset_network
in extension admin-actions also. I think we can keep them. But can 
we delete

multinic for V3?

Thanks
Alex


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-de

[openstack-dev] [qa] How to do nova v3 tests in tempest

[Zhu Bo]

hi,
  I'm working on bp:nova-v3-tests in tempest.  The nova tests in 
tempest mostly have been ported into v3 and sent off.
but we got some feedbacks that there was mass code duplication and 
suggested to do this by inheritance.
So I have sent another patch to do this by inheritance. But in this way, 
another issue is not easy to drop v2 client and tests.
I want to get more feedbacks about this blue-print to make sure we do 
this in the right way, which is the better one or is there

another better way? I'd appreciate every suggestion and comment.

the first way to do this in separate files:
https://review.openstack.org/#/c/39609/ and 
https://review.openstack.org/#/c/39621/6


the second way to do this by inheritance.
https://review.openstack.org/#/c/44876/

Thanks & Best Regards

Ivan


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [qa][tempest] Core review request

[Zhu Bo]

Hi, guys,

Please help to review my changes:https://review.openstack.org/#/c/39609/

Thanks in advance
Ivan


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev