Comments Inline.
-----Original Message-----
From: Edison Su [mailto:edison...@citrix.com]
Sent: Tuesday, October 08, 2013 4:18 AM
To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
Few questions:
1. About the "more object-oriented" CloudStack API python binding: Is the
proposed api good enough?
For example,
The current hand written create virtual machine looks like:
class VirtualMachine(object):
....
@classmethod
def create(cls, apiclient, services, templateid=None, accountid=None,
domainid=None, zoneid=None, networkids=None,
serviceofferingid=None,
securitygroupids=None, projectid=None, startvm=None,
diskofferingid=None, affinitygroupnames=None, group=None,
hostid=None, keypair=None, mode='basic', method='GET'):
the proposed api may look like:
class VirtualMachine(object):
def create(self, apiclient, accountId, templateId, **kwargs)
The proposed api will look better than previous one, and it's automatically
generated, so easy to maintain. But as a consumer of the api, how do people
know what kind of parameters should be passed in? Will you have an online
document for your api? Or you assume people will look at the api docs generated
by CloudStack?
Or why not make the api itself as self-contained? For example, add docs before
create method:
class VirtualMachine(object):
'''
Args:
accountId: what ever
templateId: whatever
networkids: whatever
'''
'''
Response:
'''
def create(self, apiclient, accountId, templateId, **kwargs)
All the api documents should be included in api discovery already, so it should
be easy to add them in your api binding.
>> [Santhosh]: Each verb as an action on entity, will have provision as earlier
>> to have all required and as well optional arguments. Regarding doc strings,
>> If the API docs are having this facilitation, we will add them as
>> corresponding doc strings during generation of python binding and as well
>> entities. As you rightly mentioned, it will good to add this . We will make
>> sure to get it. Adding adequate doc strings applies even while writing test
>> feature\lib as well, it will improve ease ,readability,usage etc. Anyways a
>> wiki page, and additional pydoc documents posted online will be there.
2. Regarding to data factories. From the proposed factories, in each test case,
does test writer still need to write the code to get data, such as writing code
to get account during the setupclass?
I looked at some of the existing test cases, most of them have the same code
snippet:
class Services:
def __init__(self):
self.services = {
"account": {
"email": "t...@test.com<mailto:t...@test.com>",
"firstname": "Test",
"lastname": "User",
"username": "test",
"password": "password",
},
"virtual_machine": {
"displayname": "Test VM",
"username": "root",
"password": "password",
"ssh_port": 22,
"hypervisor": 'XenServer',
"privateport": 22,
"publicport": 22,
"protocol": 'TCP',
},
With the data factories, the code will look like the following?
Class TestFoo:
Def setupClass():
Account = UserAccount(apiclient)
VM = UserVM(apiClient)
And if I want to customize the default data factories, I should be able to use
something like: UserAccount(apiclient, username='myfoo')?
And the data factories should be able to customized based on test environment,
right?
For example, the current iso test cases are hardcoded to test against
http://people.apache.org/~tsp/dummy.iso, but it won't work for devcloud, or in
an internal network. The ISO data factory should be able to return an url based
on different test environment, thus iso test cases can be reused.
>>[Santhosh] : Currently, as you mentioned, Services class is part of many test
>>modules, this is basically data part for the test. We are separating this
>>with factory approach. Thus, segregating data from test. Compare the earlier
>>mention of Services class in earlier test code without Service class in the
>>below test code.
class TestVpcLifeCycle(cloudstackTestCase):
def setUp(self):
self.apiclient = super(TestVpcLifeCycle,
self).getClsTestClient().getApiClient()
self.zoneid = get_zone(self.apiclient).id
self.templateid = get_template(self.apiclient).id
self.serviceofferingid = get_service_offering(self.apiclient).id
self.account = UserAccount( apiclient=self.apiclient ) ---> Data
factory creation
@attr(tags='debug')
def test_deployvm(self):
vm = VpcVirtualMachine(
apiclient=self.apiclient,
account=self.account.name,
domainid=self.account.domainid,
zoneid=self.zoneid,
templateid=self.templateid,
serviceofferingid=self.serviceofferingid
)
vm.state | should | equal_to('Running')
There could be many ways of data driving\In this case providing simple data to
the test cases. Factory approach though better is driven to create factories
with default sequences once and reused. Yes, user can customize a data factory
as you mentioned. customizations specific to environment are not available
though, but doable while creating a factory instance. We can take a care to
look at it. Basic data factory for an entity by default will have values based
upon environment available and then user can customize accordingly, this can be
added to proposal as edit. But using factories logic, do you see any issues
as a whole in long run? Any disadvantages you foresee? Or using plain data
driving approach through sample xml\csv file for a given test module should
suffice i.e., provide a facilitation as part of current testclient itself as
dynamic data object creation as similar to the one provided by db connection as
part of test client currently, the new data object therefore as part of
testclient will then replace the current Service Class\object creation for a
given test module. Loading the values from xml\csv in to data object for test
can be controlled through a config value( path to data values file(xml\csv) for
a given test module ) and are loaded during the object creation? This object
through testclient is available for test module to be used as a whole?
Thanks!
Santhosh
> -----Original Message-----
> From: Santhosh Edukulla [mailto:santhosh.eduku...@citrix.com]
> Sent: Monday, October 07, 2013 7:06 AM
> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> Subject: RE: [DISCUSS] Breaking out Marvin from CloudStack
>
> Team,
>
> Apart\Away from breaking out marvin from cloudstack, please check the
> other new details provided as part of the new proposal for marvin
> refactoring. Your inputs out of experience are invaluable. Any new
> feature tests for CS will be followed with the new approach, provided
> if we agree to all. Pasting the proposal link one more time below.
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+Refactor
>
> Regards,
> Santhosh
> ________________________________________
> From: Daan Hoogland [daan.hoogl...@gmail.com]
> Sent: Sunday, October 06, 2013 3:05 PM
> To: dev
> Subject: Re: [DISCUSS] Breaking out Marvin from CloudStack
>
> On Sun, Oct 6, 2013 at 8:10 PM, Animesh Chaturvedi <
> animesh.chaturv...@citrix.com<mailto:animesh.chaturv...@citrix.com>> wrote:
>
> > > Yes and we will need to work down a backlog of scenarios before we
> > > ever can rely on guys like me doing that. Not because they won't
> > > but because there is to much to write tests for edging on the new
> > > features they write. Just because those tests aren't there yet. I
> > > think giving Citrix QA a repo to work on is fine but I would like
> > > to see it merged back at some point and a continued possibility to
> > > write
> them in the main tree.
> > >
> > [Animesh>] While I don't agree to a separate repo for tests (marvin
> > framework is ok) I do want to call out the proposal is not for
> > giving Citrix QA a repo to work on and I don't think Prasanna meant that
> > way.
> >
>
>
> I have to apologize for the formulations I choose to express my
> thoughts with. I did not mean to talk of a department of a certain
> company donating human effort for testing to the community. I was
> talking of the frustration of the individuals working and how the
> separate repo would smoothen their workflow. The new repo is in the
> apache domain, no question whether the work in their is done by one person or
> 100 companies.
