Umesh, This could be probably due to the bug under get_template function under marvin/integration/lib/common.py. Though, the ostype description is used and corresponding id is retrieved.
But the later part, listTemplate API call returning the list and we are picking up either the template which is ready or matching with id we retrieved. This case is what could be causing the issue you mentioned. Regards, Santhosh ________________________________________ From: umesh kute [umeshvk...@gmail.com] Sent: Friday, January 03, 2014 5:04 AM To: Santhosh Edukulla; users@cloudstack.apache.org Cc: Chandan Purushothama; Sudha Ponnaganti Subject: Re: Regarding cloudstack automation Thanks for your help Santhosh!!! You are right... The public IP address range was exhausted. After cleaning up the resources, I was able to run the sample script "test_deploy_vm.py" successfully. One question - i have provided the "ostype": 'CentOS 6.4 (64-bit)'. What is the use of this parameter? When observed in management server logs and on cloudstack UI (while running script), VM was created of type: "Windows Server 2008 R2 Enterprise with SQL 2012 Web Edition". This is the first one in the list. Does it choose the one which we provided in script or picks it from the template list that is returned as a result of response to listTemplate API call? Thanks and Regards -- Umesh Kute On Thu, Jan 2, 2014 at 7:22 PM, Santhosh Edukulla <santhosh.eduku...@citrix.com<mailto:santhosh.eduku...@citrix.com>> wrote: Umesh, 1. It seems you have exhausted few of CloudStack Resources. These resources usage should show up on your cloudstack dashboard. Anything appears in Red are exhausted resources. 2. Few of the cases could be EX: Guest VM IPs\Management ip addresses are exhausted, 3. Check for these resources in Red on your dashboard EX: http://localhost:8080/client/->Dashboard and clean up them or add new resources. Santhosh ________________________________________ From: umesh kute [umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>] Sent: Thursday, January 02, 2014 7:06 AM To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org> Cc: Chandan Purushothama; Sudha Ponnaganti Subject: Re: Regarding cloudstack automation Thank you once again Santhosh & Chandan. I have gone through the management server logs. The deployment of VM got fired to the setup correctly and there are couple of errors in the management-server log. I am summarizing those below: test data in "test_deploy_vm.py" def __init__(self): self.testdata = { #data to create an account "account": { "email": "t...@test.com<mailto:t...@test.com><mailto:t...@test.com<mailto:t...@test.com>>", "firstname": "Test", "lastname": "User", "username": "test", "password": "password", }, #data reqd for virtual machine creation "virtual_machine" : { "name" : "testvmshr", "displayname" : "Test VM shr", }, #small service offering "service_offering": { "small": { "name": "SOSmall", "displaytext": "Small Instance", "cpunumber": 1, "cpuspeed": 2000, "memory": 1024, }, }, "ostype": 'CentOS 6.4 (64-bit)', } 1. Account is successfully created 2. Service offering is successfully created 3. Management server logs - management-server.log has following error: 2014-01-02 01:43:40,693 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-109:job-26355 = [ 78b8af27-89dc-426e-bba7-acf693f071f2 ]) Lock is released for network Ntwk[326|Guest|15] as a part of network shutdown 2014-01-02 01:43:40,695 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-109:job-26355 = [ 78b8af27-89dc-426e-bba7-acf693f071f2 ]) Lock is released for network id 326 as a part of network implement 2014-01-02 01:43:40,695 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-109:job-26355 = [ 78b8af27-89dc-426e-bba7-acf693f071f2 ]) Insufficient capacity com.cloud.exception.InsufficientAddressCapacityException: Insufficient address capacityScope=interface com.cloud.dc.DataCenter; id=1 at com.cloud.network.NetworkManagerImpl.fetchNewPublicIp(NetworkManagerImpl.java:479) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.network.NetworkManagerImpl.assignDedicateIpAddress(NetworkManagerImpl.java:613) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.network.NetworkManagerImpl.assignSourceNatIpAddressToGuestNetwork(NetworkManagerImpl.java:582) at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2013) at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1939) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2120) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:887) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578) at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227) at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966) at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) 2014-01-02 01:43:40,703 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-109:job-26355 = [ 78b8af27-89dc-426e-bba7-acf693f071f2 ]) Cleaning up resources for the vm VM[User|testvmshr] in Starting state 4. When i checked the database for the network id 326 in table "networks", it has following entry: | 326 | test-TestDeployVM-test_deploy_vm-QQ91N0-network | b7cb9e31-835f-47e3-9585-97e3b4084e4d | test-TestDeployVM-test_deploy_vm-QQ91N0-network | Guest | Vlan | NULL | 10.1.1.1 | 10.1.1.0/24<http://10.1.1.0/24><http://10.1.1.0/24> | Dhcp | 15 | 200 | 1 | ExternalGuestNetworkGuru | Destroy | 326 | 1 | 305 | 8.8.8.8 | 8.8.4.4 | NULL | 0 | Account | cs131cloud.internal | ecf5b617-04da-469b-b378-f53e05d8cc53 | Isolated | 0 | 2014-01-02 06:43:39 | 2014-01-02 06:43:46 | 0 | NULL | NULL | NULL | NULL | 1 | NULL | Basically it tries to create a network offering (id=15) : "HA Isolated Network With Source Nat" 5. I think the script creates a network shown in above table: "test-TestDeployVM-test_deploy_vm-QQ91N0-network". I am not sure why it still gives the Insufficient address capacityScope exception. 6. One more thing i observed. I have provided the "ostype": 'CentOS 6.4 (64-bit)'. But in DeployVMCmd it uses the template id: "03ffb96f-2cfa-11e3-a4f0-f245a5b3ba0f" which is for template "Windows Server 2008 R2 Enterprise with SQL 2012 Web Edition". I am attaching the management-server logs and script logs for references. If somebody can help me what's wrong here, would really be helpful. Thanks and Regards -- Umesh Kute On Thu, Jan 2, 2014 at 2:59 PM, Santhosh Edukulla <santhosh.eduku...@citrix.com<mailto:santhosh.eduku...@citrix.com><mailto:santhosh.eduku...@citrix.com<mailto:santhosh.eduku...@citrix.com>>> wrote: 1. Is it still failing post this registration? 2. If yes, may be you can verify\provide the management server logs. ________________________________________ From: umesh kute [umeshvk...@gmail.com<mailto:umeshvk...@gmail.com><mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>>] Sent: Thursday, January 02, 2014 4:16 AM To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>> Cc: Chandan Purushothama; Sudha Ponnaganti Subject: Re: Regarding cloudstack automation Thanks Santhosh. Yes, I have registered the following template which i am using. "ostype": 'CentOS 6.4 (64-bit)', Thanks and Regards -- Umesh Kute On Mon, Dec 30, 2013 at 5:50 PM, Santhosh Edukulla < santhosh.eduku...@citrix.com<mailto:santhosh.eduku...@citrix.com><mailto:santhosh.eduku...@citrix.com<mailto:santhosh.eduku...@citrix.com>>> wrote: > Umesh, > > One question : Did you registered the template for ostype: "ostype": > 'CentOS 5.3 (64-bit)'? > > Regards, > Santhosh > ________________________________________ > From: umesh kute > [umeshvk...@gmail.com<mailto:umeshvk...@gmail.com><mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>>] > Sent: Monday, December 30, 2013 6:32 AM > To: Chandan Purushothama > Cc: > users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>>; > Sudha Ponnaganti > Subject: Re: Regarding cloudstack automation > > Hi Chandan, > Thanks for your response. > I have the latest Marvin repository (December 20th). I am able to proceed > further. There was an issue of firewall for that machine and it got > resolved. > > But now i am getting another error when i tried to run the script > "test_deploy_vm.py" > > I have attached the log. Can somebody please have a look on it and help me > in resolving the issue? > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > test_deploy_vm (integration.smoke.test_deploy_vm.TestDeployVM): CRITICAL: > EXCEPTION: test_deploy_vm: Traceback (most recent call last): > File "/usr/local/lib/python2.7/unittest/case.py", line 327, in run > testMethod() > File > "/DataDisk/temp/cloudstack/test/integration/smoke/test_deploy_vm.py", line > 116, in test_deploy_vm > > templateid=self.template.id<http://self.template.id><http://self.template.id><http://self.template.id> > File > "/usr/local/lib/python2.7/site-packages/marvin/integration/lib/base.py", > line 415, in create > virtual_machine = apiclient.deployVirtualMachine(cmd, method=method) > File > "/usr/local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", > line 593, in deployVirtualMachine > response = self.connection.marvinRequest(command, > response_type=response, method=method) > File > "/usr/local/lib/python2.7/site-packages/marvin/cloudstackConnection.py", > line 280, in marvinRequest > response = self.poll(asyncJobId, response_type) > File > "/usr/local/lib/python2.7/site-packages/marvin/cloudstackConnection.py", > line 86, in poll > "asyncquery", asyncResonse.jobresult) > cloudstackAPIException: Execute cmd: asyncquery failed, due to: {errorcode > : 533, errortext : u'Unable to create a deployment for VM[User|testvmshr]'} > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > PFA entire logs. > > Please let me know in case if anything is required. > > Thanks and Regards > -- Umesh Kute > > > On Sun, Dec 29, 2013 at 10:32 PM, Chandan Purushothama < > chandan.purushoth...@citrix.com<mailto:chandan.purushoth...@citrix.com><mailto:chandan.purushoth...@citrix.com<mailto:chandan.purushoth...@citrix.com>><mailto:chandan.purushoth...@citrix.com<mailto:chandan.purushoth...@citrix.com><mailto:chandan.purushoth...@citrix.com<mailto:chandan.purushoth...@citrix.com>>>> > wrote: > > Hello Umesh, > > I had a similar experience with an older Marvin build. I updated mine to > the latest and tried again and it worked. I recommend you to kindly try > again with the latest Marvin build and see if you are able to work with the > automation framework, > > Thank you, > Chandan > > > -------- Original message -------- > From: umesh kute > Date:12/26/2013 12:29 AM (GMT-08:00) > To: > users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>>>,Sudha > Ponnaganti > Subject: Re: Regarding cloudstack automation > > Hi All, > I have created a Marvin setup on CentOS6.3 by following the document: > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python > > All the pre-requisites are installed successfully. > > I tried to run the sample script test_deploy_vm.py > (cloudstack/test/integration/smoke/), I am getting following error: > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > [root@localhost smoke]# nosetests --with-marvin > --marvin-config=/DataDisk/temp/cloudstack/tools/marvin/dist/demo.cfg --load > test_deploy_vm.py > > Exception Occurred Under __deployDC : > HTTPConnectionPool(host='10.88.91.100', port=8096): Max retries exceeded > with url: /client/api?account=admin&command=listUsers&response=json (Caused > by <class 'socket.error'>: [Errno 110] Connection timed out) > Traceback (most recent call last): > File "/usr/local/bin/nosetests", line 9, in <module> > load_entry_point('nose==1.3.0', 'console_scripts', 'nosetests')() > File > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/core.py", > line 118, in __init__ > **extra_args) > File "/usr/local/lib/python2.7/unittest/main.py", line 95, in __init__ > self.runTests() > File > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/core.py", > line 197, in runTests > result = self.testRunner.run(self.test) > File > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/core.py", > line 61, in run > test(result) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 176, in __call__ > return self.run(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/suite.py", > line 223, in run > test(orig) > File > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", > line 45, in __call__ > return self.run(*arg, **kwarg) > File > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/case.py", > line 138, in run > result.addError(self, err) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/proxy.py", > line 124, in addError > plugin_handled = plugins.handleError(self.test, err) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/plugins/manager.py", > line 99, in __call__ > return self.call(*arg, **kw) > File > > "/usr/local/lib/python2.7/site-packages/nose-1.3.0-py2.7.egg/nose/plugins/manager.py", > line 167, in simple > result = meth(*arg, **kw) > File "/usr/local/lib/python2.7/site-packages/marvin/marvinPlugin.py", > line 155, in handleError > self.tcRunLogger.fatal("%s: %s: %s" % > AttributeError: 'NoneType' object has no attribute 'fatal' > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > The logs are generated at /tmp and the log has following error. > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 2013-12-26 13:40:51,797 - DEBUG - CSLog - sending GET request: listUsers > {'account': 'admin'} > 2013-12-26 13:41:54,825 - DEBUG - CSLog - Connection refused. Reason: > http://10.88.91.100:8096/client/api : > HTTPConnectionPool(host='10.88.91.100', port=8096): Max retries exceeded > with url: /client/api?account=admin&command=listUsers&response=json (Caused > by <class 'socket.error'>: [Errno 110] Connection timed out) > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > Please note: I have opened the integration port 8096 on management server. > Also, changed the global settings integration.api.port > on cloudstack GUI to 8096 and restarted the management server. > > Still I am getting the above issue. I am not sure if i missed any > configuration. > > Can somebody please help me with resolution of this issue? > Appreciate your help here!!! > > Thanks and Regards > -- Umesh Kute > > > On Mon, Dec 9, 2013 at 1:57 PM, umesh kute > <umeshvk...@gmail.com<mailto:umeshvk...@gmail.com><mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>><mailto: > umeshvk...@gmail.com<mailto:umeshvk...@gmail.com><mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>>>> > wrote: > > > Thanks a lot Sudha for your quick response. > > I will setup the Marvin on local machine and get back in case if I face > > any challenges in setup. > > > > Thanks and Regards > > -- Umesh Kute > > > > > > > > On Sun, Dec 8, 2013 at 4:05 PM, Sudha Ponnaganti < > > sudha.ponnaga...@citrix.com<mailto:sudha.ponnaga...@citrix.com><mailto:sudha.ponnaga...@citrix.com<mailto:sudha.ponnaga...@citrix.com>><mailto:sudha.ponnaga...@citrix.com<mailto:sudha.ponnaga...@citrix.com><mailto:sudha.ponnaga...@citrix.com<mailto:sudha.ponnaga...@citrix.com>>>> > > wrote: > > > >> Hi Umesh, > >> > >> Looks like you have started on right track - Marvin is the framework to > >> use for functional or integration automation. > >> There are sizable number of test cases exist and you can review them. > >> There are some wiki pages already written explaining Marvin [1]. Also > best > >> practices are published by community as well [2]. You can contribute to > it > >> as much as you can. > >> > >> Besides this, there is Junit framework (unit Test), Selenium framework > >> (UI automation) which are not as popular as Marvin. Marvin is perfect > for > >> majority of the validation. You can use simulator as well with Marvin. > >> > >> Please do pursue these and post questions to the community if you run in > >> to any issues in setting up env and running tests. There are jobs setup > by > >> various community members to test Master or current releases in various > >> labs and can help answer questions. > >> > >> [1] > >> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python > >> > >> > >> Thanks > >> /Sudha > >> > >> > >> -----Original Message----- > >> From: umesh kute > >> [mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com><mailto:umeshvk...@gmail.com<mailto:umeshvk...@gmail.com>>] > >> Sent: Saturday, December 07, 2013 10:29 PM > >> To: > >> users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org<mailto:users@cloudstack.apache.org>>> > >> Subject: Regarding cloudstack automation > >> > >> Hi All, > >> Good morning !!! > >> I am working as QA engineer and I am new to the cloudstack community. > >> We have started using cloudstack and deployed it in our infrastructure. > >> > >> Can somebody please help me in identifying the good automation > >> framework/tool/anything? The main intention is we want to automate the > >> infrastructure validation related cases (for e.g. create/update/delete > >> cases for instances, project, network, etc...) > >> > >> It would be good if you can help in identifying it. > >> > >> Please note: I have gone through cloudmonkey. But i haven't found much > >> help on it. May be i need to search a bit more.. I am also going through > >> the marvin and planning to setup it. If apart from these two, if anyone > has > >> different view please suggest. > >> > >> Also, if somebody has any pointers on cloudmonkey and marvin or any more > >> info on this (like, if it would suffice the infrastructure validation > >> related cases i mentioned above) any pros/cons would really be helpful > for > >> me... > >> > >> Appreciate any help on this and looking forward for the response on > >> this!!! > >> > >> Thanks and Regards > >> -- Umesh Kute > >> > > > > > >
