[ https://issues.apache.org/jira/browse/CLOUDSTACK-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Animesh Chaturvedi updated CLOUDSTACK-7287: ------------------------------------------- Assignee: Nitin Mehta > [Automation] listIsos returns empty status > ------------------------------------------ > > Key: CLOUDSTACK-7287 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7287 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: API, Management Server > Affects Versions: 4.5.0 > Environment: Advanced Zone deployment: 1 Zone, 1 Pod, 1 Custer, 2x > XenServer 6.2.0 hosts > Reporter: Doug Clark > Assignee: Nitin Mehta > Priority: Critical > Fix For: 4.5.0 > > Attachments: management-server.log > > > This issues cases basic validation tests to fail > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle.test_10_attachAndDetach_iso) > *Steps to reproduce:* > 1 - Register an ISO - this appears to succeed > {noformat} > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: Payload: > {'account': u'test-a-TestVMLifeCycle-test_deploy_vm_multiple-D8PR8T', > 'domainid': u'8b113124-1e41-11e4-93e7-928e4d7c768a', 'name': u'ISO 1', > 'ispublic': True, 'isextractable': True, 'zoneid': > u'6c65f2f9-5dc3-4884-80fb-b78b85fd3f9c', 'isfeatured': True, 'apiKey': > u'0qJ7uUgC7vftyj2k8FQZOA5idiUYRTVxshkwqGWrqjvGxfZALmPJlupRn0KUsRUDJcz6A3udjKIPGydh96GgNQ', > 'displaytext': u'Test ISO 1', 'ostypeid': > u'8b23076e-1e41-11e4-93e7-928e4d7c768a', 'signature': > 'nzuaT3TOXRHHmlofoKzunO+sYq0=', 'url': > u'http://people.apache.org/~tsp/dummy.iso', 'command': 'registerIso', > 'response': 'json'} > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: > ========Sending GET Cmd : registerIso======= > requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection > (1): 10.220.135.8 > requests.packages.urllib3.connectionpool: DEBUG: "GET > /client/api?account=test-a-TestVMLifeCycle-test_deploy_vm_multiple-D8PR8T&domainid=8b113124-1e41-11e4-93e7-928e4d7c768a&name=ISO+1&isfeatured=True&ispublic=True&isextractable=True&zoneid=6c65f2f9-5dc3-4884-80fb-b78b85fd3f9c&url=http%3A%2F%2Fpeople.apache.org%2F~tsp%2Fdummy.iso&apiKey=0qJ7uUgC7vftyj2k8FQZOA5idiUYRTVxshkwqGWrqjvGxfZALmPJlupRn0KUsRUDJcz6A3udjKIPGydh96GgNQ&displaytext=Test+ISO+1&ostypeid=8b23076e-1e41-11e4-93e7-928e4d7c768a&signature=nzuaT3TOXRHHmlofoKzunO%2BsYq0%3D&command=registerIso&response=json > HTTP/1.1" 200 630 > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: Response : > [{status : u'', ostypename : u'CentOS 5.6 (64-bit)', isready : False, > domainid : u'8b113124-1e41-11e4-93e7-928e4d7c768a', name : u'ISO 1', created > : u'2014-08-07T15:38:46+0000', bootable : True, isdynamicallyscalable : > False, ispublic : True, account : > u'test-a-TestVMLifeCycle-test_deploy_vm_multiple-D8PR8T', isextractable : > True, zoneid : u'6c65f2f9-5dc3-4884-80fb-b78b85fd3f9c', isfeatured : False, > displaytext : u'Test ISO 1', ostypeid : > u'8b23076e-1e41-11e4-93e7-928e4d7c768a', domain : u'ROOT', crossZones : > False, id : u'a093a29a-f12b-4b48-856f-d385dadc5478', tags : [], zonename : > u'XenRT-Zone-0'}] > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: Successfully > created ISO with ID: a093a29a-f12b-4b48-856f-d385dadc5478 > {noformat} > 2 - Call listIsos - the ISO from step 1 is listed but the status parameter is > empty > {noformat} > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: Payload: > {'apiKey': > u'0qJ7uUgC7vftyj2k8FQZOA5idiUYRTVxshkwqGWrqjvGxfZALmPJlupRn0KUsRUDJcz6A3udjKIPGydh96GgNQ', > 'response': 'json', 'command': 'listIsos', 'signature': > 'aiTQBwVzDZ1QZFLIMDz4PP4dckA=', 'id': u'a093a29a-f12b-4b48-856f-d385dadc5478'} > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: > ========Sending GET Cmd : listIsos======= > requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection > (1): 10.220.135.8 > requests.packages.urllib3.connectionpool: DEBUG: "GET > /client/api?id=a093a29a-f12b-4b48-856f-d385dadc5478&apiKey=0qJ7uUgC7vftyj2k8FQZOA5idiUYRTVxshkwqGWrqjvGxfZALmPJlupRn0KUsRUDJcz6A3udjKIPGydh96GgNQ&command=listIsos&response=json&signature=aiTQBwVzDZ1QZFLIMDz4PP4dckA%3D > HTTP/1.1" 200 628 > test_10_attachAndDetach_iso > (integration.smoke.test_vm_life_cycle.TestVMLifeCycle): DEBUG: Response : > [{status : u' ', ostypename : u'CentOS 5.6 (64-bit)', isready : False, > domainid : u'8b113124-1e41-11e4-93e7-928e4d7c768a', name : u'ISO 1', created > : u'2014-08-07T15:38:46+0000', bootable : True, isdynamicallyscalable : > False, ispublic : True, account : > u'test-a-TestVMLifeCycle-test_deploy_vm_multiple-D8PR8T', isextractable : > True, zoneid : u'6c65f2f9-5dc3-4884-80fb-b78b85fd3f9c', isfeatured : False, > displaytext : u'Test ISO 1', ostypeid : > u'8b23076e-1e41-11e4-93e7-928e4d7c768a', domain : u'ROOT', crossZones : > False, id : u'a093a29a-f12b-4b48-856f-d385dadc5478', tags : [], zonename : > u'XenRT-Zone-0'}] > {noformat} > From apilog.log > {noformat} > 2014-08-07 15:39:47,039 INFO [a.c.c.a.ApiServer] > (catalina-exec-9:ctx-500db984 ctx-8881134d ctx-61bad595) (userId=2 > accountId=2 sessionId=null) 10.220.135.9 -- GET > id=a093a29a-f12b-4b48-856f-d385dadc5478&apiKey=0qJ7uUgC7vftyj2k8FQZOA5idiUYRTVxshkwqGWrqjvGxfZALmPJlupRn0KUsRUDJcz6A3udjKIPGydh96GgNQ&command=listIsos&response=json&signature=aiTQBwVzDZ1QZFLIMDz4PP4dckA%3D > 200 { "listisosresponse" : { "count":1 ,"iso" : [ > {"id":"a093a29a-f12b-4b48-856f-d385dadc5478","name":"ISO > 1","displaytext":"Test ISO > 1","ispublic":true,"created":"2014-08-07T15:38:46+0000","isready":false,"bootable":true,"isfeatured":false,"crossZones":false,"ostypeid":"8b23076e-1e41-11e4-93e7-928e4d7c768a","ostypename":"CentOS > 5.6 > (64-bit)","account":"test-a-TestVMLifeCycle-test_deploy_vm_multiple-D8PR8T","zoneid":"6c65f2f9-5dc3-4884-80fb-b78b85fd3f9c","zonename":"XenRT-Zone-0","status":" > > ","domain":"ROOT","domainid":"8b113124-1e41-11e4-93e7-928e4d7c768a","isextractable":true,"tags":[],"isdynamicallyscalable":false} > ] } } > {noformat} > There also seem to be a problem when the associated account is later deleted > - from MS.log (attached) > {noformat} > 2014-08-07 15:39:47,137 DEBUG [c.c.u.AccountManagerImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Successfully deleted > snapshots directories for all volumes under account 13 across all zones > 2014-08-07 15:39:47,146 INFO [c.c.t.HypervisorTemplateAdapter] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Delete template from > image store: nfs://10.220.128.55/vol/xenrtnfs/781738-bferyW > 2014-08-07 15:39:47,148 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) template 205 is > already in store:1, type:Image > 2014-08-07 15:39:47,149 WARN [c.c.u.AccountManagerImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Failed to delete > template while removing account: ISO 1 due to: > com.cloud.utils.exception.CloudRuntimeException: Failed to update state > at > org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:173) > at > org.apache.cloudstack.storage.image.TemplateServiceImpl.deleteTemplateAsync(TemplateServiceImpl.java:601) > at > com.cloud.template.HypervisorTemplateAdapter.delete(HypervisorTemplateAdapter.java:360) > at > com.cloud.template.TemplateManagerImpl.delete(TemplateManagerImpl.java:756) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) > at $Proxy173.delete(Unknown Source) > at > com.cloud.user.AccountManagerImpl.cleanupAccount(AccountManagerImpl.java:719) > at > com.cloud.user.AccountManagerImpl.deleteAccount(AccountManagerImpl.java:667) > at > com.cloud.user.AccountManagerImpl.deleteUserAccount(AccountManagerImpl.java:1441) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) > at > com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) > at $Proxy102.deleteUserAccount(Unknown Source) > at > org.apache.cloudstack.region.RegionManagerImpl.deleteUserAccount(RegionManagerImpl.java:187) > at > org.apache.cloudstack.region.RegionServiceImpl.deleteUserAccount(RegionServiceImpl.java:121) > at > org.apache.cloudstack.api.command.admin.account.DeleteAccountCmd.execute(DeleteAccountCmd.java:104) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141) > at > com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > Caused by: com.cloud.utils.fsm.NoTransitionException: Unable to transition to > a new state from Creating via DestroyRequested > at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:86) > at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:102) > at > org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:290) > at > org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:171) > ... 50 more > 2014-08-07 15:39:47,151 WARN [c.c.u.AccountManagerImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Failed to delete > templates while removing account id=13 > 2014-08-07 15:39:47,154 DEBUG [c.c.u.AccountManagerImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Expunging # of vms > (accountId=13): 3 > 2014-08-07 15:39:47,162 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Sync job-242 > execution on object VmWorkJobQueue.26 > 2014-08-07 15:39:47,164 WARN [c.c.u.d.Merovingian2] > (API-Job-Executor-116:ctx-b80174a5 job-241 ctx-b0b4c52f) Was unable to find > lock for the key vm_instance26 and thread id 1204392491 > 2014-08-07 15:39:47,489 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] > (AsyncJobMgr-Heartbeat-1:ctx-133db1c5) Execute sync-queue item: > SyncQueueItemVO {id:77, queueId: 67, contentType: AsyncJob, contentId: 240, > lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, > created: Thu Aug 07 15:39:45 UTC 2014} > {noformat} -- This message was sent by Atlassian JIRA (v6.2#6252)