[ https://issues.apache.org/jira/browse/CLOUDSTACK-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13798561#comment-13798561 ]
ASF subversion and git services commented on CLOUDSTACK-3472: ------------------------------------------------------------- Commit ec3302afd6c1676f766b6e731c101eaf316cd7cf in branch refs/heads/master from [~minchen07] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=ec3302a ] CLOUDSTACK-3472: [Object_Store_Refactor] System VMs are not coming up in initial attempt, but they are coming up after multiple attempts. > [Object_Store_Refactor] System VMs are not coming up in initial attempt, but > they are coming up after multiple attempts > ----------------------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-3472 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3472 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Install and Setup > Affects Versions: 4.2.0 > Environment: Latest build from ACS4.2 branch: > Cluster:XenServer6.1 > Reporter: Sanjeev N > Assignee: Min Chen > Priority: Critical > Fix For: 4.2.0 > > Attachments: management-server.rar > > > System VMs are not coming up in initial attempt, but they are coming up after > multiple attempts. > Failed to create VDI in SR initially but succeeding after several failed > attempts. > Copycommand is failing with Runtime exception: > 2013-07-11 11:25:17,554 DEBUG [agent.transport.Request] (DirectAgent-73:null) > Seq 1-557449293: Processing: { Ans: , MgmtId: 6615759585382, via: 1, Ver: > v1, Flags: 110, > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Catch > Exception com.cloud.utils.exception.CloudRuntimeException for template + > due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in > sr f3429124-745a-70d1-da86-ce97bfb2570d","wait":0}}] } > sr-list on hypervisor: > ================ > [root@Rack1Pod1Host13 ~]# xe sr-list type=nfs > uuid ( RO) : f3429124-745a-70d1-da86-ce97bfb2570d > name-label ( RW): c65a038a-750c-3b4f-bf26-7ce3b74e1c85 > name-description ( RW): 1 > host ( RO): Rack1Pod1Host13 > type ( RO): nfs > content-type ( RO): user > Log snippet from management server log: > 2013-07-11 11:25:15,239 DEBUG [agent.transport.Request] (DirectAgent-15:null) > Seq 1-557449293: Executing: { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, > Flags: 100111, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/1/1/ddc39e82-13b4-4d30-9149-5ea6c1b801d5.vhd","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM > Template > (XenServer)","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_xen_os","_role":"ImageCache"}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM > Template > (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"wait":10800}}] > } > 2013-07-11 11:25:15,241 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-73:null) Seq 1-557449293: Executing request > 2013-07-11 11:25:16,175 DEBUG [agent.transport.Request] (secstorage-1:null) > Seq 1-557449302: Waiting for Seq 557449293 Scheduling: { Cmd , MgmtId: > 6615759585382, via: 1, Ver: v1, Flags: 100111, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"5b6d5297-e270-4d01-ad4c-f5a75d96134c","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM > Template > (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"a11bf731-9838-4f1a-9472-655086d2104d","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"ROOT-3","size":222905783,"volumeId":3,"vmName":"s-3-VM","accountId":1,"format":"VHD","id":3}},"wait":0}}] > } > 2013-07-11 11:25:17,551 WARN [xen.resource.XenServerStorageProcessor] > (DirectAgent-73:null) can not create vdi in sr > f3429124-745a-70d1-da86-ce97bfb2570d > 2013-07-11 11:25:17,552 WARN [xen.resource.XenServerStorageProcessor] > (DirectAgent-73:null) Catch Exception > com.cloud.utils.exception.CloudRuntimeException for template + due to > com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr > f3429124-745a-70d1-da86-ce97bfb2570d > com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr > f3429124-745a-70d1-da86-ce97bfb2570d > at > com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:792) > at > com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:863) > at > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) > at > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:628) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73) > at > com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:104) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-11 11:25:17,553 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-73:null) Seq 1-557449293: Response Received: > 2013-07-11 11:25:17,554 DEBUG [agent.transport.Request] (DirectAgent-73:null) > Seq 1-557449293: Processing: { Ans: , MgmtId: 6615759585382, via: 1, Ver: > v1, Flags: 110, > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Catch > Exception com.cloud.utils.exception.CloudRuntimeException for template + > due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in > sr f3429124-745a-70d1-da86-ce97bfb2570d","wait":0}}] } > 2013-07-11 11:25:17,555 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 1-557449293: Received: { Ans: , MgmtId: 6615759585382, via: 1, Ver: v1, > Flags: 110, { CopyCmdAnswer } } > 2013-07-11 11:25:17,558 DEBUG [storage.image.BaseImageStoreDriverImpl] > (consoleproxy-1:null) Unable to destoy TEMPLATE: 1 > com.cloud.utils.exception.CloudRuntimeException: not implemented yet > at > org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.select(DefaultEndPointSelector.java:230) > at > org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.select(DefaultEndPointSelector.java:216) > at > org.apache.cloudstack.storage.image.BaseImageStoreDriverImpl.deleteAsync(BaseImageStoreDriverImpl.java:189) > at > org.apache.cloudstack.storage.image.store.ImageStoreImpl.delete(ImageStoreImpl.java:148) > at > org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.deleteCacheObject(StorageCacheManagerImpl.java:258) > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:193) > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:351) > at > org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490) > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-11 11:25:17,559 DEBUG [agent.manager.AgentAttache] > (DirectAgent-73:null) Seq 1-557449302: Sending now. is current sequence. > 2013-07-11 11:25:17,563 DEBUG [agent.transport.Request] (DirectAgent-73:null) > Seq 1-557449302: Executing: { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, > Flags: 100111, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"5b6d5297-e270-4d01-ad4c-f5a75d96134c","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM > Template > (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"a11bf731-9838-4f1a-9472-655086d2104d","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"ROOT-3","size":222905783,"volumeId":3,"vmName":"s-3-VM","accountId":1,"format":"VHD","id":3}},"wait":0}}] > } > 2013-07-11 11:25:17,566 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-82:null) Seq 1-557449302: Executing request > 2013-07-11 11:25:17,602 DEBUG [storage.volume.VolumeServiceImpl] > (consoleproxy-1:null) failed to create template on storage > java.lang.RuntimeException: InvocationTargetException when invoking RPC > callback for command: copyBaseImageCallback > at > org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:148) > at > org.apache.cloudstack.framework.async.InplaceAsyncCallbackDriver.performCompletionCallback(InplaceAsyncCallbackDriver.java:26) > at > org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.complete(AsyncCallbackDispatcher.java:120) > at > org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:363) > at > org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490) > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:142) > ... 28 more > Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to update > state > at > org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:160) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.copyBaseImageCallback(VolumeServiceImpl.java:411) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ... 32 more > Caused by: com.cloud.utils.fsm.NoTransitionException: Unable to transition to > a new state from Ready via OperationFailed > at > com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:83) > at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:100) > at > org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:247) > at > org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:158) > ... 34 more > 2013-07-11 11:25:17,622 WARN > [storage.datastore.ObjectInDataStoreManagerImpl] (consoleproxy-1:null) > Template 1 is not found on storage pool 1, so no need to delete > 2013-07-11 11:25:17,623 ERROR [cloud.vm.VirtualMachineManagerImpl] > (consoleproxy-1:null) Failed to start instance VM[ConsoleProxy|v-2-VM] > com.cloud.utils.exception.CloudRuntimeException: can't find mapping in > ObjectInDataStore table for: > org.apache.cloudstack.storage.image.store.TemplateObject@111b863 > at > org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:228) > at > org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:158) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:393) > at > org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490) > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664) > at > com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-11 11:25:17,632 DEBUG [cloud.vm.VirtualMachineManagerImpl] > (consoleproxy-1:null) Cleaning up resources for the vm > VM[ConsoleProxy|v-2-VM] in Starting state > 2013-07-11 11:25:17,636 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 1-557449303: Sending { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, > Flags: 100011, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"v-2-VM","wait":0}}] > } > 2013-07-11 11:25:17,637 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 1-557449303: Executing: { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, > Flags: 100011, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"v-2-VM","wait":0}}] > } > Image store info from cloud DB: > mysql> select * from image_store; > +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+ > | id | name | image_provider_name > | protocol | url | > data_center_id | scope | role | uuid | > parent | created | removed | total_size | used_bytes | > +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+ > | 1 | nfs://10.147.28.7/export/home/sanjeev/sec_xen_os | NFS > | nfs | nfs://10.147.28.7/export/home/sanjeev/sec_xen_os | > 1 | ZONE | ImageCache | 8e43fa4b-cf0c-46c2-9e68-ead6bd481742 | NULL | > 2013-07-11 14:51:18 | NULL | NULL | NULL | > | 2 | obj_store | S3 > | http | NULL | > NULL | REGION | Image | 73729a9c-c221-4532-a5c2-745b834f6b3f | NULL | > 2013-07-11 14:51:18 | 2013-07-11 14:59:17 | NULL | NULL | > | 3 | objstore | S3 > | http | NULL | > NULL | REGION | Image | 0c444304-7e5f-4137-8da1-a812c3b81cf7 | NULL | > 2013-07-11 15:07:56 | NULL | NULL | NULL | > +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+ > 3 rows in set (0.03 sec) > mysql> select * from storage_pool; > +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+ > | id | name | uuid | pool_type > | port | data_center_id | pod_id | cluster_id | used_bytes | > capacity_bytes | host_address | user_info | path | > created | removed | update_time | status | storage_provider_name > | scope | hypervisor | managed | capacity_iops | > +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+ > | 1 | pri_xen_os | c65a038a-750c-3b4f-bf26-7ce3b74e1c85 | NetworkFilesystem > | 2049 | 1 | 1 | 1 | 2258215108608 | > 5902284816384 | 10.147.28.7 | NULL | /export/home/sanjeev/pri_xen_os | > 2013-07-11 14:51:08 | NULL | NULL | Up | DefaultPrimary > | CLUSTER | NULL | 0 | NULL | > +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+ > 1 row in set (0.00 sec) -- This message was sent by Atlassian JIRA (v6.1#6144)