[ https://issues.apache.org/jira/browse/CLOUDSTACK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Min Chen resolved CLOUDSTACK-4131. ---------------------------------- Resolution: Fixed > [VMWARE] ROOT volumes are not getting deleted when VM's (User/Sytem) are > failed to deploy/Start > ------------------------------------------------------------------------------------------------ > > Key: CLOUDSTACK-4131 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4131 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Storage Controller, VMware > Affects Versions: 4.2.0 > Reporter: Sailaja Mada > Assignee: Min Chen > Priority: Critical > Fix For: 4.2.0 > > Attachments: apilog.log, management-server.log, newdb.sql, > volsvmware.png > > > Setup : VMWARE ESXi servers > Steps: > 1. Configure Adv Zone with VMWARE Clutser > 2. There is a issue with public traffic label name so System VM's failed > start while creating Public port groups. (Its one of the scenario where > system VM's can fail to start) > 3. It started creating System VM's again and again and every time it failed > with same error > Observation: > 1. When ever there is failure to create instances we are deleting the ROOT > volumes , We are narking them as Destory state but Removed is not updated . > 2. These volumes are physically present in the primary storage and on the > ESXi server > 3. After having the actual issue solved also we will not be able to get the > system VM's running as it fails saying ROOT Volume already exists : > 2013-08-07 12:20:54,024 DEBUG [cloud.storage.VolumeManagerImpl] > (consoleproxy-1:null) Checking if we need to prepare 1 volumes for > VM[ConsoleProxy|v-2-VM] > 2013-08-07 12:20:54,049 DEBUG [storage.image.TemplateDataFactoryImpl] > (consoleproxy-1:null) template 8 is already in store:1, type:Image > 2013-08-07 12:20:54,059 DEBUG [storage.image.TemplateDataFactoryImpl] > (consoleproxy-1:null) template 8 is already in store:1, type:Primary > 2013-08-07 12:20:54,097 DEBUG [storage.motion.AncientDataMotionStrategy] > (consoleproxy-1:null) copyAsync inspecting src type TEMPLATE copyAsync > inspecting dest type VOLUME > 2013-08-07 12:20:54,119 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 2-1287389204: Sending { Cmd , MgmtId: 187767034175903, via: 2, Ver: v1, > Flags: 100011, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"1d8a03ea04a03db296074bcf3fb39c35","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova","uuid":"4a64b93e-ff20-11e2-a5c2-aac5eb5f619f","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM > Template > (vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a24ea7dc-cd9b-35bd-a7d4-9e8b4b8614c5","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/finalps1","port":2049}},"name":"routing-8","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"d8c4189f-fcaa-4942-84df-9c05f29fc408","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a24ea7dc-cd9b-35bd-a7d4-9e8b4b8614c5","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/finalps1","port":2049}},"name":"ROOT-2","size":0,"volumeId":15,"vmName":"v-2-VM","accountId":1,"format":"OVA","id":15,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}] > } > 2013-08-07 12:20:54,123 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 2-1287389204: Executing: { Cmd , MgmtId: 187767034175903, via: 2, Ver: > v1, Flags: 100011, > [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"1d8a03ea04a03db296074bcf3fb39c35","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova","uuid":"4a64b93e-ff20-11e2-a5c2-aac5eb5f619f","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM > Template > (vSphere)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a24ea7dc-cd9b-35bd-a7d4-9e8b4b8614c5","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/finalps1","port":2049}},"name":"routing-8","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"d8c4189f-fcaa-4942-84df-9c05f29fc408","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a24ea7dc-cd9b-35bd-a7d4-9e8b4b8614c5","id":1,"poolType":"NetworkFilesystem","host":"10.102.192.100","path":"/cpg_vol/sailaja/finalps1","port":2049}},"name":"ROOT-2","size":0,"volumeId":15,"vmName":"v-2-VM","accountId":1,"format":"OVA","id":15,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}] > } > 2013-08-07 12:20:54,125 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-11:null) Seq 2-1287389204: Executing request > 2013-08-07 12:20:54,297 INFO [storage.resource.VmwareStorageProcessor] > (DirectAgent-11:10.102.192.18) creating linked clone from template > 2013-08-07 12:20:55,378 ERROR [storage.resource.VmwareStorageProcessor] > (DirectAgent-11:10.102.192.18) CreateCommand failed due to Exception: > java.lang.RuntimeException > Message: The name 'ROOT-2' already exists. > java.lang.RuntimeException: The name 'ROOT-2' already exists. > at > com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:290) > at > com.cloud.hypervisor.vmware.mo.VirtualMachineMO.createLinkedClone(VirtualMachineMO.java:641) > at > com.cloud.storage.resource.VmwareStorageProcessor.createVMLinkedClone(VmwareStorageProcessor.java:251) > at > com.cloud.storage.resource.VmwareStorageProcessor.cloneVolumeFromBaseTemplate(VmwareStorageProcessor.java:368) > at > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:73) > at > com.cloud.storage.resource.VmwareStorageSubsystemCommandHandler.execute(VmwareStorageSubsystemCommandHandler.java:109) > at > com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:564) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-08-07 12:20:55,383 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-11:null) Seq 2-1287389204: Response Received: > 2013-08-07 12:20:55,383 DEBUG [agent.transport.Request] (DirectAgent-11:null) > Seq 2-1287389204: Processing: { Ans: , MgmtId: 187767034175903, via: 2, Ver: > v1, Flags: 10, > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"java.lang.RuntimeException: > The name 'ROOT-2' already exists.","wait":0}}] } > 2013-08-07 12:20:55,383 DEBUG [agent.transport.Request] (consoleproxy-1:null) > Seq 2-1287389204: Received: { Ans: , MgmtId: 187767034175903, via: 2, Ver: > v1, Flags: 10, { CopyCmdAnswer } } > 2013-08-07 12:20:55,425 WARN > [storage.datastore.ObjectInDataStoreManagerImpl] (consoleproxy-1:null) > Unsupported data object (VOLUME, > org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@37655fe4), no > need to delete from object in store ref table > 2013-08-07 12:20:55,426 DEBUG [cloud.storage.VolumeManagerImpl] > (consoleproxy-1:null) Unable to create > Vol[15|vm=2|ROOT]:java.lang.RuntimeException: The name 'ROOT-2' already > exists. > 2013-08-07 12:20:55,426 INFO [cloud.vm.VirtualMachineManagerImpl] > (consoleproxy-1:null) Unable to contact resource. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira