[
https://issues.apache.org/jira/browse/CLOUDSTACK-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619655#comment-13619655
]
Mice Xia commented on CLOUDSTACK-1687:
--------------------------------------
Cannot reproduce it in my env, Chandan, can you always reproduce it?
> VMSnapshot: VM Fails to Start after a RevertToVMSnapshot Operation
> ------------------------------------------------------------------
>
> Key: CLOUDSTACK-1687
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1687
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Reporter: Chandan Purushothama
> Assignee: Mice Xia
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: management-server.zip
>
>
> ========================
> Steps to Reproduce the Bug:
> ========================
> 1. Deploy Vm. Attach muiltiple data disks to this VM.
> 2. Log in to the Vm and create few directories and files on the root volume.
> Create few more directories on each of the data disks.
> 3. Stop the VM.
> 4. Create VMSnapshot for this VM with "vm_snapshot_type" set to "Disk"
> 5. Start the VM.
> 6. Delete few of the above created directories and add few more directories
> on the root volume.Delete few of the above created directories and add few
> more directories on the data disks.
> 7. Stop the VM.
> 8. Revert this Vm to the above created Snapshot.
> 9. Start the VM.
> ===========
> Observations:
> ===========
> *Management Server Log*
> 2013-03-14 16:06:15,703 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Executing resource StartCommand:
> {"vm":{"id":9,"name":"i-3-9-VMSNAP","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
> 5.3
> (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"ec119fd35127c44e","params":{"rootDiskController":"ide","nicAdapter":"E1000"},"uuid":"1c64977b-db4f-415e-98ce-83b53a612493","disks":[{"id":14,"name":"ROOT-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"ROOT-9-14-000001","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":0},{"id":15,"name":"DATA-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"c7adbf5e2509484d8ac4959ae53e6146-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":1},{"id":16,"name":"NEWDISK-3","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"fba51dce35d24c19a224090f3cb597e8-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":2},{"id":9,"name":"CentOS
> 5.3(64-bit) no GUI
> (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}],"nics":[{"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}]},"hostIp":"10.223.58.2","wait":0}
> 2013-03-14 16:06:15,703 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) VM i-3-9-VMSNAP will be started with NIC device
> type: E1000
> 2013-03-14 16:06:15,703 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Drop invalid disk option, volumeTO:
> {"id":9,"name":"CentOS 5.3(64-bit) no GUI
> (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}
> 2013-03-14 16:06:15,723 DEBUG [vmware.mo.HostMO]
> (DirectAgent-242:10.223.58.2) find VM i-3-9-VMSNAP on host
> 2013-03-14 16:06:15,723 DEBUG [vmware.mo.HostMO]
> (DirectAgent-242:10.223.58.2) load VM cache on host
> 2013-03-14 16:06:15,728 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) VM i-3-9-VMSNAP already exists, tear down
> devices for reconfiguration
> 2013-03-14 16:06:15,750 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare ISO volume at existing device
> {"key":3000,"deviceInfo":{"label":"CD/DVD drive 1","summary":"Remote
> device"},"backing":{"exclusive":false,"deviceName":""},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":0}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare volume at new device
> {"capacityInKB":0,"key":-2,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
>
> ROOT-9-14-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":200,"unitNumber":1}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare volume at new device
> {"capacityInKB":0,"key":-3,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
>
> c7adbf5e2509484d8ac4959ae53e6146-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":2}
> 2013-03-14 16:06:15,753 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare volume at new device
> {"capacityInKB":0,"key":-4,"backing":{"diskMode":"persistent","fileName":"[f42a60c20940305bbb1423d8b40f42bc]
>
> fba51dce35d24c19a224090f3cb597e8-000001.vmdk","datastore":{"value":"datastore-22919","type":"Datastore"}},"connectable":{"startConnected":true,"allowGuestControl":false,"connected":true},"controllerKey":1000,"unitNumber":3}
> 2013-03-14 16:06:15,753 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare NIC device based on NicTO:
> {"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}
> 2013-03-14 16:06:15,757 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare network on vmwaresvs
> P[vSwitch0:untagged] with name prefix: cloud.guest
> 2013-03-14 16:06:15,841 INFO [vmware.mo.HypervisorHostHelper]
> (DirectAgent-242:10.223.58.2) Network cloud.guest.2254.495.1-vSwitch0 is
> ready on vSwitch vSwitch0
> 2013-03-14 16:06:15,841 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Preparing NIC device on network
> cloud.guest.2254.495.1-vSwitch0
> 2013-03-14 16:06:15,841 DEBUG [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Prepare NIC at new device
> {"operation":"ADD","device":{"addressType":"Manual","macAddress":"02:00:48:7a:00:07","key":-5,"backing":{"network":{"value":"network-22944","type":"Network"},"deviceName":"cloud.guest.2254.495.1-vSwitch0"},"connectable":{"startConnected":true,"allowGuestControl":true,"connected":true},"unitNumber":4}}
> 2013-03-14 16:06:15,841 DEBUG [vmware.mo.HostMO]
> (DirectAgent-242:10.223.58.2) find VM i-3-9-VMSNAP on host
> 2013-03-14 16:06:15,841 DEBUG [vmware.mo.HostMO]
> (DirectAgent-242:10.223.58.2) VM i-3-9-VMSNAP found in host cache
> 2013-03-14 16:06:15,869 INFO [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) Configure VNC port for VM i-3-9-VMSNAP, port:
> 5914, host: 10.223.58.2
> 2013-03-14 16:06:15,923 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 2 routers to update status.
> 2013-03-14 16:06:15,924 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 0 networks to update RvR status.
> 2013-03-14 16:06:16,054 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 2 routers to update status.
> 2013-03-14 16:06:16,055 DEBUG
> [network.router.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:null) Found 0 networks to update RvR status.
> 2013-03-14 16:06:16,066 WARN [vmware.resource.VmwareResource]
> (DirectAgent-242:10.223.58.2) StartCommand failed due to Exception:
> java.lang.RuntimeException
> Message: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not
> found
> java.lang.RuntimeException: File
> []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not found
> at
> com.cloud.hypervisor.vmware.util.VmwareClient.waitForTask(VmwareClient.java:291)
> at
> com.cloud.hypervisor.vmware.mo.VirtualMachineMO.configureVm(VirtualMachineMO.java:806)
> at
> com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2345)
> at
> com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:432)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-03-14 16:06:16,067 DEBUG [agent.manager.DirectAgentAttache]
> (DirectAgent-242:null) Seq 1-970065218: Response Received:
> 2013-03-14 16:06:16,068 DEBUG [agent.transport.Request]
> (DirectAgent-242:null) Seq 1-970065218: Processing: { Ans: , MgmtId:
> 7200344900649, via: 1, Ver: v1, Flags: 110,
> [{"StartAnswer":{"vm":{"id":9,"name":"i-3-9-VMSNAP","bootloader":"HVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
> 5.3
> (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"ec119fd35127c44e","params":{"rootDiskController":"ide","nicAdapter":"E1000"},"uuid":"1c64977b-db4f-415e-98ce-83b53a612493","disks":[{"id":14,"name":"ROOT-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"ROOT-9-14-000001","size":2147483648,"type":"ROOT","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":0},{"id":15,"name":"DATA-9","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"c7adbf5e2509484d8ac4959ae53e6146-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":1},{"id":16,"name":"NEWDISK-3","mountPoint":"/export/home/chandan/asf4-131-170/primary","path":"fba51dce35d24c19a224090f3cb597e8-000001","size":5368709120,"type":"DATADISK","storagePoolType":"NetworkFilesystem","storagePoolUuid":"f42a60c2-0940-305b-bb14-23d8b40f42bc","deviceId":2},{"id":9,"name":"CentOS
> 5.3(64-bit) no GUI
> (vSphere)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}],"nics":[{"deviceId":0,"networkRateMbps":495,"defaultNic":true,"uuid":"06e76915-e16d-4f60-a311-3a5fbdd7d5ab","ip":"10.1.1.66","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:48:7a:00:07","dns1":"8.8.8.8","dns2":"8.8.4.4","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2254","isolationUri":"vlan://2254","isSecurityGroupEnabled":false}]},"result":false,"details":"StartCommand
> failed due to Exception: java.lang.RuntimeException\nMessage: File
> []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not
> found\n","wait":0}}] }
> 2013-03-14 16:06:16,068 DEBUG [agent.transport.Request]
> (Job-Executor-87:job-93) Seq 1-970065218: Received: { Ans: , MgmtId:
> 7200344900649, via: 1, Ver: v1, Flags: 110, { StartAnswer } }
> 2013-03-14 16:06:16,070 DEBUG [agent.manager.AgentAttache]
> (DirectAgent-242:null) Seq 1-970065218: No more commands found
> 2013-03-14 16:06:16,073 INFO [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-87:job-93) Unable to start VM on Host[-1-Routing] due to
> StartCommand failed due to Exception: java.lang.RuntimeException
> Message: File []/vmfs/volumes/23dd55b9-15861742/ROOT-9-14-000001.vmdk was not
> found
> 2013-03-14 16:06:16,079 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-87:job-93) Cleaning up resources for the vm VM[User|boron-VM-6]
> in Starting state
> 2013-03-14 16:06:16,081 DEBUG [agent.transport.Request]
> (Job-Executor-87:job-93) Seq 1-970065219: Sending { Cmd , MgmtId:
> 7200344900649, via: 1, Ver: v1, Flags: 100111,
> [{"StopCommand":{"isProxy":false,"vmName":"i-3-9-VMSNAP","wait":0}}] }
> 2013-03-14 16:06:16,081 DEBUG [agent.transport.Request]
> (Job-Executor-87:job-93) Seq 1-970065219: Executing: { Cmd , MgmtId:
> 7200344900649, via: 1, Ver: v1, Flags: 100111,
> [{"StopCommand":{"isProxy":false,"vmName":"i-3-9-VMSNAP","wait":0}}] }
> **On the primary Storage, the ROOT Volume is indeed missing**
> [root@nfs2 primary]# ls
> 08c2826505b646b3bf3d3e03dfc2bce2-000001-delta.vmdk
> fba51dce35d24c19a224090f3cb597e8.vmdk ROOT-11-18.vmdk
> ROOT-7-11.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2-000001.vmdk i-3-3-VMSNAP
> ROOT-1-1-delta.vmdk ROOT-8-12-000001-delta.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2-flat.vmdk i-3-5-VMSNAP
> ROOT-1-1.vmdk ROOT-8-12-000001.vmdk
> 08c2826505b646b3bf3d3e03dfc2bce2.vmdk i-3-6-VMSNAP
> ROOT-2-2-delta.vmdk ROOT-8-12-delta.vmdk
> 8c26613e6e5f344b928e0530ec4817c3 i-3-7-VMSNAP
> ROOT-2-2.vmdk ROOT-8-12.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-000002-delta.vmdk i-3-8-VMSNAP
> ROOT-3-3-delta.vmdk ROOT-9-14-000002-delta.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-000002.vmdk i-3-9-VMSNAP
> ROOT-4-5-delta.vmdk ROOT-9-14-000002.vmdk
> c7adbf5e2509484d8ac4959ae53e6146-flat.vmdk i-6-10-VMSNAP
> ROOT-4-5.vmdk ROOT-9-14-delta.vmdk
> c7adbf5e2509484d8ac4959ae53e6146.vmdk r-11-VMSNAP
> ROOT-5-6-delta.vmdk ROOT-9-14.vmdk
> cf04b6e2026f3369b893bdab955c3ff0 r-4-VMSNAP
> ROOT-6-7-delta.vmdk s-1-VMSNAP
> fba51dce35d24c19a224090f3cb597e8-000002-delta.vmdk ROOT-10-17-delta.vmdk
> ROOT-7-11-000001-delta.vmdk v-2-VMSNAP
> fba51dce35d24c19a224090f3cb597e8-000002.vmdk ROOT-10-17.vmdk
> ROOT-7-11-000001.vmdk
> fba51dce35d24c19a224090f3cb597e8-flat.vmdk ROOT-11-18-delta.vmdk
> ROOT-7-11-delta.vmdk
> [root@nfs2 primary]#
> ==========
> Bug Impact:
> ==========
> RevertToVMSnapshot Operations are blocked.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
