[
https://issues.apache.org/jira/browse/CLOUDSTACK-4456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13748030#comment-13748030
]
ASF subversion and git services commented on CLOUDSTACK-4456:
-------------------------------------------------------------
Commit a49d39fe4d7555797e6753680f7e2d4723c6a3fd in branch refs/heads/4.2 from
[~edison]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a49d39f ]
CLOUDSTACK-4456:
CLOUDSTACK-4457:
CLOUDSTACK-4459:
harden kvm getvolume. It's possible that one volume created on other kvm host,
won't show up on another host, try more times to refresh storage pool if volume
won't shown up
(cherry picked from commit a0de0d0177a9251756d57ebdbea93f51a597ac1f)
Signed-off-by: animesh <anim...@apache.org>
> [Automation] Vm deployment from template is failed; due to some race
> condition in KVM
> -------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4456
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4456
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Automation, KVM, Management Server, Template
> Affects Versions: 4.2.0
> Environment: Automation
> 4.2
> Reporter: Rayees Namathponnan
> Assignee: edison su
> Priority: Critical
> Fix For: 4.2.1
>
> Attachments: CLOUDSTACK-4456.rar
>
>
> This issue observed during automation run, In attached log i can see multiple
> deployment failed due to same issue, here the once effected test case
> integration.component.test_blocker_bugs.TestTemplate.test_01_create_template
> Test case performing below operation
> 1) Create template from http url (ubuntu-10-04-64bit-server.qcow2)
> 2) Deploy vm from this template
> Actual Result
> ---------------------
> Deployment failed with below error; look like MS deleting template after
> creating the volume
> IN MS Search for job-899
> 2013-08-21 22:07:12,383 DEBUG [cloud.network.NetworkManagerImpl]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Asking
> BareMetalUserdata to prepare for
> Nic[194-188-97ae23d5-67ea-46c1-91e6-4e6ef35c22ae-10.223.250.231]
> 2013-08-21 22:07:12,389 DEBUG [cloud.storage.VolumeManagerImpl]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Checking
> if we need to prepare 1 volumes for
> VM[User|daf71095-b010-45f0-9e46-3bf51ee6a59c]
> 2013-08-21 22:07:12,389 DEBUG [cloud.storage.VolumeManagerImpl]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) No need
> to recreate the volume: Vol[238|vm=188|ROOT], since it already has a pool
> assigned: 1, adding disk to VM
> 2013-08-21 22:07:12,408 DEBUG [agent.transport.Request]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Seq
> 2-949159112: Sending { Cmd , MgmtId: 73187150500751, via: 2, Ver: v1, Flags:
> 100011,
> [{"com.cloud.agent.api.StartCommand":{"vm":{"id":188,"name":"i-287-188-TestVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
> 5.5
> (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"a79bd4d41d554b1a","params":{"Message.ReservedCapacityFreed.Flag":"false"},"uuid":"daf71095-b010-45f0-9e46-3bf51ee6a59c","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"eba8fc74-aa6e-43a9-8dae-5501a0dd490f","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9d444441-b138-3a13-974c-fa506683da4d","id":1,"poolType":"NetworkFilesystem","host":"nfs1-ccp.citrix.com","path":"/home/common/automation/SC_QA_AUTO5/primary1","port":2049}},"name":"ROOT-188","size":5368709120,"path":"5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635","volumeId":238,"vmName":"i-287-188-TestVM","accountId":287,"format":"QCOW2","id":238,"hypervisorType":"KVM"}},"diskSeq":0,"type":"ROOT"},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"3cf9efdc-212d-46f6-b818-071450e602fd","ip":"10.223.250.231","netmask":"255.255.255.192","gateway":"10.223.250.193","mac":"06:5f:a0:00:00:24","dns1":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":true}]},"hostIp":"10.223.250.195","executeInSequence":false,"wait":0}}]
> }
> 2013-08-21 22:07:12,957 DEBUG [agent.transport.Request]
> (AgentManager-Handler-7:null) Seq 2-949159112: Processing: { Ans: , MgmtId:
> 73187150500751, via: 2, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":188,"name":"i-287-188-TestVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
> 5.5
> (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"a79bd4d41d554b1a","vncAddr":"10.223.250.195","params":{"Message.ReservedCapacityFreed.Flag":"false"},"uuid":"daf71095-b010-45f0-9e46-3bf51ee6a59c","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"eba8fc74-aa6e-43a9-8dae-5501a0dd490f","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9d444441-b138-3a13-974c-fa506683da4d","id":1,"poolType":"NetworkFilesystem","host":"nfs1-ccp.citrix.com","path":"/home/common/automation/SC_QA_AUTO5/primary1","port":2049}},"name":"ROOT-188","size":5368709120,"path":"5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635","volumeId":238,"vmName":"i-287-188-TestVM","accountId":287,"format":"QCOW2","id":238,"hypervisorType":"KVM"}},"diskSeq":0,"type":"ROOT"},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"3cf9efdc-212d-46f6-b818-071450e602fd","ip":"10.223.250.231","netmask":"255.255.255.192","gateway":"10.223.250.193","mac":"06:5f:a0:00:00:24","dns1":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":true}]},"result":false,"details":"internal
> error Process exited while reading console log output: char device
> redirected to /dev/pts/3\nqemu-kvm: -drive
> file=/mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635,if=none,id=drive-virtio-disk0,format=qcow2,cache=none:
> could not open disk image
> /mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635:
> Operation not supported\n","wait":0}}] }
> 2013-08-21 22:07:12,957 DEBUG [agent.transport.Request]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Seq
> 2-949159112: Received: { Ans: , MgmtId: 73187150500751, via: 2, Ver: v1,
> Flags: 10, { StartAnswer } }
> 2013-08-21 22:07:12,960 INFO [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Unable
> to start VM on Host[-2-Routing] due to internal error Process exited while
> reading console log output: char device redirected to /dev/pts/3
> qemu-kvm: -drive
> file=/mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635,if=none,id=drive-virtio-disk0,format=qcow2,cache=none:
> could not open disk image
> /mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635:
> Operation not supported
> 2013-08-21 22:07:12,963 DEBUG [cloud.vm.VirtualMachineManagerImpl]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Cleaning
> up resources for the vm VM[User|daf71095-b010-45f0-9e46-3bf51ee6a59c] in
> Starting state
> 2013-08-21 22:07:12,964 DEBUG [agent.transport.Request]
> (Job-Executor-66:job-899 = [ 461fc03f-6d0a-488f-9cf3-7cf1a537b26c ]) Seq
> 2-949159113: Sending { Cmd , MgmtId: 73187150500751, via: 2, Ver: v1, Flags:
> 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-287-188-TestVM","wait":0}}]
> }
> 2013-08-21 22:07:13,253 DEBUG [agent.transport.Request]
> (AgentManager-Handler-4:null) Seq 2-949159113: Processing: { Ans: , MgmtId:
> 73187150500751, via: 2, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"wait":0}}] }
> Agent log
> --------------
> <on_reboot>restart</on_reboot>
> <on_poweroff>destroy</on_poweroff>
> <on_crash>destroy</on_crash>
> </domain>
> 2013-08-21 22:33:40,793 WARN [kvm.resource.LibvirtComputingResource]
> (agentRequest-Handler-1:null) LibvirtException
> org.libvirt.LibvirtException: internal error Process exited while reading
> console log output: char device redirected to /dev/pts/3
> qemu-kvm: -drive
> file=/mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635,if=none,id=drive-virtio-disk0,format=qcow2,cache=none:
> could not open disk image
> /mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635:
> Operation not supported
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.Connect.processError(Unknown Source)
> at org.libvirt.Connect.domainCreateXML(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startVM(LibvirtComputingResource.java:1156)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3524)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1252)
> at com.cloud.agent.Agent.processRequest(Agent.java:525)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> 2013-08-21 22:33:40,796 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Seq 2-949159112: { Ans: , MgmtId:
> 73187150500751, via: 2, Ver: v1, Flags: 10,
> [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":188,"name":"i-287-188-TestVM","type":"User","cpus":1,"minSpeed":100,"maxSpeed":100,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"CentOS
> 5.5
> (64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"a79bd4d41d554b1a","vncAddr":"10.223.250.195","params":{"Message.ReservedCapacityFreed.Flag":"false"},"uuid":"daf71095-b010-45f0-9e46-3bf51ee6a59c","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"eba8fc74-aa6e-43a9-8dae-5501a0dd490f","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"9d444441-b138-3a13-974c-fa506683da4d","id":1,"poolType":"NetworkFilesystem","host":"nfs1-ccp.citrix.com","path":"/home/common/automation/SC_QA_AUTO5/primary1","port":2049}},"name":"ROOT-188","size":5368709120,"path":"5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635","volumeId":238,"vmName":"i-287-188-TestVM","accountId":287,"format":"QCOW2","id":238,"hypervisorType":"KVM"}},"diskSeq":0,"type":"ROOT"},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"id":0,"format":"ISO","accountId":0,"hvm":false}},"diskSeq":3,"type":"ISO"}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"uuid":"3cf9efdc-212d-46f6-b818-071450e602fd","ip":"10.223.250.231","netmask":"255.255.255.192","gateway":"10.223.250.193","mac":"06:5f:a0:00:00:24","dns1":"8.8.8.8","broadcastType":"Native","type":"Guest","broadcastUri":"vlan://untagged","isolationUri":"ec2://untagged","isSecurityGroupEnabled":true}]},"result":false,"details":"internal
> error Process exited while reading console log output: char device
> redirected to /dev/pts/3\nqemu-kvm: -drive
> file=/mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635,if=none,id=drive-virtio-disk0,format=qcow2,cache=none:
> could not open disk image
> /mnt/9d444441-b138-3a13-974c-fa506683da4d/5ce3b7d0-07c1-4d3b-9c00-2c6ad8716635:
> Operation not supported\n","wait":0}}] }
> 2013-08-21 22:33:40,811 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) Request:Seq 2-949159113: { Cmd , MgmtId:
> 73187150500751, via: 2, Ver: v1, Flags: 100011,
> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-287-188-TestVM","wait":0}}]
> }
> 2013-08-21 22:33:40,811 DEBUG [cloud.agent.Agent]
> (agentRequest-Handler-5:null) Processing command:
> com.cloud.agent.api.StopCommand
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
