[ https://issues.apache.org/jira/browse/CLOUDSTACK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13759495#comment-13759495 ]
ASF subversion and git services commented on CLOUDSTACK-4430: ------------------------------------------------------------- Commit 25281ae7a7209ad71781b29802e97a25535e7be6 in branch refs/heads/master from [~minchen07] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=25281ae ] CLOUDSTACK-4430: Add retry logic back in case of template reload needed for vmware. > [Automation][vmware] Failed to deploy vm, if one host is down in a cluster > -------------------------------------------------------------------------- > > Key: CLOUDSTACK-4430 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4430 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Automation, Management Server, VMware > Affects Versions: 4.2.0 > Environment: Automation > vmware > Reporter: Rayees Namathponnan > Assignee: Min Chen > Priority: Critical > Fix For: 4.2.1 > > Attachments: management-server.log.2013-08-20.gz, > management-server.rar > > > This issue observed during automation run > 1) Automation running with 2 zones (advanced ), both zone contions one > cluster; first cluster having 2 hosts (10.223.250.130 and 10.223.250.131) > and another cluster have 1 host. > 2) Create vm one first zone; > 3) make one host (10.223.250.130) down from first zone > 4) deploy an another vm after the first zone is down > Expected Behavior > --------------------------- > New vm should be deployed with first zone, on the second host (10.223.250.131) > Actual Result > -------------------- > VM deployment failed with below error > 2013-08-21 15:49:49,125 DEBUG [cloud.storage.VolumeManagerImpl] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) > Checking if we need to prepare 1 volumes for VM[DomainRouter|r-524-TestVM] > 2013-08-21 15:49:49,131 DEBUG [storage.image.TemplateDataFactoryImpl] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) > template 8 is already in store:2, type:Image > 2013-08-21 15:49:49,155 DEBUG [storage.image.TemplateDataFactoryImpl] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) > template 8 is already in store:1, type:Primary > 2013-08-21 15:49:49,224 DEBUG [storage.motion.AncientDataMotionStrategy] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) > copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME > 2013-08-21 15:49:49,232 DEBUG [agent.transport.Request] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Seq > 3-1188243408: Sending { Cmd , MgmtId: 90928106758026, via: 3, Ver: v1, > Flags: 100011, [{"org.apac > he.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d559e79dc94f3ceea16e841774053435","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ova" > ,"uuid":"426a759e-09ec-11e3-a733-52b2d980df8a","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM > Template (vSphere)","imageDataStore":{"org.apache.cloudstack.st > orage.to.PrimaryDataStoreTO":{"uuid":"4faf04c2-6dd8-3025-b43f-65d32cc49d02","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary1","port":2049}},"name":"routing-8"," > hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"3c7b36c7-a034-485b-b808-501e6b8a067a","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid" > :"4faf04c2-6dd8-3025-b43f-65d32cc49d02","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary1","port":2049}},"name":"ROOT-524","size":2097152000,"volumeId":575,"vmNa > me":"r-524-TestVM","accountId":2,"format":"OVA","id":575,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}] > } > 2013-08-21 15:49:49,232 DEBUG [agent.transport.Request] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Seq > 3-1188243408: Executing: { Cmd , MgmtId: 90928106758026, via: 3, Ver: v1, > Flags: 100011, [{"org.a > pache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"d559e79dc94f3ceea16e841774053435","origUrl":"http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.o > va","uuid":"426a759e-09ec-11e3-a733-52b2d980df8a","id":8,"format":"OVA","accountId":1,"checksum":"8fde62b1089e5844a9cd3b9b953f9596","hvm":false,"displayText":"SystemVM > Template (vSphere)","imageDataStore":{"org.apache.cloudstack > .storage.to.PrimaryDataStoreTO":{"uuid":"4faf04c2-6dd8-3025-b43f-65d32cc49d02","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary1","port":2049}},"name":"routing-8 > ","hypervisorType":"VMware"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"3c7b36c7-a034-485b-b808-501e6b8a067a","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uu > id":"4faf04c2-6dd8-3025-b43f-65d32cc49d02","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/automation/SC-CLOUD-QA03/primary1","port":2049}},"name":"ROOT-524","size":2097152000,"volumeId":575,"v > mName":"r-524-TestVM","accountId":2,"format":"OVA","id":575,"hypervisorType":"None"}},"executeInSequence":false,"wait":0}}] > } > 2013-08-21 15:49:49,232 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-120:null) Seq 3-1188243408: Executing request > 2013-08-21 15:49:49,291 WARN [storage.resource.VmwareStorageProcessor] > (DirectAgent-120:10.223.250.131) Template host in vSphere is not in connected > state, request template reload > 2013-08-21 15:49:49,291 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-120:null) Seq 3-1188243408: Response Received: > 2013-08-21 15:49:49,291 DEBUG [agent.transport.Request] > (DirectAgent-120:null) Seq 3-1188243408: Processing: { Ans: , MgmtId: > 90928106758026, via: 3, Ver: v1, Flags: 10, > [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{ > "result":false,"details":"Template host in vSphere is not in connected state, > request template reload","wait":0}}] } > 2013-08-21 15:49:49,292 DEBUG [agent.transport.Request] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Seq > 3-1188243408: Received: { Ans: , MgmtId: 90928106758026, via: 3, Ver: v1, > Flags: 10, { CopyCmdAns > wer } } > 2013-08-21 15:49:49,297 WARN > [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-29:job-1549 = > [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Unsupported data object (VOLUME, > org.apache.cloudstack.storage.datastore.Primar > yDataStoreImpl@c3b35a4), no need to delete from object in store ref table > 2013-08-21 15:49:49,306 DEBUG [cloud.storage.VolumeManagerImpl] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Unable > to create Vol[575|vm=524|ROOT]:Template host in vSphere is not in connected > state, requ > est template reload > 2013-08-21 15:49:49,306 INFO [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-29:job-1549 = [ cf900a2a-6c83-4ddd-b898-e19c90cef0a6 ]) Unable > to contact resource. > com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is > unreachable: Unable to create Vol[575|vm=524|ROOT]:Template host in vSphere > is not in connected state, request template reload > at > com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2534) > at > com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2582) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:885) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:575) > In Vcenter i can see below template created for first host; and i these > templates are disconnected state; but above command we are trying to deploy > router with template d559e79dc94f3ceea16e841774053435 to running host > 10.223.250.131 > d559e79dc94f3ceea16e841774053435 > ff7df1a2e65d380f9da8edc17429a3f6 > 9b67e18994663183b7c41d81dd42d57f > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira