[ https://issues.apache.org/jira/browse/CLOUDSTACK-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edison su resolved CLOUDSTACK-3538. ----------------------------------- Resolution: Fixed > [Automation]Router and SSVM not coming up after restart > -------------------------------------------------------- > > Key: CLOUDSTACK-3538 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3538 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Affects Versions: 4.2.0 > Environment: KVM > Build 4.2 > Reporter: Rayees Namathponnan > Assignee: edison su > Priority: Critical > Fix For: 4.2.0 > > Attachments: CLOUDSTACK-3538_Debug.rar, CLOUDSTACK-3538.rar > > > Below BVT test cases failed in latest 4.2 run > integration.smoke.test_network.TestRebootRouter.test_reboot_router > integration.smoke.test_network.TestRebootRouter.test_reboot_router > integration.smoke.test_routers.TestRouterServices.test_08_start_router > integration.smoke.test_routers.TestRouterServices.test_09_reboot_router > integration.smoke.test_ssvm.TestSSVMs.test_07_reboot_ssvm > All test cases failed to reboot router or SSVM, also observed cloudstack > taking more than 20 minute to create new router , > Please see the attached log after restart router "r-133-QA" > ----------------------------------------------------------------------------------- > 1) Issued stop command @ 11:42:02 > 2013-07-15 11:42:02,765 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) Execution is successful. > 2013-07-15 11:42:02,765 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) Try to stop the vm at first > 2013-07-15 11:42:10,777 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-1:null) successfully shut down vm r-133-QA > 2013-07-15 11:42:52,454 DEBUG [kvm.resource.LibvirtComputingResource] > (UgentTask-5:null) Ignoring VM r-133-QA in transition state stopping. > 2013-07-15 11:42:52,455 DEBUG [kvm.resource.LibvirtComputingResource] > (UgentTask-5:null) Executing: > /usr/share/cloudstack-common/scripts/vm/network/security_group.py > get_rule_logs_for_vms > 2013-07-15 11:42:52,759 DEBUG [kvm.resource.LibvirtComputingResource] > (UgentTask-5:null) Execution is successful. > 2) Started Command issue after 15 minute @ 11:58:03 > 2013-07-15 11:58:03,238 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Executing: > /usr/share/cloudstack-common/scripts/network/domr/router_proxy.sh netusage.sh > 169.254.0.187 -c > 2013-07-15 11:58:03,291 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-5:null) Request:Seq 5-1532821715: { Cmd , MgmtId: > 29066118877352, via: 5, Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.StartCommand":{"vm":{"id":133,"name":"r-133-QA","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian > GNU/Linux 5.0 (32-bit)","bootArgs":" template=domP name=r-133-QA > eth2ip=10.223.122.92 eth2mask=255.255.255.192 gateway=10.223.122.65 > eth0ip=10.1.1.1 eth0mask=255.255.255.0 domain=cs43auto.advanced > dhcprange=10.1.1.1 eth1ip=169.254.1.214 eth1mask=255.255.0.0 type=router > disable_rp_filter=true > dns1=8.8.8.8","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"ee61a6681d35c765","params":{},"uuid":"240b6249-e20f-4c35-93b4-024303bef80a","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"5c91d846-6310-4440-aa1d-67b35f2d2c36","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"fff90cb5-06dd-33b3-8815-d78c08ca01d9","id":1,"poolType":"NetworkFilesystem","host":"10.223.110.232","path":"/export/home/rayees/SC_QA_AUTO4/primary","port":2049}},"name":"ROOT-133","size":147456,"path":"d4476933-b42c-45f4-a061-22c1908edca5","volumeId":138,"vmName":"r-133-QA","accountId":67,"format":"QCOW2","id":138}},"diskSeq":0,"type":"ROOT"}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"1b55ad5b-8516-4e79-90a4-33052c43b0e0","ip":"10.223.122.92","netmask":"255.255.255.192","gateway":"10.223.122.65","mac":"06:a0:90:00:00:55","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1221","isolationUri":"vlan://1221","isSecurityGroupEnabled":false},{"deviceId":0,"networkRateMbps":200,"defaultNic":false,"uuid":"859700c2-124b-4183-a64c-8357f57a7671","ip":"10.1.1.1","netmask":"255.255.255.0","mac":"02:00:21:a1:00:02","dns1":"8.8.8.8","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://2311","isolationUri":"vlan://2311","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"4ce6c02e-6df6-4960-857c-3e5fdd2d8e1b","ip":"169.254.1.214","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:d6","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"hostIp":"10.223.50.67","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.214","port":3922,"interval":6,"retries":100,"name":"r-133-QA","wait":0}},{"com.cloud.agent.api.GetDomRVersionCmd":{"accessDetails":{"router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}},{},{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":67,"publicIp":"10.223.122.92","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"1221","vlanGateway":"10.223.122.65","vlanNetmask":"255.255.255.192","vifMacAddress":"06:92:84:00:00:55","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}},{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"02:00:39:b4:00:01","vmIpAddress":"10.1.1.120","vmName":"dVM","defaultRouter":"10.1.1.1","defaultDns":"10.1.1.1","duid":"00:03:00:01:02:00:39:b4:00:01","isDefault":true,"executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.1.214","router.name":"r-133-QA"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.120","vmName":"dVM","executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":"r-133-QA","router.ip":"169.254.1.214"},"wait":0}}] > } > 2013-07-15 11:58:03,291 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-5:null) Processing command: > com.cloud.agent.api.StartCommand > 2013-07-15 11:58:03,363 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Execution is successful. > 3) r-133-QA still in starting state in UI , even after 20 minute > See the "virsh list" from the host, r-133-QA is missing > [root@Rack2Host12 agent]# virsh list > Id Name State > ---------------------------------------------------- > 3 r-130-QA running > 5 r-128-QA running > 6 r-126-QA running > 7 i-65-129-QA running > 10 i-71-139-QA running > 11 r-57-QA running > 13 r-144-QA running > Also below refer below check-in this might be caused this > https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=patch;h=22c6df0ba25f1f1fd3cf489a29ed1f4f4097c89c -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira