Hi,
Could anyone please help with the following issue I am seeing?
I am running CloudStack 4.3 and XenServer 6.2 (SP1)
I have removed a host which was working normally and re-added it (it is Host 15
in the log appended).
The communication with the host seems to be established as shown in the GUI
(State is "Up").
However, every attempt to start a Virtual Router on that host is failing. The
reasons that seem to be related to the issue are (as seen in the appended log):
"Unable to get the template/scripts version of router r-187-VM due to:
getDomRVersionCmd failed"
"com.cloud.exception.AgentUnavailableException: Resource [Host:15] is
unreachable"
How can I further debug this ? How can I check the state of the agent running
on XenServer ?
Thank you for your help,
Florin
Debug Log
---------------
2014-06-10 14:14:10,549 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-292:ctx-fb46119b) Seq 15-1518796837: Response Received:
2014-06-10 14:14:10,551 DEBUG [c.c.a.t.Request] (DirectAgent-292:ctx-fb46119b)
Seq 15-1518796837: Processing: { Ans: , MgmtId: 123561959226, via: 15, Ver:
v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{
"vm":{"id":187,"name":"r-187-VM","bootloader":"PyGrub","type":"DomainRouter","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":134217728,"maxRam":134217728,"arch":"x86_64","os":"Debian
GNU/Linux 7(32-bit)","boo
tArgs":" template=domP name=r-187-VM eth2ip=192.168.18.22
eth2mask=255.255.252.0 gateway=192.168.16.1 eth0ip=10.10.2.1
eth0mask=255.255.255.0 domain=cs2cloud.internal dhcprange=10.10.2.1
eth1ip=169.254.2.254
eth1mask=255.255.0.0 type=router disable_rp_filter=true dns1=192.168.20.18
dns2=192.168.20.81","rebootOnCrash":false,"enableHA":true,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"128a547
57b1466cc","params":{},"uuid":"035c06a8-8b08-46e7-88ef-ac128b1e2aeb","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"9c8b32ff-8864-4681-acb3-3a76b0c04d91","volumeType":"ROOT","dat
aStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"d9242b8f-5342-3225-bda3-8b79202af571","id":19,"poolType":"IscsiLUN","host":"10.20.1.12","path":"/iqn.2014-03.local.intune:cloudstorsrv3.
target1/1","port":3260,"url":"IscsiLUN://10.20.1.12//iqn.2014-03.local.intune:cloudstorsrv3.target1/1/?ROLE=Primary&STOREUUID=d9242b8f-5342-3225-bda3-8b79202af571"}},"name":"ROOT-187","size":2621440000,"path"
:"05b0fb51-f725-46e5-8af0-43884bfd6edf","volumeId":228,"vmName":"r-187-VM","accountId":2,"format":"VHD","id":228,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"05b0fb51-f725-46e5-8af0-43884bf
d6edf","type":"ROOT","_details":{"managed":"false","storagePort":"3260","storageHost":"10.20.1.12","volumeSize":"2621440000"}}],"nics":[{"deviceId":2,"networkRateMbps":200,"defaultNic":true,"uuid":"52d1b19d-b
053-4358-9659-6462f4d320ba","ip":"192.168.18.22","netmask":"255.255.252.0","gateway":"192.168.16.1","mac":"06:2c:bc:00:00:22","dns1":"192.168.20.18","dns2":"192.168.20.81","broadcastType":"Vlan","type":"Publi
c","broadcastUri":"vlan://untagged","isolationUri":"vlan://untagged","isSecurityGroupEnabled":false,"name":"PUBLIC"},{"deviceId":0,"networkRateMbps":200,"defaultNic":false,"uuid":"ea1180aa-9889-4db5-8414-6d0b
b9653f65","ip":"10.10.2.1","netmask":"255.255.255.0","mac":"02:00:4a:0a:00:02","dns1":"192.168.20.18","dns2":"192.168.20.81","broadcastType":"Vswitch","type":"Guest","broadcastUri":"vs://385","isolationUri":"
vs://385","isSecurityGroupEnabled":false,"name":"GUEST"},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"uuid":"c17a414d-e0d1-4ff8-80a3-6db65c5af96e","ip":"169.254.2.254","netmask":"255.255.0.0","gatew
ay":"169.254.0.1","mac":"0e:00:a9:fe:02:fe","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false}]},"_iqnToPath":{},"result":true,"wait":0}},{"com.cloud.agent.api.check.CheckSshAnswer":
{"result":true,"wait":0}},{"com.cloud.agent.api.GetDomRVersionAnswer":{"result":false,"details":"getDomRVersionCmd
failed","wait":0}}] }
2014-06-10 14:14:10,551 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98c6
ctx-8fd8ddc7) Seq 15-1518796837: Received: { Ans: , MgmtId: 123561959226, via:
15, Ver: v1, Flags: 10, { StartAnswer, CheckSshAns
wer, GetDomRVersionAnswer } }
2014-06-10 14:14:10,602 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) Unable to get the template/scripts
version of router r-187-VM due to: getDomRVersionCmd f
ailed
2014-06-10 14:14:10,602 INFO [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) The guru did not like the answers
so stopping VM[DomainRouter|r-187-VM]
2014-06-10 14:14:10,605 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98c6
ctx-8fd8ddc7) Seq 15-1518796840: Sending { Cmd , MgmtId: 123561959226, via:
15(xenserver2), Ver: v1, Flags: 100011, [{"com.cloud.
agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-187-VM","wait":0}}]
}
2014-06-10 14:14:10,605 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98c6
ctx-8fd8ddc7) Seq 15-1518796840: Executing: { Cmd , MgmtId: 123561959226, via:
15(xenserver2), Ver: v1, Flags: 100011, [{"com.clo
ud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-187-VM","wait":0}}]
}
2014-06-10 14:14:10,606 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-494:ctx-2c6d4aa5) Seq 15-1518796840: Executing request
2014-06-10 14:14:10,694 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-494:ctx-2c6d4aa5) 9. The VM r-187-VM is in Stopping state
2014-06-10 14:14:20,772 DEBUG [c.c.a.t.Request] (Job-Executor-17:ctx-7e9e98c6
ctx-8fd8ddc7) Seq 15-1518796840: Received: { Ans: , MgmtId: 123561959226, via:
15, Ver: v1, Flags: 10, { StopAnswer } }
2014-06-10 14:14:20,772 DEBUG [c.c.a.m.AgentManagerImpl]
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) Details from executing class
com.cloud.agent.api.StopCommand: Stop VM r-187-VM Succeed
2014-06-10 14:14:20,772 ERROR [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) Failed to start instance
VM[DomainRouter|r-187-VM]
com.cloud.utils.exception.ExecutionException: Unable to start
VM[DomainRouter|r-187-VM] due to error in finalizeStart, not retrying
2014-06-10 14:14:22,490 INFO [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-17:ctx-7e9e98c6 ctx-8fd8ddc7) Unable to contact resource.
com.cloud.exception.AgentUnavailableException: Resource [Host:15] is
unreachable: Host 15: Unable to start instance due to Unable to start
VM[DomainRouter|r-187-VM] due to error in finalizeStart, not retrying
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1072)
IMPORTANT NOTE: The information in this e-mail (and any attachments) is
confidential. The contents may not be disclosed or used by anyone other than
the addressee. If you are not the intended recipient, please notify the sender
immediately or telephone: +353 (0)1 6204700. We cannot accept any
responsibility for the accuracy or completeness of this message as it has been
transmitted over a public network. If you suspect that the message may have
been intercepted or amended, please call the sender.
