Hello All
I'm facing the following issue since early morning today. I tried to update
the system offering of the virtual router and restart the VM, but the VM starts
and then after it reaches the login prompt throws some dnsmasq errors and halts.
I've not been able to start the Virtual Router after that.
Any help in this issue would be much appreciated.
The errors found in the log files:
2014-11-05 15:29:20,327 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-210:ctx-193f97e2) callHostPlugin failed for cmd: createFileInDomr
with args filepath:
/var/cache/cloud/VR-5ef8abcd-0141-4f3f-9bcd-c5cc8a8221be.cfg, domrip:
169.254.0.178, filecontents: #Apache CloudStack Virtual Router Config File
<version>
1.0
</version>
<script>
/opt/cloud/bin/monitor_service.sh -c
[dhcp]:processname=dnsmasq:servicename=dnsmasq:pidfile=/var/run/dnsmasq/dnsmasq.pid:,[ssh]:processname=sshd:servicename=ssh:pidfile=/var/run/sshd.pid:,[webserver]:processname=apache2:servicename=apache2:pidfile=/var/run/apache2.pid:,
</script>
<script>
...
....
..
/opt/cloud/bin/vmdata.py -d
eyIxNDYuMTY5LjQ2LjE4OCI6W1sidXNlcmRhdGEiLCJ1c2VyLWRhdGEiLG51bGxdLFsibWV0YWRhdGEiLCJzZXJ2aWNlLW9mZmVyaW5nIiwiTG9jYWwgU3RvcmFnZSJdLFsibWV0YWRhdGEiLCJhdmFpbGFiaWxpdHktem9uZSIsImRvYy00NiJdLFsibWV0YWRhdGEiLCJsb2NhbC1pcHY0IiwiMTQ2LjE2OS40Ni4xODgiXSxbIm1ldGFkYXRhIiwibG9jYWwtaG9zdG5hbWUiLCJ0ZXN0Y3NnMyJdLFsibWV0YWRhdGEiLCJwdWJsaWMtaXB2NCIsIjE0Ni4xNjkuNDYuMTg4Il0sWyJtZXRhZGF0YSIsInB1YmxpYy1ob3N0bmFtZSIsInRlc3Rjc2czIl0sWyJtZXRhZGF0YSIsImluc3RhbmNlLWlkIiwiZWIxMDIxYTEtMTlmOS00YjY5LWFjNGMtODExMTg0NjE5ZGQ4Il0sWyJtZXRhZGF0YSIsInZtLWlkIiwiZWIxMDIxYTEtMTlmOS00YjY5LWFjNGMtODExMTg0NjE5ZGQ4Il0sWyJtZXRhZGF0YSIsInB1YmxpYy1rZXlzIixudWxsXSxbIm1ldGFkYXRhIiwiY2xvdWQtaWRlbnRpZmllciIsIkNsb3VkU3RhY2stezRkNzVjZDBlLTZmOTMtNDdiZC1iOGE0LTlmODNhMzU3NDlmY30iXV19
</script>
, due to There was a failure communicating with the plugin.
2014-11-05 15:29:20,337 WARN [c.c.a.m.DirectAgentAttache]
(DirectAgent-210:ctx-193f97e2) Seq 11-7965178890958143578: Exception Caught
while executing command
com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd:
createFileInDomr with args filepath:
/var/cache/cloud/VR-5ef8abcd-0141-4f3f-9bcd-c5cc8a8221be.cfg, domrip:
169.254.0.178, filecontents: #Apache CloudStack Virtual Router Config File
<version>
1.0
</version>
.....
.....
<script>
/opt/cloud/bin/vmdata.py -d
eyIxNDYuMTY5LjQ2LjE4OCI6W1sidXNlcmRhdGEiLCJ1c2VyLWRhdGEiLG51bGxdLFsibWV0YWRhdGEiLCJzZXJ2aWNlLW9mZmVyaW5nIiwiTG9jYWwgU3RvcmFnZSJdLFsibWV0YWRhdGEiLCJhdmFpbGFiaWxpdHktem9uZSIsImRvYy00NiJdLFsibWV0YWRhdGEiLCJsb2NhbC1pcHY0IiwiMTQ2LjE2OS40Ni4xODgiXSxbIm1ldGFkYXRhIiwibG9jYWwtaG9zdG5hbWUiLCJ0ZXN0Y3NnMyJdLFsibWV0YWRhdGEiLCJwdWJsaWMtaXB2NCIsIjE0Ni4xNjkuNDYuMTg4Il0sWyJtZXRhZGF0YSIsInB1YmxpYy1ob3N0bmFtZSIsInRlc3Rjc2czIl0sWyJtZXRhZGF0YSIsImluc3RhbmNlLWlkIiwiZWIxMDIxYTEtMTlmOS00YjY5LWFjNGMtODExMTg0NjE5ZGQ4Il0sWyJtZXRhZGF0YSIsInZtLWlkIiwiZWIxMDIxYTEtMTlmOS00YjY5LWFjNGMtODExMTg0NjE5ZGQ4Il0sWyJtZXRhZGF0YSIsInB1YmxpYy1rZXlzIixudWxsXSxbIm1ldGFkYXRhIiwiY2xvdWQtaWRlbnRpZmllciIsIkNsb3VkU3RhY2stezRkNzVjZDBlLTZmOTMtNDdiZC1iOGE0LTlmODNhMzU3NDlmY30iXV19
</script>
, due to There was a failure communicating with the plugin.
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.callHostPlugin(CitrixResourceBase.java:3782)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.createFileInVR(CitrixResourceBase.java:588)
at
com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.execute(VirtualRoutingResource.java:1161)
at
com.cloud.agent.resource.virtualnetwork.VirtualRoutingResource.executeRequest(VirtualRoutingResource.java:171)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:444)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:61)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
at
com.cloud.hypervisor.xen.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
...
...
2014-11-05 15:29:20,344 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-6:ctx-e652ea17 job-9535/job-9536 ctx-f3deae88) Seq
11-7965178890958143578: Received: { Ans: , MgmtId: 233845174477512, via: 11,
Ver: v1, Flags: 10, { StartAnswer, CheckSshAnswer, GetDomRVersionAnswer,
NetworkUsageAnswer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answe
r
, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answ
e
r, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer, Answer,
Answer, Answer, Answer, Answer, Answer, Answer } }
2014-11-05 15:29:20,350 WARN [o.a.c.alerts] (Work-Job-Executor-6:ctx-e652ea17
job-9535/job-9536 ctx-f3deae88) alertType:: 9 // dataCenterId:: 1 // podId:: 1
// clusterId:: null // message:: Command: com.cloud.agent.api.Command failed
while starting virtual router
2014-11-05 15:29:20,355 WARN [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Work-Job-Executor-6:ctx-e652ea17 job-9535/job-9536 ctx-f3deae88) Command:
com.cloud.agent.api.Command failed while starting virtual router
2014-11-05 15:29:20,355 INFO [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-6:ctx-e652ea17 job-9535/job-9536 ctx-f3deae88) The guru did
not like the answers so stopping VM[DomainRouter|r-673-VM]
2014-11-05 15:29:20,358 DEBUG [c.c.a.t.Request] (Work-Job-Executor-6:ctx-e652ea17 job-9535/job-9536 ctx-f3deae88) Seq 11-7965178890958143581: Sending
{ Cmd , MgmtId: 233845174477512, via: 11(motmot), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"r-673-VM","wait":0}}]
}
2014-11-05 15:29:20,359 DEBUG [c.c.a.t.Request] (Work-Job-Executor-6:ctx-e652ea17 job-9535/job-9536 ctx-f3deae88) Seq 11-7965178890958143581:
Executing: { Cmd , MgmtId: 233845174477512, via: 11(motmot), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"r-673-VM","wait":0}}]
}
2014-11-05 15:29:20,359 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-354:ctx-9b42d6df) Seq 11-7965178890958143581: Executing request
2014-11-05 15:29:20,382 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-354:ctx-9b42d6df) 9. The VM r-673-VM is in Stopping state
2014-11-05 15:29:20,682 INFO [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-354:ctx-9b42d6df) Removed network rules for vm r-673-VM
--
best wishes,
Thomas Joseph
