Hi All,
I am running cloudstack 4.4.1 in ubuntu 14.04 with three kvm Hosts. I was
successfully able to start the system virtual machines. And i was
successful in running the health script and no errors occurred. Now when i
am trying to create and instance i am getting two exceptions.
2015-01-04 02:45:06,605 ERROR [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Failed to
start instance VM[User|i-2-16-VM]
com.cloud.utils.exception.CloudRuntimeException: Unable to get answer that
is of class com.cloud.agent.api.StartAnswer
at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1020)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-01-04 02:45:06,688 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Cleaning up
resources for the vm VM[User|i-2-16-VM] in Starting state
2015-01-04 02:45:06,690 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Seq
6-3627649499846934559: Sending { Cmd , MgmtId: 248795600505608, via:
6(virtualops-h4), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"i-2-16-VM","wait":0}}]
}
2015-01-04 02:45:06,828 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-13:null) Seq 6-3627649499846934559: Processing: {
Ans: , MgmtId: 248795600505608, via: 6, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2015-01-04 02:45:06,828 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Seq
6-3627649499846934559: Received: { Ans: , MgmtId: 248795600505608, via: 6,
Ver: v1, Flags: 10, { StopAnswer } }
2015-01-04 02:45:06,836 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Changing
active number of nics for network id=204 on -1
2015-01-04 02:45:07,043 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Asking
VirtualRouter to release
NicProfile[30-16-a82c2881-1c14-4494-89fa-cde39b406313-10.8.100.27-vlan://untagged
2015-01-04 02:45:07,045 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Asking
SecurityGroupProvider to release
NicProfile[30-16-a82c2881-1c14-4494-89fa-cde39b406313-10.8.100.27-vlan://untagged
2015-01-04 02:45:07,045 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Successfully
released network resources for the vm VM[User|i-2-16-VM]
2015-01-04 02:45:07,045 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Successfully
cleanued up resources for the vm VM[User|i-2-16-VM] in Starting state
2015-01-04 02:45:07,298 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) VM state
transitted from :Starting to Stopped with event: OperationFailedvm's
original host id: null new host id: null host id before state transition: 6
2015-01-04 02:45:07,304 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Hosts's
actual total CPU: 13200 and CPU after applying overprovisioning: 13200
2015-01-04 02:45:07,304 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Hosts's
actual total RAM: 4104921088 and RAM after applying overprovisioning:
4104921088
2015-01-04 02:45:07,304 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) release cpu
from host: 6, old used: 500,reserved: 0, actual total: 13200, total with
overprovisioning: 13200; new used: 0,reserved:0; movedfromreserved:
false,moveToReserveredfalse
2015-01-04 02:45:07,304 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) release mem
from host: 6, old used: 536870912,reserved: 0, total: 4104921088; new used:
0,reserved:0; movedfromreserved: false,moveToReserveredfalse
2015-01-04 02:45:07,481 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Invocation
exception, caused by: com.cloud.exception.AgentUnavailableException:
Resource [Host:6] is unreachable: Host 6: Unable to start instance due to
Unable to get answer that is of class com.cloud.agent.api.StartAnswer
2015-01-04 02:45:07,482 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98 ctx-8772bf32) Rethrow
exception com.cloud.exception.AgentUnavailableException: Resource [Host:6]
is unreachable: Host 6: Unable to start instance due to Unable to get
answer that is of class com.cloud.agent.api.StartAnswer
2015-01-04 02:45:07,482 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98) Done with run of VM work
job: com.cloud.vm.VmWorkStart for VM 16, job origin: 97
2015-01-04 02:45:07,482 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-23:ctx-afbc7bf5 job-97/job-98) Unable to complete
AsyncJobVO {id:98, userId: 2, accountId: 2, instanceType: null, instanceId:
null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAAEHQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAXBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAAFzcQB-AAgAAAAAAAAABnBwcHEAfgAKcHNyABFqYXZhLnV0aWwuSGFzaE1hcAUH2sHDFmDRAwACRgAKbG9hZEZhY3RvckkACXRocmVzaG9sZHhwP0AAAAAAAAx3CAAAABAAAAABdAAKVm1QYXNzd29yZHQAHHJPMEFCWFFBRG5OaGRtVmtYM0JoYzNOM2IzSmt4cA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 248795600505608, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Sun Jan 04 02:44:31 IST 2015}, job
origin:97
com.cloud.exception.AgentUnavailableException: Resource [Host:6] is
unreachable: Host 6: Unable to start instance due to Unable to get answer
that is of class com.cloud.agent.api.StartAnswer
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1106)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to get
answer that is of class com.cloud.agent.api.StartAnswer
at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80)
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1020)
... 20 more
The two exceptions are com.cloud.exception.AgentUnavailableException and
com.cloud.utils.exception.CloudRuntimeException. And all my agents are
running fine and they are connected to the MS. And when i create the vm
instance the virtual router is getting started and it is running fine. But
still unable to create a vm instance. Any help will be greatly appreciated.
Thank you.
