Re: Cloudstack 4.4.2 - unable to start new systemvm's

[ilya]

I see insufficient capacity..

post content of
grep -E 'job-184934|job-185963' management-server.log

 



On 3/25/15 2:52 PM, cs user wrote:
Hello,

We have upgraded from 4.3 to 4.4.2 and can no longer start new systemvm's
(or normal instances in zones which have both a secondary storage vm and
router running). Existing system vm's appear to have been upgraded.

We are using Xenserver version 6.1.

They appear for a split second in XenCenter but then immediately die.

There isn't much in the logs to go off, I will paste that in below.

One thing I have noticed is that a bunch of files got created in this
directory on our xenservers today during the upgrade.

/opt/cloud/bin/  (All new files have been placed here)

Where as we already have a bunch of files in this location with similar
names.

/opt/xensource/bin/ (Old files with dates from 2013)


For example the vhd-util file is present in both, but an older version of
it exists in /opt/xensource/bin/.

Does cloudstack 4.4.2 support Xenserver 6.1?

Have the above files been moved to the correct location for my version of
xen? Should I try to move them into /opt/xensource/bin/ ?

I've tried this with the vhd-util file, but it didn't seem to make a
difference. Perhaps some of the scripts need updating?


As promised here are the logs we can see when an instance tries to start.
All hosts are up and running and their state is green.

Any help would be appreciated, thanks!

ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-1e15b764
job-184934/job-185963) Unable to complete AsyncJobVO {id:185963, userId: 1,
accountId: 1, instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAqTnQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 345049290103, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Wed Mar 25 21:41:13 GMT 2015}, job
origin:184934
com.cloud.exception.InsufficientServerCapacityException: Unable to create
a deployment for VM[ConsoleProxy|v-10830-VM]Scope=interface
com.cloud.dc.DataCenter; id=20
         at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:947)
         at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5195)
         at sun.reflect.GeneratedMethodAccessor281.invoke(Unknown Source)

         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:601)

         at
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
         at
com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5340)
         at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
         at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
         at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
         at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
         at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
         at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
         at java.util.concurrent.FutureTask.run(FutureTask.java:166)

         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:722)

WARN  [c.c.v.SystemVmLoadScanner] (consoleproxy-1:ctx-df8209d7) Unexpected
exception Job failed due to exception Unable to create a deployment for
VM[ConsoleProxy|v-10830-VM]
java.lang.RuntimeException: Job failed due to exception Unable to create a
deployment for VM[ConsoleProxy|v-10830-VM]
         at
com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:114)
         at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
         at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
         at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
         at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
         at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
         at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
         at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
         at java.util.concurrent.FutureTask.run(FutureTask.java:166)

         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:722)