Please check the connection between your XenServer and NFS primary storage
-----邮件原件----- 发件人: pradeep pal <pspa...@hotmail.com> 发送时间: 2019年12月1日 7:23 收件人: users@cloudstack.apache.org 主题: Unable to start a VM due to insufficient capacity ________________________________ Hi, I am new for the CloudStack. I have installed and configure ACS 4.11.3.0+mysql5.6+XenServer 7.1.CU2 in my office. But when we launch instance I always get "Unable to start a VM due to insufficient capacity" Can anyone help me to solve this issue. Please find the attached log. 2019-12-01 02:58:13,483 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-4:ctx-a1580e0a job-90) (logid:ede69855) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to insufficient capacity Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-13-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 02:58:13,486 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-4:ctx-a1580e0a job-90) (logid:ede69855) Complete async job-90, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a VM due to insufficient capacity"} 2019-12-01 03:07:17,393 ERROR [c.c.c.ClusterServiceServletContainer] (Thread-20:null) (logid:) Unexpected exception 2019-12-01 03:08:29,357 WARN [n.s.e.c.ConfigurationFactory] (main:null) (logid:) No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.11.3.0.jar!/ehcache-failsafe.xml 2019-12-01 03:08:50,346 INFO [c.c.h.x.r.XenServerConnectionPool] (main:null) (logid:) XenServer Connection Pool Configs: sleep.interval.on.error=10000 2019-12-01 03:08:59,000 ERROR [c.c.u.s.SshHelper] (AgentTaskPool-1:ctx-55835787) (logid:58f334e3) SSH execution of command xe sm-list | grep "resigning of duplicates" has an error status code in return. Result output: 2019-12-01 03:08:59,124 ERROR [c.c.u.s.SshHelper] (AgentTaskPool-2:ctx-79d54dc9) (logid:b94602f7) SSH execution of command xe sm-list | grep "resigning of duplicates" has an error status code in return. Result output: com.cloud.utils.exception.CloudRuntimeException: Unable to setup heartbeat file entry on SR 1562e840-29e3-ba00-e148-6dad6353042a due to #3# Unable to find the host uuid: 0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 2019-12-01 03:09:10,715 INFO [c.c.u.e.CSExceptionErrorCode] (AgentTaskPool-2:ctx-79d54dc9) (logid:b94602f7) Could not find exception: com.cloud.exception.ConnectionException in error code list for exceptions 2019-12-01 03:09:10,715 WARN [c.c.a.m.AgentManagerImpl] (AgentTaskPool-2:ctx-79d54dc9) (logid:b94602f7) Monitor StoragePoolMonitor says there is an error in the connect process for 4 due to Unable to connect host 4 to storage pool id 1 due to com.cloud.utils.exception.CloudRuntimeException: Unable establish connection from storage head to storage pool 1 due to ModifyStoragePoolCommand add XenAPIException:Unable to setup heartbeat file entry on SR 1562e840-29e3-ba00-e148-6dad6353042a due to #3# Unable to find the host uuid: 0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 host:0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 pool: 172.29.1.97/nfs/export/primary1 com.cloud.exception.ConnectionException: Unable to connect host 4 to storage pool id 1 due to com.cloud.utils.exception.CloudRuntimeException: Unable establish connection from storage head to storage pool 1 due to ModifyStoragePoolCommand add XenAPIException:Unable to setup heartbeat file entry on SR 1562e840-29e3-ba00-e148-6dad6353042a due to #3# Unable to find the host uuid: 0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 host:0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 pool: 172.29.1.97/nfs/export/primary1 Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable establish connection from storage head to storage pool 1 due to ModifyStoragePoolCommand add XenAPIException:Unable to setup heartbeat file entry on SR 1562e840-29e3-ba00-e148-6dad6353042a due to #3# Unable to find the host uuid: 0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 host:0e9f60c3-cfb7-4038-b00f-319d7e22d7f6 pool: 172.29.1.97/nfs/export/primary1 2019-12-01 03:10:27,667 ERROR [c.c.u.s.SshHelper] (AgentTaskPool-3:ctx-f701d1f9) (logid:ceb4daae) SSH execution of command xe sm-list | grep "resigning of duplicates" has an error status code in return. Result output: 2019-12-01 03:23:49,476 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-1:ctx-f035b3f6 job-99/job-100 ctx-1ee73f54) (logid:0eb71d41) Cleaning up because we're unable to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 03:23:50,229 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-1:ctx-f035b3f6 job-99/job-100 ctx-1ee73f54) (logid:0eb71d41) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:23:50,229 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-1:ctx-f035b3f6 job-99/job-100 ctx-1ee73f54) (logid:0eb71d41) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:23:51,612 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-2:ctx-219e47df job-99/job-101 ctx-74b313f0) (logid:0eb71d41) Cleaning up because we're unable to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 03:23:52,142 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-2:ctx-219e47df job-99/job-101 ctx-74b313f0) (logid:0eb71d41) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:23:52,142 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-2:ctx-219e47df job-99/job-101 ctx-74b313f0) (logid:0eb71d41) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:23:52,239 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-1:ctx-2ddeb2a4 job-99 ctx-f6560edc) (logid:0eb71d41) Destroying vm VM[User|i-2-14-VM] as it failed to create on Host with Id:null 2019-12-01 03:23:52,640 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-1:ctx-2ddeb2a4 job-99) (logid:0eb71d41) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to insufficient capacity Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-14-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:23:52,644 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-1:ctx-2ddeb2a4 job-99) (logid:0eb71d41) Complete async job-99, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a VM due to insufficient capacity"} 2019-12-01 03:43:27,177 INFO [c.c.a.ApiServer] (qtp504527234-18:ctx-dce5313f ctx-f343b0ac) (logid:e90a7d5c) Source cidrs formatting error 0.0.0.0/0.0.0.0 2019-12-01 03:44:41,436 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-6:ctx-446aa701 job-109/job-110 ctx-6755b29a) (logid:8f5f512c) Cleaning up because we're unabl to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 03:44:42,069 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-6:ctx-446aa701 job-109/job-110 ctx-6755b29a) (logid:8f5f512c) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:44:42,069 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-6:ctx-446aa701 job-109/job-110 ctx-6755b29a) (logid:8f5f512c) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:44:43,465 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-7:ctx-e0086abb job-109/job-111 ctx-d2bcce7f) (logid:8f5f512c) Cleaning up because we're unabl to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 03:44:43,948 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-7:ctx-e0086abb job-109/job-111 ctx-d2bcce7f) (logid:8f5f512c) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:44:43,948 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-7:ctx-e0086abb job-109/job-111 ctx-d2bcce7f) (logid:8f5f512c) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:44:44,024 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-6:ctx-360c0e90 job-109 ctx-3497c21d) (logid:8f5f512c) Destroying vm VM[User|i-2-15-VM] as it failed to create on Host with Id:null 2019-12-01 03:44:44,513 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-6:ctx-360c0e90 job-109) (logid:8f5f512c) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to insufficient capacity Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-15-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 03:44:44,518 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-6:ctx-360c0e90 job-109) (logid:8f5f512c) Complete async job-109, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a VM due to insufficient capacity"} 2019-12-01 04:20:29,350 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-14:ctx-c18e434f job-120/job-121 ctx-0e8d39cb) (logid:04300826) Cleaning up because we're unable to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 04:20:29,840 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-14:ctx-c18e434f job-120/job-121 ctx-0e8d39cb) (logid:04300826) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 04:20:29,841 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-14:ctx-c18e434f job-120/job-121 ctx-0e8d39cb) (logid:04300826) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 04:20:31,506 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-15:ctx-a4b1bc3a job-120/job-122 ctx-2b626f79) (logid:04300826) Cleaning up because we're unable to implement the network Ntwk[206|Guest|11] com.cloud.exception.InsufficientVirtualNetworkCapacityException: Unable to allocate vnet as a part of network Ntwk[206|Guest|11] implement Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Can't find at least one router! 2019-12-01 04:20:31,984 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-15:ctx-a4b1bc3a job-120/job-122 ctx-2b626f79) (logid:04300826) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 04:20:31,984 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-15:ctx-a4b1bc3a job-120/job-122 ctx-2b626f79) (logid:04300826) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 04:20:32,111 DEBUG [c.c.v.UserVmManagerImpl] (API-Job-Executor-9:ctx-e3883c7d job-120 ctx-55f72391) (logid:04300826) Destroying vm VM[User|i-2-17-VM] as it failed to create on Host with Id:null 2019-12-01 04:20:32,494 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-9:ctx-e3883c7d job-120) (logid:04300826) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to insufficient capacity Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1 2019-12-01 04:20:32,496 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-9:ctx-e3883c7d job-120) (logid:04300826) Complete async job-120, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a VM due to insufficient capacity"} Thanks Pradeep