I am using 4.2.1-src branch, now I used smaller vbox image (900MB, I can't remember which image I used earlier),
below is full log of jetty, starting from execution of mvn -P developer -pl tools/devcloud -Ddeploysvr INFO [cloud.configuration.ConfigurationManagerImpl] (507621488@qtp- 562387789-3:) adding a new subnet to the network 204 INFO [cloud.resource.ResourceManagerImpl] (507621488@qtp-562387789-3:) Trying to add a new host at http://192.168.56.10/ in data center 1 INFO [xen.discoverer.XcpServerDiscoverer] (507621488@qtp-562387789-3:) Found host devcloud ip=192.168.56.10 product version=1.6.0 INFO [xen.resource.CitrixResourceBase] (507621488@qtp-562387789-3:) Private Network is Pool-wide network associated with eth0 for host 192.168.56.10 INFO [xen.resource.CitrixResourceBase] (507621488@qtp-562387789-3:) Guest Network is Pool-wide network associated with eth0 for host 192.168.56.10 INFO [xen.resource.CitrixResourceBase] (507621488@qtp-562387789-3:) Public Network is Pool-wide network associated with eth0 for host 192.168.56.10 INFO [xen.discoverer.XcpServerDiscoverer] (507621488@qtp-562387789-3:) Host: devcloud connected with hypervisor type: XenServer. Checking CIDR... INFO [agent.manager.DirectAgentAttache] (507621488@qtp-562387789-3:) StartupAnswer received 1 Interval = 60 WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../ovsgre WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//ipassoc.sh WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//getDomRVersion.sh WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//getRouterStatus.sh WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//networkUsage.sh WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../../../../network/domr//l2tp_vpn.sh WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) We cannot locate /opt/cloudstack/apache-cloudstack-4.2.1-src/client/target/cloud-client-ui-4.2.1/WEB-INF/classes/scripts/vm/hypervisor/xenserver/xcposs/../vhd-util INFO [network.security.SecurityGroupListener] (507621488@qtp-562387789-3:) Received a host startup notification INFO [network.security.SecurityGroupListener] (507621488@qtp-562387789-3:) Scheduled network rules cleanup, interval=1841 INFO [network.security.SecurityGroupListener] (507621488@qtp-562387789-3:) Received a host startup notification INFO [cloud.resource.ResourceManagerImpl] (507621488@qtp-562387789-3:) server resources successfully discovered by XCP Agent INFO [datastore.lifecycle.CloudStackImageStoreLifeCycleImpl] (507621488@qtp-562387789-3:) Trying to add a new data store at nfs:// 192.168.56.10/opt/storage/secondary to data center 1 INFO [cloud.configuration.ConfigurationManagerImpl] (507621488@qtp-562387789-3:) No storage traffic type was specified by admin, create default storage traffic on physical network 200 with same configure of management traffic type INFO [storage.endpoint.DefaultEndPointSelector] (StatsCollector-3:) No running ssvm is found, so command will be sent to LocalHostEndPoint INFO [cloud.secstorage.PremiumSecondaryStorageManagerImpl] (secstorage-1:) No running secondary storage vms found in datacenter id=1, starting one INFO [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:) No stopped secondary storage vm is available, need to allocate a new secondary storage vm INFO [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:) No stopped console proxy is available, need to allocate a new console proxy INFO [storage.volume.VolumeServiceImpl] (consoleproxy-1:) lock is acquired for VMTemplateStoragePool 1 WARN [xen.resource.CitrixResourceBase] (DirectAgent-4:) kill_copy_process failed WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-4:) can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-4:) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:793) at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:864) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:625) at com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:143) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [storage.volume.VolumeServiceImpl] (consoleproxy-1:) releasing lock for VMTemplateStoragePool 1 WARN [utils.db.Merovingian2] (consoleproxy-1:) Was unable to find lock for the key template_spool_ref1 and thread id 960730019 INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[2|vm=2|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2590) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2641) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:570) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:928) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1672) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [xen.resource.CitrixResourceBase] (DirectAgent-3:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-3:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-3:) Seq 1-490078221: Exception Caught while executing commandWARN [utils.db.Merovingian2] (secstorage-1:) Was unable to find lock for the key template_spool_ref2 and thread id 1787476943 INFO [xen.resource.CitrixResourceBase] (DirectAgent-14:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-14:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-14:) Seq 1-490078225: Exception Caught while executing commandWARN [utils.db.Merovingian2] (consoleproxy-1:) Was unable to find lock for the key template_spool_ref3 and thread id 960730019 WARN [agent.manager.DirectAgentAttache] (DirectAgent-18:) Seq 1-490078229: Exception Caught while executing command WARN [xen.resource.CitrixResourceBase] (DirectAgent-1:) kill_copy_process failed WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-1:) can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-1:) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:793) at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:864) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:625) at com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:143) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [storage.volume.VolumeServiceImpl] (secstorage-1:) releasing lock for VMTemplateStoragePool 4 WARN [utils.db.Merovingian2] (secstorage-1:) Was unable to find lock for the key template_spool_ref4 and thread id 1787476943 INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[3|vm=3|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2590) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2641) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:570) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:267) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:696) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300) at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123) at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50) at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104) at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [xen.resource.CitrixResourceBase] (DirectAgent-6:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea INFO [xen.resource.CitrixResourceBase] (DirectAgent-10:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-10:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-10:) Seq 1-490078233: Exception Caught while executing commandWARN [xen.resource.CitrixResourceBase] (DirectAgent-23:) kill_copy_process failed WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-23:) can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-23:) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:793) at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:864) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:625) at com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:143) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [storage.volume.VolumeServiceImpl] (consoleproxy-1:) releasing lock for VMTemplateStoragePool 5 WARN [utils.db.Merovingian2] (consoleproxy-1:) Was unable to find lock for the key template_spool_ref5 and thread id 960730019 INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[2|vm=2|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2590) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2641) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:570) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:928) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1672) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [xen.resource.CitrixResourceBase] (DirectAgent-15:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-15:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-15:) Seq 1-490078237: Exception Caught while executing command INFO [xen.resource.CitrixResourceBase] (DirectAgent-25:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adceaWARN [xen.resource.CitrixResourceBase] (DirectAgent-26:) kill_copy_process failed WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-26:) can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-26:) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:793) at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:864) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:625) at com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:143) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [storage.volume.VolumeServiceImpl] (secstorage-1:) releasing lock for VMTemplateStoragePool 6 WARN [utils.db.Merovingian2] (secstorage-1:) Was unable to find lock for the key template_spool_ref6 and thread id 1787476943 INFO [cloud.vm.VirtualMachineManagerImpl] (secstorage-1:) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[4|vm=4|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2590) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2641) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:570) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:267) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:696) at com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300) at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:123) at com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:50) at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104) at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [xen.resource.CitrixResourceBase] (DirectAgent-27:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea INFO [xen.resource.CitrixResourceBase] (DirectAgent-12:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-12:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-12:) Seq 1-490078241: Exception Caught while executing commandWARN [xen.resource.CitrixResourceBase] (DirectAgent-4:) kill_copy_process failed WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-4:) can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 WARN [xen.resource.XenServerStorageProcessor] (DirectAgent-4:) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:793) at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:864) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70) at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:625) at com.cloud.hypervisor.xen.resource.XcpOssResource.executeRequest(XcpOssResource.java:143) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [storage.volume.VolumeServiceImpl] (consoleproxy-1:) releasing lock for VMTemplateStoragePool 7 WARN [utils.db.Merovingian2] (consoleproxy-1:) Was unable to find lock for the key template_spool_ref7 and thread id 960730019 INFO [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:) Unable to contact resource. com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[2|vm=2|ROOT]:Catch Exception com.cloud.utils.exception.CloudRuntimeException for template + due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr 9f3f9262-3f77-09cc-2df7-0d8475676260 at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2590) at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2641) at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577) at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:570) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:928) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1672) at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157) at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111) at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) INFO [xen.resource.CitrixResourceBase] (DirectAgent-6:) VM does not exist on XenServer9d4c9db8-32f7-25c3-0435-eab4bf3adcea WARN [xen.resource.CitrixResourceBase] (DirectAgent-6:) callHostPlugin failed for cmd: setDNATRule with args add: false, due to UNKNOWN_XENAPI_PLUGIN_FUNCTIONsetDNATRule WARN [agent.manager.DirectAgentAttache] (DirectAgent-6:) Seq 1-490078245: Exception Caught while executing command ......
