Hi, I am having very weird issue(since it did not happened before in same ACS version). I have created new domain. I have set resource limits on domain as well as on account. And I am making new instance , but CS is giving me error that,
"Maximum number of resources of type 'primary_storage' for account name=Ibh in domain id=17 has been exceeded. " I checked volumes table for domain id 17, there are no volumes present. Even resource count table. mysql> select * from volumes where domain_id=17; Empty set (0.00 sec) mysql> select * from resource_count where domain_id=17; +-----+------------+-----------+-------------------+-------+ | id | account_id | domain_id | type | count | +-----+------------+-----------+-------------------+-------+ | 521 | NULL | 17 | user_vm | 0 | | 522 | NULL | 17 | public_ip | 0 | | 523 | NULL | 17 | volume | 0 | | 524 | NULL | 17 | snapshot | 0 | | 525 | NULL | 17 | template | 0 | | 526 | NULL | 17 | project | 0 | | 527 | NULL | 17 | network | 0 | | 528 | NULL | 17 | vpc | 0 | | 529 | NULL | 17 | cpu | 0 | | 530 | NULL | 17 | memory | 0 | | 531 | NULL | 17 | primary_storage | 0 | | 532 | NULL | 17 | secondary_storage | 0 | +-----+------------+-----------+-------------------+-------+ 12 rows in set (0.00 sec) mysql> Check management logs below, 2015-09-30 14:17:51,446 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-440a0b8d) End cleanup expired async-jobs 2015-09-30 14:17:51,488 DEBUG [c.c.a.ApiServlet] (http-6443-exec-30:ctx-d50ea4f7) ===START=== 1.14.34.4 -- GET command=deployVirtualMachine&response=json&sessionkey=OpRnMrnYLMKtAXEB3pG6gsV4sl0%3D&zoneid=1baf17c9-8325-4fa6-bffc-e502a33b578b&templateid=9b335528-26e3-42d5-a49d-54e8c12c36b7&hypervisor=XenServer&serviceofferingid=275a8bf8-dc73-4520-99d0-d40d097575d0&details%5B0%5D.cpuNumber=2&details%5B0%5D.cpuSpeed=2400&details%5B0%5D.memory=2048&diskofferingid=a938f7a7-7bed-4737-a7b9-269eb255cbc4&iptonetworklist%5B0%5D.networkid=16ef9070-ea3f-4553-b241-f4704353b225&displayname=ddtests05&name=ddtests05&keyboard=us&_=1443615466300 2015-09-30 14:17:51,498 DEBUG [o.a.c.a.BaseCmd] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Ignoring paremeter displayvm as the caller is not authorized to pass it in 2015-09-30 14:17:51,498 DEBUG [o.a.c.a.BaseCmd] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Ignoring paremeter deploymentplanner as the caller is not authorized to pass it in 2015-09-30 14:17:51,519 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Access to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] granted to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] by DomainChecker 2015-09-30 14:17:51,531 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Access to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] granted to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] by DomainChecker 2015-09-30 14:17:51,545 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Access to Tmpl[202-ISO-202-2-a8c63bc3-71b5-34fc-87c4-fc298a4a6915 granted to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] by DomainChecker 2015-09-30 14:17:51,545 DEBUG [o.a.c.a.BaseCmd] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Ignoring paremeter displayvm as the caller is not authorized to pass it in 2015-09-30 14:17:51,545 DEBUG [o.a.c.a.BaseCmd] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Ignoring paremeter deploymentplanner as the caller is not authorized to pass it in 2015-09-30 14:17:51,556 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Access to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] granted to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] by DomainChecker 2015-09-30 14:17:51,580 DEBUG [c.c.u.AccountManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Access to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] granted to Acct[cf0040a8-a2e0-4358-b357-ed2123d0cac5-Ibh] by DomainChecker 2015-09-30 14:17:51,661 ERROR [c.c.r.ResourceLimitManagerImpl] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Maximum number of resources of type 'primary_storage' for account name=Ibh in domain id=17 has been exceeded. com.cloud.exception.ResourceAllocationException: Maximum number of resources of type 'primary_storage' for account name=Ibh in domain id=17 has been exceeded. at com.cloud.resourcelimit.ResourceLimitManagerImpl$1.doInTransactionWithoutResult(ResourceLimitManagerImpl.java:428) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:25) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:21) at com.cloud.utils.db.Transaction.execute(Transaction.java:45) at com.cloud.resourcelimit.ResourceLimitManagerImpl.checkResourceLimit(ResourceLimitManagerImpl.java:407) at com.cloud.vm.UserVmManagerImpl.createVirtualMachine(UserVmManagerImpl.java:2715) at com.cloud.vm.UserVmManagerImpl.createAdvancedVirtualMachine(UserVmManagerImpl.java:2630) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy213.createAdvancedVirtualMachine(Unknown Source) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.create(DeployVMCmd.java:599) at com.cloud.api.dispatch.CommandCreationWorker.handle(CommandCreationWorker.java:47) at com.cloud.api.dispatch.DispatchChain.dispatch(DispatchChain.java:37) at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:79) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:635) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:529) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:284) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:120) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:117) at com.cloud.api.ApiServlet.doGet(ApiServlet.java:79) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-09-30 14:17:51,663 DEBUG [c.c.u.d.T.Transaction] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Rolling back the transaction: Time = 11 Name = http-6443-exec-30; called by -TransactionLegacy.rollback:902-TransactionLegacy.removeUpTo:845-TransactionLegacy.close:669-Transaction.execute:49-ResourceLimitManagerImpl.checkResourceLimit:407-UserVmManagerImpl.createVirtualMachine:2715-UserVmManagerImpl.createAdvancedVirtualMachine:2630-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:57-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317 2015-09-30 14:17:51,701 WARN [o.a.c.a.c.u.v.DeployVMCmd] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Exception: com.cloud.exception.ResourceAllocationException: Maximum number of resources of type 'primary_storage' for account name=Ibh in domain id=17 has been exceeded. at com.cloud.resourcelimit.ResourceLimitManagerImpl$1.doInTransactionWithoutResult(ResourceLimitManagerImpl.java:428) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:25) at com.cloud.utils.db.TransactionCallbackWithExceptionNoReturn.doInTransaction(TransactionCallbackWithExceptionNoReturn.java:21) at com.cloud.utils.db.Transaction.execute(Transaction.java:45) at com.cloud.resourcelimit.ResourceLimitManagerImpl.checkResourceLimit(ResourceLimitManagerImpl.java:407) at com.cloud.vm.UserVmManagerImpl.createVirtualMachine(UserVmManagerImpl.java:2715) at com.cloud.vm.UserVmManagerImpl.createAdvancedVirtualMachine(UserVmManagerImpl.java:2630) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy213.createAdvancedVirtualMachine(Unknown Source) at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.create(DeployVMCmd.java:599) at com.cloud.api.dispatch.CommandCreationWorker.handle(CommandCreationWorker.java:47) at com.cloud.api.dispatch.DispatchChain.dispatch(DispatchChain.java:37) at com.cloud.api.ApiDispatcher.dispatchCreateCmd(ApiDispatcher.java:79) at com.cloud.api.ApiServer.queueCommand(ApiServer.java:635) at com.cloud.api.ApiServer.handleRequest(ApiServer.java:529) at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:284) at com.cloud.api.ApiServlet$1.run(ApiServlet.java:120) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:117) at com.cloud.api.ApiServlet.doGet(ApiServlet.java:79) at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2015-09-30 14:17:51,703 INFO [c.c.a.ApiServer] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) Maximum number of resources of type 'primary_storage' for account name=Ibh in domain id=17 has been exceeded. 2015-09-30 14:17:51,706 DEBUG [c.c.a.ApiServlet] (http-6443-exec-30:ctx-d50ea4f7 ctx-d7fca4e9) ===END=== 1.14.34.4 -- GET command=deployVirtualMachine&response=json&sessionkey=OpRnMrnYLMKtAXEB3pG6gsV4sl0%3D&zoneid=1baf17c9-8325-4fa6-bffc-e502a33b578b&templateid=9b335528-26e3-42d5-a49d-54e8c12c36b7&hypervisor=XenServer&serviceofferingid=275a8bf8-dc73-4520-99d0-d40d097575d0&details%5B0%5D.cpuNumber=2&details%5B0%5D.cpuSpeed=2400&details%5B0%5D.memory=2048&diskofferingid=a938f7a7-7bed-4737-a7b9-269eb255cbc4&iptonetworklist%5B0%5D.networkid=16ef9070-ea3f-4553-b241-f4704353b225&displayname=ddtests05&name=ddtests05&keyboard=us&_=1443615466300 2015-09-30 14:17:56,135 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-e6167e3d) Zone 1 is ready to launch secondary storage VM 2015-09-30 14:17:56,473 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-cf2c0ca6) Zone 1 is ready to launch console proxy /sonali