OK. i try to simulate again see if get it back On Thu, Apr 15, 2021 at 12:09 AM Nicolas Vazquez < nicolas.vazq...@shapeblue.com> wrote:
> Ok, that looks like an issue. Can you please report it if not existing > already on: https://github.com/apache/cloudstack/issues with the steps > you have done to reproduce it? > > > Regards, > > Nicolas Vazquez > > ________________________________ > From: Hean Seng <heans...@gmail.com> > Sent: Wednesday, April 14, 2021 11:49 AM > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > Subject: Re: Request failed with status code 530 > > Hi > > I think this is really a bug, each host suppose have one local storage > only, but it return two, after digging to storage_pool , it become NULL in > uuid , after i remove this record, then listHosts accessible now. > > select * from storage_pool_host_ref where host_id=17; > > > +----+---------+---------+---------------------+--------------+-------------------------------------------+ > > | id | host_id | pool_id | created | last_updated | local_path > | > > > +----+---------+---------+---------------------+--------------+-------------------------------------------+ > > | 27 | 17 | 3 | 2021-04-12 18:45:27 | NULL | > /mnt/1f45642b-f8f3-349f-8e46-8f3eb983da23 | > > | 28 | 17 | 14 | 2021-04-12 18:45:27 | NULL | > /var/lib/libvirt/images | > > | 29 | 17 | 15 | 2021-04-12 18:51:50 | NULL | > /var/lib/libvirt/images | > > > +----+---------+---------+---------------------+--------------+-------------------------------------------+ > > > > mysql> select id, uuid, name, status from storage_pool where id=14; > > +----+------+---------------------------------+-------------+ > > | id | uuid | name | status | > > +----+------+---------------------------------+-------------+ > > | 14 | NULL | lax-06 | Maintenance | > > +----+------+---------------------------------+-------------+ > > 1 row in set (0.00 sec) > > > mysql> select id, uuid, name, status from storage_pool where id=15; > > > +----+--------------------------------------+---------------------------------+-------------+ > > | id | uuid | name > | status | > > > +----+--------------------------------------+---------------------------------+-------------+ > > | 15 | 30717adf-46ba-43fe-bac9-038ef6039fda | lax-06 > | Maintenance | > > > +----+--------------------------------------+---------------------------------+-------------+ > > 1 row in set (0.00 sec) > > > > > > > On Wed, Apr 14, 2021 at 10:17 PM Nicolas Vazquez < > nicolas.vazq...@shapeblue.com> wrote: > > > Thanks, > > > > Have you deleted any storage pool accesible to the host? Please check if > > there are not missing references on the table 'storage_pool_host_ref' for > > host_id = 17. It could also be the case that a record on that table is > > pointing to a removed storage pool > > > > Can you try putting the host into maintenance first before deleting it? > > > > > > Regards, > > > > Nicolas Vazquez > > > > ________________________________ > > From: Hean Seng <heans...@gmail.com> > > Sent: Wednesday, April 14, 2021 10:30 AM > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > Subject: Re: Request failed with status code 530 > > > > Hi Nicolas, > > > > Thanks for reply , following is error : > > >>> # # cmk listHosts id=17 > > > > 2021-04-14 13:26:47,391 DEBUG [c.c.a.ApiServer] > > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) CIDRs > from > > which account 'Acct[b10cca30-6fb6-11eb-b8cc-ce70b13735fa-admin]' is > allowed > > to perform API calls: 0.0.0.0/0,::/0 > > > > 2021-04-14 13:26:47,396 DEBUG [c.c.a.q.QueryManagerImpl] > > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) > > >>>Searching for hosts>>> > > > > 2021-04-14 13:26:47,402 DEBUG [c.c.a.q.QueryManagerImpl] > > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) > > >>>Generating Response>>> > > > > 2021-04-14 13:26:47,408 ERROR [c.c.a.ApiServer] > > (qtp1644231115-2156:ctx-47a63ef7 ctx-6c8f0fae) (logid:f5043840) unhandled > > exception executing api command: [Ljava.lang.String;@34315235 > > > > java.lang.NullPointerException > > > > at > > > > > com.cloud.storage.StorageManagerImpl.isLocalStorageActiveOnHost(StorageManagerImpl.java:404) > > > > at jdk.internal.reflect.GeneratedMethodAccessor372.invoke(Unknown Source) > > > > at > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > at > > > > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > > > at > > > > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > > > at > > > > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > > > at com.sun.proxy.$Proxy96.isLocalStorageActiveOnHost(Unknown Source) > > > > at > com.cloud.api.ApiDBUtils.isLocalStorageActiveOnHost(ApiDBUtils.java:955) > > > > at > > > > > com.cloud.api.query.dao.HostJoinDaoImpl.newHostResponse(HostJoinDaoImpl.java:238) > > > > at jdk.internal.reflect.GeneratedMethodAccessor370.invoke(Unknown Source) > > > > at > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > at > > > > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > > > at > > > > > com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) > > > > at > > > > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > > > at > > > > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > > > at com.sun.proxy.$Proxy316.newHostResponse(Unknown Source) > > > > at com.cloud.api.ApiDBUtils.newHostResponse(ApiDBUtils.java:1870) > > > > at > > > > > com.cloud.api.query.ViewResponseHelper.createHostResponse(ViewResponseHelper.java:248) > > > > at > > > > > com.cloud.api.query.QueryManagerImpl.searchForServers(QueryManagerImpl.java:1748) > > > > at > > > > > org.apache.cloudstack.api.command.admin.host.ListHostsCmd.getHostResponses(ListHostsCmd.java:208) > > > > at > > > > > org.apache.cloudstack.api.command.admin.host.ListHostsCmd.execute(ListHostsCmd.java:233) > > > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > > > at > > > > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > > > at > > > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > > > at > > > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > > > at > > org.eclipse.jetty.io > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > > > at > > > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > > > > > > > >>> cmk deletehost id=17 forced=true > > > > > > > > 2021-04-14 13:29:21,967 DEBUG [c.c.r.ResourceManagerImpl] > > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) Cannot > > transmit host 17 to Enabled state > > > > com.cloud.utils.fsm.NoTransitionException: No next resource state found > for > > current state = Enabled event = DeleteHost > > > > at > > > > > com.cloud.resource.ResourceManagerImpl.resourceStateTransitTo(ResourceManagerImpl.java:1213) > > > > at > > > > > com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:904) > > > > at > > > > > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > > > > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > > > > at > > > > > com.cloud.resource.ResourceManagerImpl.doDeleteHost(ResourceManagerImpl.java:873) > > > > at > > > > > com.cloud.resource.ResourceManagerImpl.deleteHost(ResourceManagerImpl.java:967) > > > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > > > at > > > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > at > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > at > > > > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > > > at > > > > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > > > at > > > > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > > > at com.sun.proxy.$Proxy188.deleteHost(Unknown Source) > > > > at > > > > > org.apache.cloudstack.api.command.admin.host.DeleteHostCmd.execute(DeleteHostCmd.java:88) > > > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > > > at > > > > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > > > at > > > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > > > at > > > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > > > at > > org.eclipse.jetty.io > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > > > at > > > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > 2021-04-14 13:29:21,971 DEBUG [c.c.u.d.T.Transaction] > > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) Rolling > > back the transaction: Time = 19 Name = qtp1644231115-2157; called by > > > > > -TransactionLegacy.rollback:888-TransactionLegacy.removeUpTo:831-TransactionLegacy.close:655-Transaction.execute:38-Transaction.execute:47-ResourceManagerImpl.doDeleteHost:873-ResourceManagerImpl.deleteHost:967-NativeMethodAccessorImpl.invoke0:-2-NativeMethodAccessorImpl.invoke:62-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:566-AopUtils.invokeJoinpointUsingReflection:344 > > > > 2021-04-14 13:29:21,974 ERROR [c.c.a.ApiServer] > > (qtp1644231115-2157:ctx-6167b69c ctx-f0d83d44) (logid:11607e2f) unhandled > > exception executing api command: [Ljava.lang.String;@38e732da > > > > java.lang.NullPointerException > > > > at > > > > > com.cloud.resource.ResourceManagerImpl$1.doInTransactionWithoutResult(ResourceManagerImpl.java:926) > > > > at > > > > > com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25) > > > > at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50) > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:40) > > > > at com.cloud.utils.db.Transaction.execute(Transaction.java:47) > > > > at > > > > > com.cloud.resource.ResourceManagerImpl.doDeleteHost(ResourceManagerImpl.java:873) > > > > at > > > > > com.cloud.resource.ResourceManagerImpl.deleteHost(ResourceManagerImpl.java:967) > > > > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > > Method) > > > > at > > > > > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > at > > > > > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > > > > at > > > > > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) > > > > at > > > > > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) > > > > at > > > > > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) > > > > at > > > > > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) > > > > at com.sun.proxy.$Proxy188.deleteHost(Unknown Source) > > > > at > > > > > org.apache.cloudstack.api.command.admin.host.DeleteHostCmd.execute(DeleteHostCmd.java:88) > > > > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156) > > > > at com.cloud.api.ApiServer.queueCommand(ApiServer.java:764) > > > > at com.cloud.api.ApiServer.handleRequest(ApiServer.java:588) > > > > at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:321) > > > > at com.cloud.api.ApiServlet$1.run(ApiServlet.java:134) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102) > > > > at > > > > > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52) > > > > at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:131) > > > > at com.cloud.api.ApiServlet.doGet(ApiServlet.java:93) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:645) > > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:750) > > > > at > > > > > org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1386) > > > > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:755) > > > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:547) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) > > > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) > > > > at > > > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) > > > > at > > > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) > > > > at > > > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > > > > at > > > > > org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:767) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) > > > > at > > > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) > > > > at org.eclipse.jetty.server.Server.handle(Server.java:500) > > > > at > > > org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) > > > > at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) > > > > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) > > > > at > > > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) > > > > at > > org.eclipse.jetty.io > > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) > > > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) > > > > at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) > > > > at > > > > > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) > > > > at > > > > > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) > > > > at > > > > > org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) > > > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > On Wed, Apr 14, 2021 at 6:58 PM Nicolas Vazquez < > > nicolas.vazq...@shapeblue.com> wrote: > > > > > Hi Hean, > > > > > > What does your management server logs show after the listHosts and the > > > deleteHost API calls? > > > > > > > > > Regards, > > > > > > Nicolas Vazquez > > > > > > ________________________________ > > > From: Hean Seng <heans...@gmail.com> > > > Sent: Tuesday, April 13, 2021 4:47 PM > > > To: users@cloudstack.apache.org <users@cloudstack.apache.org> > > > Subject: Request failed with status code 530 > > > > > > Hi > > > > > > I added a host to a wrong cluster, deleted it and re-add to the > correct > > > cluster . After that found that duplicate host inside the cluster , > and > > > after delete it the duplicated record, it come with error > > > "Request failed with status code 530" > > > > > > MYSQL record ; > > > > > > 16 | lax-06 | Removed | Disabled | NULL > > > > > > 17 | lax-06 | Up | Enabled | 2130706689 > > > > > > > > > Duplicated Host deleted is id=16 , but now the error happens is 17. > > > > > > In API : > > > > > > cmk listHosts id=17 > > > > > > 🙈 Error: (HTTP 530, error code 9999) <nil> > > > > > > for 16 , is return empty record. > > > > > > API log show" > > > > > > 2021-04-13 19:41:41,395 INFO [a.c.c.a.ApiServer] > > > (qtp1644231115-1244:ctx-eba7bffd ctx-95331a28) (logid:1e346c12) > (userId=2 > > > accountId=2 sessionId=node01qfrx0iypf3ts1q1kf43fexp7u174) 127.0.0.1 -- > > GET > > > > > > > > > command=listHosts&id=17&response=json&sessionkey=X3J4ui0AxuwP2zbYqrTrZxHzYLg > > > 530 null > > > > > > I tried to forced delete host 17. but seems error also; > > > > > > cmk deletehost id=17 forced=true > > > > > > 🙈 Error: (HTTP 530, error code 9999) <nil> > > > > > > > > > Any one help to give me idea of what's going on, and how to solve > > > this problem > > > > > > > > > > > > > > > -- > > > Regards, > > > Hean Seng > > > > > > nicolas.vazq...@shapeblue.com > > > www.shapeblue.com<http://www.shapeblue.com> > > > 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK > > > @shapeblue > > > > > > > > > > > > > > > > -- > > Regards, > > Hean Seng > > > > nicolas.vazq...@shapeblue.com > > www.shapeblue.com<http://www.shapeblue.com> > > 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK > > @shapeblue > > > > > > > > > > -- > Regards, > Hean Seng > > nicolas.vazq...@shapeblue.com > www.shapeblue.com > 3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK > @shapeblue > > > > -- Regards, Hean Seng