lichuan6 opened a new issue #9491: URL: https://github.com/apache/pulsar/issues/9491
**Describe the bug** I have a producer which produces logs to `public/logging/app` and a consumer which consumes the messages and sinks to elaticsearch. Occasionally, for debug purpose, I delete the topic or namespace using `pulsar-admin topics delete public/logging/app --force` or `pulsar-admin namespaces delete public/logging --force`. The reason why I use `--force` is that I cannot delete them when there're still consumers or producers for the topic: ``` 09:20:03.977 [AsyncHttpClient-7-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://pulsar-broker.pulsar:8080/admin/v2/persistent/public/logging/app?force=false&deleteSchema=false] Failed to perform http delete request: javax.ws.rs.ClientErrorException: HTTP 412 Precondition Failed Topic has active producers/subscriptions Reason: Topic has active producers/subscriptions command terminated with exit code 1 ``` After trying few times of force delete , I find out I can't delete topics any more: ``` 08:56:53.184 [AsyncHttpClient-7-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [http://pulsar-broker.pulsar:8080/admin/v2/persistent/public/logging/app?force=true&deleteSchema=false] Failed to perform http delete request: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error HTTP 500 Internal Server Error Reason: HTTP 500 Internal Server Error command terminated with exit code 1 ``` Here's the broker log ``` 09:01:44.460 [pulsar-web-45-4] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [admin] Failed to delete topic forcefully persistent://public/logging/app java.util.concurrent.ExecutionException: org.apache.pulsar.broker.service.BrokerServiceException$TopicBusyException: Topic has -13 connected producers/consumers at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) ~[?:1.8.0_282] at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalDeleteTopicForcefully(PersistentTopicsBase.java:374) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalDeleteTopic(PersistentTopicsBase.java:814) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at org.apache.pulsar.broker.admin.v2.PersistentTopics.deleteTopic(PersistentTopics.java:444) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?] at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1633) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:58) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.apache.pulsar.broker.web.AuthenticationFilter.doFilter(AuthenticationFilter.java:82) ~[org.apache.pulsar-pulsar-broker-common-2.6.3.jar:2.6.3] at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:561) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[org.eclipse.jetty-jetty-servlet-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) ~[org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) [org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) [org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [org.eclipse.jetty-jetty-server-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [org.eclipse.jetty-jetty-io-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [org.eclipse.jetty-jetty-io-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [org.eclipse.jetty-jetty-io-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar:9.4.33.v20201020] at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [org.eclipse.jetty-jetty-util-9.4.33.v20201020.jar:9.4.33.v20201020] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_282] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_282] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.48.Final.jar:4.1.48.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282] Caused by: org.apache.pulsar.broker.service.BrokerServiceException$TopicBusyException: Topic has -13 connected producers/consumers at org.apache.pulsar.broker.service.persistent.PersistentTopic.lambda$delete$22(PersistentTopic.java:940) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:683) ~[?:1.8.0_282] at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2010) ~[?:1.8.0_282] at org.apache.pulsar.broker.service.persistent.PersistentTopic.delete(PersistentTopic.java:880) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at org.apache.pulsar.broker.service.persistent.PersistentTopic.deleteForcefully(PersistentTopic.java:829) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] at org.apache.pulsar.broker.service.BrokerService.deleteTopic(BrokerService.java:777) ~[org.apache.pulsar-pulsar-broker-2.6.3.jar:2.6.3] ... 69 more 09:01:44.464 [pulsar-web-45-4] INFO org.eclipse.jetty.server.RequestLog - 172.24.170.210 - - [05/Feb/2021:09:01:44 +0000] "DELETE /admin/v2/persistent/public/logging/app?force=true&deleteSchema=false HTTP/1.1" 500 7353 "-" "Pulsar-Java-v2.6.3" 6 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
