oneby-wang commented on issue #24879: URL: https://github.com/apache/pulsar/issues/24879#issuecomment-3471009946
Sorry for my misleading. Race condition is more complicated in partitioned-topic auto deletion. Apparently I overlooked this key detail, it's a fan-out process(see still_happening3.txt). Race condition: ourproduct.v1.4e54a89c-2141-11f0-85b1-2e02ebf6218c-partition-0 and ourproduct.v1.4e54a89c-2141-11f0-85b1-2e02ebf6218c-partition-4 triggered partitioned-topic deletion at the same time. 1. ourproduct.v1.4e54a89c-2141-11f0-85b1-2e02ebf6218c-partition-0 fan-out connection num is 7(including itself) 2. ourproduct.v1.4e54a89c-2141-11f0-85b1-2e02ebf6218c-partition-4 fan-out connection num is 7(including itself) Did you set `brokerClient_connectionsPerBroker=15`? but I can only see 14 request failures in your log. Is there any http call not in the logs at the same time? In still_happening1.txt, still_happening2.txt and still_happening4.txt, I see some topics are triggered nearly at the same time(Oct 27 16:07:09~Oct 27 16:07:10), are these logs happens at the same time but separated into different files? Please set `brokerClient_connectionsPerBroker=0` before this issue is solved. If there are many partitioned-topics gc at the same time, it will also cause connection pool deadlock. Sorry for that again :). > I'm not sure if this case would be handled by your open PR or not. Yes, I think it will solve the http timeout exception. > I'm saying this because it may be a different code path for the "topic not loaded, trying to clean metadata" approach Seems this is kind of expect behavior before this issue is solved, because partition is deleted, broker owning may unload this partition. https://github.com/apache/pulsar/blob/39bb67542f2a7b849acaff681d408c693e1a2a18/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L1300-L1330 BTW, I'll try to fix this race condition to avoid multiple calls to partitioned-topic deletion admin api after this open PR is merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
