messere1 opened a new issue, #11354: URL: https://github.com/apache/rocketmq/issues/11354
### Before Creating the Bug Report - [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment OS: Windows 11 / Linux (broker module, lite topic path) ### RocketMQ version branch: develop Git commit id: 9ea2ccdc2b41445aee5e177e14e501d606f5a1ed (2026-10-08) ### JDK Version Compiler: Temurin JDK 17.0.19 ### Describe the Bug In exclusive mode, when clientB claims a liteTopic that clientA still holds, `LiteSubscriptionRegistryImpl.excludeClientByLmqName` evicts clientA and records a tombstone so clientA's stale pulls are rejected until its next full subscription sync: ```java exclusiveEvictionTombstones.add(clientGroup.clientId, lmqName); ``` A tombstone can be cleared in three ways: 1. the evicted client re-claims the liteTopic itself (`addPartialSubscription` calls `exclusiveEvictionTombstones.remove`); 2. the evicted client performs a full sync (`addCompleteSubscription` calls `removeStale` for liteTopics no longer in its set); 3. `removeCompleteSubscription` calls `removeAllOf` — invoked either by an explicit `COMPLETE_REMOVE` or by the expiry cleaner `cleanupExpiredSubscriptions`. However, when the eviction took the client's **last** liteTopic, its `LiteSubscription` is dropped from `client2Subscription` immediately (the same "remove client if no more liteTopic" branch). If that client then terminates without sending `COMPLETE_REMOVE` (crash, kill, instance replaced by a new clientId), none of the three paths can ever fire: - paths 1 and 2 need a request from the client, which never arrives; - path 3 via the expiry cleaner cannot fire because `cleanupExpiredSubscriptions` only walks `client2Subscription`, which no longer contains the evicted clientId. The tombstone entry therefore stays in `ExclusiveEvictionTombstones` until the broker restarts. The expiry cleaner even logs its size every check interval (`ExclusiveEvictionTombstones size: {}`), so on a broker with regular exclusive takeovers this is a grow-only counter. Impact: under exclusive-mode client churn (every takeover evicts the previous holder; autoscaling/rolling restarts produce a stream of clients that lose their last liteTopic and then go away), the tombstone set grows unboundedly — an O(number of takeovers) memory leak of `clientId$lmqName` keys held for the broker's lifetime. This is the same reachability gap as the evicted client's channel entry (issue #11352): the eviction path drops the subscription directly, making every cleaner that only walks `client2Subscription` blind to the remaining per-client state. ### Steps to Reproduce 1. Create a subscription group with `liteSubExclusive=true` bound to a LITE topic. 2. Start clientA and subscribe (PARTIAL_ADD) to liteTopic L — its only liteTopic. 3. Start clientB and subscribe to the same liteTopic L. clientA is evicted; since L was its last liteTopic, clientA is removed from `client2Subscription`, and a tombstone `(clientA, L)` is recorded. 4. clientA now terminates without sending `COMPLETE_REMOVE`. 5. Run the expiry cleaner (`cleanupExpiredSubscriptions`) any number of times: it walks `client2Subscription`, does not find clientA, and never clears the tombstone. `hasExclusiveEvictionTombstone(clientA, L)` keeps returning true forever. Unit-level reproduction: in `LiteSubscriptionRegistryImplTest`, perform the takeover of step 3, then call `registry.cleanupExpiredSubscriptions(...)` — `registry.hasExclusiveEvictionTombstone("clientA", "lmq1")` is still true afterwards even though clientA has had no subscription, no channel and no requests for arbitrarily long. ### What Did You Expect to See? Tombstones of clients that will never sync again should be bounded by the same horizon the broker already uses to consider a client gone: `liteSubscriptionCheckTimeoutMills`. A tombstone older than the check timeout belongs to a client that either re-subscribed (paths 1/2 already cleared it) or is gone for good, so the expiry cleaner should discard it. ### What Did You See Instead? Tombstones of evicted clients that never sync again are never reclaimed; the set grows without bound for the lifetime of the broker process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
