lhotari commented on PR #24300: URL: https://github.com/apache/pulsar/pull/24300#issuecomment-3607845169
> It seems that the best way to store ReplicatedSubscriptionsSnapshots is in something like an increasing backoff queue: the last one — the most recent, the next — plus `snapshotFrequencyMillis`, then `snapshotFrequencyMillis`^2, then `snapshotFrequencyMillis`^3, and so on. The oldest one (the head) should remain untouched until it is consumed. > > It also seems that the solution could be simplified around `ReplicatedSubscriptionSnapshotCache`, and delayed messages would be handled much better. If the PR is stale, I may reopen a new one :) @Ksnz That could be useful if time would be the only contributing factor. One detail that has been missed before (in the comments of this PR) is the number of entries between 2 different snapshots. Optimally, the cache would keep snapshots with equal number of entries between the snapshots. When there's a burst of messages to the topic, there could be a lot of messages between a single snapshot that happens once per second. When the snapshot cache fills up, I'd assume that the snapshot entry with the shortest distance to it's adjacent entries should be evicted. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
