315157973 commented on issue #18128: URL: https://github.com/apache/pulsar/issues/18128#issuecomment-1302904213
> @315157973 Thanks for your proposal. I have a few concerns. > > 1. Should we check whether the managedLedger has been loaded into brokers before checking the retention policy? Otherwise, the leader needs to check all the managedLedgers > 2. If one topic has 10000 ledgers and only one ledger needs to be deleted, do we need to load and trigger trim? > 3. The topic retention policy has retention time and retention size, we also need to take retention size into consideration because some topics may only configure retention size. > 4. We have broker-level, namespace-level, and topic-level retention policies, how do we get each topic's retention policy before checking the ledger's retention in meta store > 5. If the user enabled tiered storage, we also need to take offloaded ledger's retention into consideration > 6. Due to the bundle being the basic unit of loading, If the leader loads topics and trims them one by one, we may load a lot of bundles into the leader and may bring more load to the load balance. 1. Broker judges whether it needs to be loaded according to the creation time and size in the metadata. If we only check the creation time, we don't need to scan all `ManagedLedgers` because they are ordered. Because we also check the size, we need to scan all. 2. Yes 3. I'll add it as a judgment item. A side effect is that all `ManagedLedgers` need to be scanned 4. When scanning, only the Broker-level Policy is used as a rough filter. The `trimLedger` method will load the full policy for judgment. 5. Since the `trimLedger` method of topic is reused, this scenario has been covered 6. I have changed to expose an admin API of `trimLedger`, Leader is only responsible for triggering, and the corresponding Broker is responsible for loading Topic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
