On 2023/02/03 20:05:59 Heesung Sohn wrote: > For these internal requesters, > 1. A cursor be removed. > 2. Close the current ledger and create a new ledger. > 3. Consumer ack the message, the slowest cursor move forward. > > How do we prevent these callers from requesting duplicate requests for the > same resources(ledgers) (how do we handle racing conditions between the > requesters and consumers? ) These 3 case all invoke `trimConsumedLedgersInBackground`. ``` executor.executeOrdered(name, safeRun(() -> internalTrimLedgers(isTruncate, promise))); ``` The managed ledger only load by one broker. And the executor is thread-safe.
> It seems like there could be duplicate requests. Are we relying on these > methods to dedup messages? > 1. the idempotency of the deletion operations > 2. enough delay in the requesters' scan cycles > 3. enable dedup in the system topic There is a case that will cause duplicate requests. The ManagedLedger wants to delete ledger 1, after sending it to the system topic, the broker shut down and didn't persist the left ledgers to the metadata store. In the next trigger, ledger 1 will be sent to the system topic again. Here we use `the idempotency of the deletion operations` to handle it.
