Hi Xiangying I think Multiple-snapshots for TB is a good idea. And I have these questions:
> The number of the transactions in a snapshot can be configured, and we hope it is small, then we can merge the small snapshots into a large snapshot when it reaches a configured number. Does the merge take place in memory or in BK? - If we merge small-snapshot in memory, can we just use large-snapshot? - If we merge small-snapshot in BK, how to do it? > The index is written after each multiple-snapshot is written. Snapshot and index are stored in different topics, right? How do we ensure the atomicity of the two writes, I suggest adding a check mechanism that snapshot not recorded in the index is invalid. > #### Clean up unused aborts data Now, this section only has instructions for clear snapshots. I think we should add this: how to delete/override the index data. Thanks Yubiao Feng On Thu, Aug 4, 2022 at 10:27 AM Xiangying Meng <xiangy...@apache.org> wrote: > Hi, Pulsar community, > I`d like to start a discussion about transaction multiple-snapshot. > In order to get rid of the capacity limitation of the bookkeeper entry, we > plan to use multiple snapshots. More details can be found here > <https://github.com/apache/pulsar/issues/16913>. > > Yours sincerely, > Xiangying Meng >
