[VOTE] PIP 296: Introduce the `getLastMessageIds` API to Reader

2023-08-24 Thread Xiangying Meng
Hi Pulsar Community, This is the vote thread for PIP 296: https://github.com/apache/pulsar/pull/21052 This PIP will help to improve the flexibility of Reader usage. Thanks, Xiangying

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-24 Thread Xiangying Meng
>IIUC, this may change the existing behavior and may introduce inconsistencies. >Suppose that we have a large message with 3 chunks. But the producer >crashes and resends the message after sending the chunk-1. It will >send a total of 5 messages to the Pulsar topic: > >1. SequenceID: 0, ChunkID: 0

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-24 Thread Xiangying Meng
>> This solution also cannot solve the out-of-order messages inside the >>chunks. For example, the above five messages will still be persisted. >The consumer already handles this case. The above 5 messages will all >be persisted but the consumer will skip message 1 and 2. >For messages 3, 4, and 5.

Re: [DISCUSS] PIP-293: Add configuration for batching/chunking for retry letter and dead letter producers

2023-08-24 Thread Zike Yang
Hi, Rin Thanks for your PIP! Chunking/batching is the feature for the producer. It would be confusing if we introduce these concepts to the DeadLetterPolicy. Regarding the issue mentioned in this PIP. I have created a PR to try to fix it: https://github.com/apache/pulsar/pull/21048 I'm trying to

Re: [DISCUSS] PIP-281: Optimize Bundle Unload(Transfer) Protocol for ExtensibleLoadManager

2023-08-24 Thread PengHui Li
> The previous source broker information is persisted in the system topic with its state info. Upon receiving the Owned state via ownership transfer or recovery commands, the source broker will close the topics in the bundle(disconnect clients), if any. Oh, got it. It looks good. Thanks for the ex

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-24 Thread Zike Yang
Hi Heesung, > I believe in this PIP "similar to the existing "sequence ID map", to facilitate effective filtering" actually means tracking the last chunkId(not all chunk ids) on the broker side. With this simple solution, I think we don't need to track the (sequenceID, chunkID) on the broker side