Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Xiangying Meng
Share more information: Even for versions before 3.0.0, the approach doesn't assemble chunks 3, 4, and 5 together. Please note this line of code: ```java chunkedMsgCtx = chunkedMessagesMap.computeIfAbsent(msgMetadata.getUuid(), (key) -> ChunkedMessageCtx.get(totalChunks, chunk

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Xiangying Meng
Hi Zike, PR [0] has already fixed this bug and won't introduce compatibility issues. PR [1] is unnecessary and can be closed. However, I still greatly appreciate the information you provided. [0] https://github.com/apache/pulsar/pull/20948 [1] https://github.com/apache/pulsar/pull/21070 On Sat,

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Zike Yang
> Hi Zike You can see the processMessageChunk method of the ConsumerImpl. Ah. That seems like a regression bug introduced by https://github.com/apache/pulsar/pull/18511. I have pushed a PR to fix it: https://github.com/apache/pulsar/pull/21070 For the behavior before Pulsar 3.0.0. The consumer sh

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Xiangying Meng
>> Consumer receive: >1. SequenceID: 0, ChunkID: 0 >2. SequenceID: 0, ChunkID: 1 >3. SequenceID: 0, ChunkID: 0 // chunk ID out of order. Release this >chunk and recycle its `chunkedMsgCtx`. >4. SequenceID: 0, ChunkID: 1 // chunkedMsgCtx == null Release it. >5. SequenceID: 0, ChunkID: 2 // chunked

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Xiangying Meng
>Regarding this comment, can you explain how the consumer only receives m1? >Here, m1's and m2's uuid and msgId will be different(if we suffix with a >chunkingSessionId), although the sequence ID is the same. They have the same sequence ID, so the first chunk will be dropped because the sequence I

Re: [DISCUSS]PIP-295: Fixing Chunk Message Duplication Issue

2023-08-26 Thread Zike Yang
> Consumer receive: 1. SequenceID: 0, ChunkID: 0 2. SequenceID: 0, ChunkID: 1 3. SequenceID: 0, ChunkID: 0 // chunk ID out of order. Release this chunk and recycle its `chunkedMsgCtx`. 4. SequenceID: 0, ChunkID: 1 // chunkedMsgCtx == null Release it. 5. SequenceID: 0, ChunkID: 2 // chunkedMsgCtx