mattisonchao commented on issue #25041:
URL: https://github.com/apache/pulsar/issues/25041#issuecomment-3637940931
Hi @TakaHiR07
Thanks for reporting the issue. The context you provided is very clear to
me.
> However, with the pr's change, although update partition is not complete,
client regard it as success and try to construct new connection to the new
topic partition. That is wrong.
Theoretically, the partition topic metadata should be the key point and a
required field to indicate success in updating partition numbers. Other steps
are just trying our best to help the client side.
In the simplest condition, we could even update the metadata itself and
return it to the client. All the other resources should be lazily created.
- Topic should be auto-created.
- Subscription should be auto-created with the correct init position.
However, above all just the ideal situation.
> There is another high risk situation, if update topic metadata and create
missing partitions succeed, but fail on create subscription. In this moment,
producer can send new msg to new partition, but after a while consumer may auto
create subscription on new partition, and the read position is latest. That
would result in consumer lost some message. It is very dangerous.
Yes, that's my fault. I didn't carefully consider the client-side situation,
which led to missing the client's auto-created consumer subscription position.
Back to the fixes, I would prefer to change the init position of the
consumer partition auto-creation to "earliest". This approach is clearer, and
no dirty data is left due to the interruption of the operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
