Thanks for your review, and I am glad to add you into shepherds list.
Exactly-once involves a larger range, such as when we should submit consumption offset and store state data, and what it's behavior is when RocketMQ-streams recover from crash. This RIP only discusses a part of them, to achieve exactly-once how we should store and recover state with RocketMQ. When a message is found from a new shard, RocketMQ-streams load it's corresponding state.When shard removed because of rebalancing, RocketMQ-streams save state to RocketMQ and remove local. When making a checkpoint, firstly save state to RocketMQ, secondly commit state consumption offset, and at last, poll new state into local. When RocketMQ-streams crash between checkpoints, state data in RocketMQ exactly correspond to the last checkpoint, and state consumption offset corresponds to the penultimate checkpoint. That is to say, we can recover state from the last checkpoint when RocketMQ-streams restart, which is state behavior in exactly-once semantics. As for the complete exact-once implementation, I would like to discuss with another RIP. vongosling <[email protected]> 于2022年2月10日周四 08:55写道： > I like IStorage design, good to start. This is a good improvement for our > streams architecture. If you need another shepherd, you could count me :-) > > BTW, your design seems to not solve the exactly-once semantic in the > streaming scenario, or say, we miss some detailed comments in the doc? > > 倪泽 <[email protected]> 于2022年2月9日周三 17:38写道： > > > Hi, RocketMQ Community, > > > > First of all, Happy Chinese New Year! > > I want to start a RIP to replace mysql with RocketMQ as state store in > > RocketMQ-streams. Base the RocketMQ storage, I will provide a > exactly-once > > supported and efficient state storage scheme. > > > > I have written my proposal and you can click on the link below： > > > > > > > https://docs.google.com/document/d/1gmwU4rC5wyG07R55jW5oN4GzxHSq8QoFNdoNL61xT0M/edit?usp=sharing > > > > Chinese version： > > > > https://shimo.im/docs/pjDRKHqtdQC9RWCg > > > > If you have any questions or suggestions, please reply to this email or > > comment on the proposal. > > > > > > Thanks > > Nize > > > > > -- > Best Regards :-) >
