empiredan opened a new issue, #2197: URL: https://github.com/apache/incubator-pegasus/issues/2197
# Motivation Pegasus does not support duplicating atomic write requests including `incr`, `check_and_set` and `check_and_mutate` since they are not idempotent. In practice, various applications use atomic write interfaces in many scenarios. However, such applications cannot use duplication to synchronize data, and therefore cannot benefit from the high performance that duplication provides. # Design Due to the urgency of the requirements, the first version of the idempotent implementation for the atomic writes should be as simple as possible, without making fundamental changes to the write path. Therefore, we decided to implement the idempotence of atomic write requests as follows: for each replica, ensure that only one atomic write request is being processed in the write pipeline at any given time. Once the replica server receives an atomic request, firstly it will be cached. It will not be pushed into the write pipeline until all requests before it have been applied. The write pipeline consists of the following stages: 1. read the current value from RocksDB, calculate the final value according to specific semantics of requested atomic write and build the idempotent request based on it; 2. append the corresponding mutation to plog; 3. broadcast the prepare requests to the secondary replicas; 4. apply the final result back to RocksDB ultimately; The primary replicas have all 1 ~ 4 stages, and at last reply to the client while the secondary replicas only have stages 2 and 4. # Task List Making incr requests idempotent: - [ ] https://github.com/apache/incubator-pegasus/pull/2185 - [ ] https://github.com/apache/incubator-pegasus/pull/2192 - [ ] https://github.com/apache/incubator-pegasus/pull/2196 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
