[
https://issues.apache.org/jira/browse/ZOOKEEPER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16971779#comment-16971779
]
Fangmin Lv commented on ZOOKEEPER-3594:
---------------------------------------
[~hanm] this is the skip error txn feature I mentioned in your ACL skip Jira,
Vlad spent lots of effort to make this working, which proved to improve a lot
of efficiency in ZK for traffic pattern which may have lots of errors.
Vlad will upstream this feature soon, please help review.
> Ability to skip proposing requests with error transactions
> ----------------------------------------------------------
>
> Key: ZOOKEEPER-3594
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3594
> Project: ZooKeeper
> Issue Type: New Feature
> Components: quorum, server
> Affects Versions: 3.5.6
> Reporter: Vladimir Ivić
> Assignee: Vladimir Ivić
> Priority: Major
>
> Ensembles that have a high write request rate could be skipping proposing
> requests that contain errors instead of having the quorum vote on those
> transactions.
> For example, having a sizable write traffic that results in error
> transactions would be creating additional network and log disk space overhead
> for each of the requests that would only return errors to the client in the
> end (e.g. delete non-existing path, version mismatch, trying to create a node
> that already exists etc).
> Currently, there is no such logic in the ProposalRequestProcessor, every
> request that comes out of PrepRequestProcessor will be proposed to the
> quorum.
> Proposed solution workflow:
> * A client sends a new write request trying to setData on an non-existing
> node
> * The server accepts the request and sends it through the
> PrepRequestProcessor
> * PrepRequestProcessor detects the error and assigns the error transaction
> to the request
> * Between PrepRequestProcessor and ProposalRequestProcessor there is another
> processor named SkipRequestProcessor which sole responsibility is to decide
> will the request be forwarded or returned to the originating quorum peer
> (Leader or Learner).
> * The quorum peer waits for all previous requests to complete before the
> error request proceeds with echoing the error back to the client.
> Requirements:
> * We should be conservative about the use of ZXID. If the request generates
> an error transaction we don't want to increment the last proposed ZXID and
> cause any gaps in the log.
> * Request that are found to be invalid should be sent directly to the origin
> (either to the corresponding Learner or to the Leader directly) so there is
> exactly one roundtrip for each request with error transaction.
> * All the requests must preserve its order, the changes must be backwards
> compatible with the Zookeeper ordering guarantees.
> Challenges:
> * Skipping request without having them go through the proposal pipeline
> poses a challenge in preserving Zookeeper transaction order.
> * Avoiding any additional changes inside CommitProcessor if possible.
> * Have unified logic for all three different paths in which write requests
> can come through:
> ## Via Leader placed directly into the PrepRequestProcessor
> ## Via Follower where the request is forwarded to the leader and also passed
> to CommitProcessor to wait for COMMIT packet
> ## Via Observer, where the request is forwarded to the Leader and the
> Observer waits for the INFORM packet
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
