[
https://issues.apache.org/jira/browse/RATIS-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17939024#comment-17939024
]
Tsz-wo Sze commented on RATIS-2273:
-----------------------------------
[~ivanandika], thanks a lot for filing and working on this JIRA!
bq. We need to write code about learner pull log, we can not reuse
GrpcLogAppender.
This may be a pros: since the new code could download the physical log files.
Then, it can avoid the serialization and deserialization cost.
Also, we could use netty directly and avoid gRPC (and its buffer copying). We
could borrow the netty code from Ratis Streaming.
> Support Ratis listener sync from non-leader
> -------------------------------------------
>
> Key: RATIS-2273
> URL: https://issues.apache.org/jira/browse/RATIS-2273
> Project: Ratis
> Issue Type: Improvement
> Components: server
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> RATIS-1298 discussed the possibility and designs of having a Ratis listener
> which listens from non-leader (e.g. follower or another listeners). The idea
> is that since Ratis listener will never be a voting-member, it is safe to
> just replicate the Raft logs of any follower up to the commitIndex.
> This can be useful in the case of HDDS-12307 for asynchronous replication
> from non-leader node since it doesn't require the most up-to-date state.
> Few modified ideas from RATIS-1298 (credits to the contributors that
> suggested the original ideas):
> * Pull-based approach: Listener pull log from follower
> ** Can be round-robin
> ** Or we can pick a single follower to subscribe from. In case of follower
> failure, the listener will pick another follower.
> ** This requires some kind of subscription API to pull committed logs from a
> single Raft server
> ** Pros & Cons
> *** Cons
> **** We need to write code about learner pull log, we can not reuse
> GrpcLogAppender.
> **** Also write a failover mechanism to detect follower crash / slowness and
> pick another follower
> *** Pros
> **** The voting-members do not need to be aware of the listener
> * Push-based approach: Follower push log to learner. When learner start, it
> register to one follower, then follower use GrpcLogAppender to append log to
> learner, if follower crash, learner need to register to other follower.
> ** Pros & Cons
> *** Cons: We need to consider the logic of register , especially when
> follower crash. And also need to consider follower become leader, we need to
> allocate listener to other followers.
> *** Pros: We can use GrpcLogAppender to append log.
> I'm leaning on the pull-based approach.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
