[
https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213817#comment-13213817
]
Jun Rao commented on KAFKA-46:
------------------------------
Some notes on the implementation:
1. If required_acks for a produce request is not 0 or 1, the commit thread will
put the request in a DelayedQueue and adds it to a watch list for that
topic/partition (similar implementation as long poll in kafka-48).
2. There will be an "offset watcher" per topic/partition. The watcher fires
everytime a follower in the in-sync set advances the offset. Every time the
watcher fires, it checks from the head of watcher list and see if any produce
request can be satisfied on this partition. If so, marks this partition as
satisfied, if all partitions of the produce requests are satisfied, dequeue the
request an send back the ack.
3. How to add/delete follower from in-sync set? We can run a background
insync-check thread that does the following:
for each topic/partition
get and save the offset of each partition of the leader replica
wait for KeepInSyncTime
check if any insync replica hasn't caught up to the saved offset, if so, drop
it out of insync set (need to fire the corresponding offset watcher)
check if any replica (not in insync set) has caught up to the saved offset,
if so, add it to insync set.
If we follow this approach, we can have 2 subtasks that implement "offset
watcher" and insync-check thread. We can also have 1 separate subtask for the
FetcherThread and another for the commit thread.
> Commit thread, ReplicaFetcherThread for intra-cluster replication
> -----------------------------------------------------------------
>
> Key: KAFKA-46
> URL: https://issues.apache.org/jira/browse/KAFKA-46
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Rao
> Assignee: Neha Narkhede
>
> We need to implement the commit thread at the leader and the fetcher thread
> at the follower for replication the data from the leader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
