[
https://issues.apache.org/jira/browse/KAFKA-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161218#comment-14161218
]
Guozhang Wang commented on KAFKA-1430:
--------------------------------------
Gone through the change list from May 6th (first patch submitted) to Aug.5th
(got committed). In the three month period there are 31 commits. I have looked
though all of them which may have code overlap with this one, and this seems
the only revert. But it would be great to borrow another pair of eyes.
Moving forward, since we are having more and more simultaneous patch commits I
would like to suggest that we think about code check in process for large
patches that may take more than a month to dev / review.
> Purgatory redesign
> ------------------
>
> Key: KAFKA-1430
> URL: https://issues.apache.org/jira/browse/KAFKA-1430
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.8.2
> Reporter: Jun Rao
> Assignee: Guozhang Wang
> Priority: Blocker
> Fix For: 0.8.2
>
> Attachments: KAFKA-1430.patch, KAFKA-1430.patch, KAFKA-1430.patch,
> KAFKA-1430.patch, KAFKA-1430.patch, KAFKA-1430_2014-06-05_17:07:37.patch,
> KAFKA-1430_2014-06-05_17:13:13.patch, KAFKA-1430_2014-06-05_17:40:53.patch,
> KAFKA-1430_2014-06-10_11:22:06.patch, KAFKA-1430_2014-06-10_11:26:02.patch,
> KAFKA-1430_2014-07-11_10:59:13.patch, KAFKA-1430_2014-07-21_12:53:39.patch,
> KAFKA-1430_2014-07-25_09:52:43.patch, KAFKA-1430_2014-07-28_11:30:23.patch,
> KAFKA-1430_2014-07-31_15:04:33.patch, KAFKA-1430_2014-08-05_14:54:21.patch
>
>
> We have seen 2 main issues with the Purgatory.
> 1. There is no atomic checkAndWatch functionality. So, a client typically
> first checks whether a request is satisfied or not and then register the
> watcher. However, by the time the watcher is registered, the registered item
> could already be satisfied. This item won't be satisfied until the next
> update happens or the delayed time expires, which means the watched item
> could be delayed.
> 2. FetchRequestPurgatory doesn't quite work. This is because the current
> design tries to incrementally maintain the accumulated bytes ready for fetch.
> However, this is difficult since the right time to check whether a fetch (for
> regular consumer) request is satisfied is when the high watermark moves. At
> that point, it's hard to figure out how many bytes we should incrementally
> add to each pending fetch request.
> The problem has been reported in KAFKA-1150 and KAFKA-703.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
