[
https://issues.apache.org/jira/browse/BOOKKEEPER-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430751#comment-13430751
]
Sijie Guo commented on BOOKKEEPER-191:
--------------------------------------
> When an op is pushed onto the TopicOpQueue, the op is first put on the work
> queue for that topic, and then run if it is the only thing in the queue. If
> it is not the only thing in the queue it will wait. Then it will be run when
> another op(sync or async) finishes and runs popAndRunNext().
Yes. that is how TopicOpQueue works. But for performance consideration,
PersistOp takes optimistic mechanism: it assign a message seq id for the
message and call BookKeeper#asyncAddEntry to issue an addEntry operation. So
the next persist op doesn't need to wait the previous one to callback,
BookKeeper could guarantee its order and correctness.
Considering how persist op works, the only chance to issue change ledger op is
in the persist's callback when those on-the-fly addEntry operations finished.
And the time between planning changing ledger and issuing change-ledger op,
other persist op could be put in queue, so we have to queue them until
change-ledger op executed.
A detail explanation is described as below:
1) several persist ops are issued. the topicqueue is as below:
(persist 1)
(persist 2)
...
(persist K)
2) persist K executes. At this point, the previous K-1 requests has been
issued, but they might still not callback. K detects it is the last entry of
the ledger. But it could not issue change ledger op now, other wise the change
ledger op would be executed immediately after K send addEntry request. It is
dangerous since all K requests might still wait for the callback from
BookKeeper.
so the only chance to issue change ledger op is when persist K is callback
(BookKeeper guarantees all previous K-1 has succeed when K succeed).
(persist 1) => addEntry
(persist 2) => addEntry
...
(persist K) <= executed now
(change ledger op) (dangerous, if issued immediately by persist K)
3) so the change ledger op is issued when persist K callback. during this time
period, other persist ops might still be put in the op queue.
(persist K) => addEntry
...
(persist N)
(persist M)
(change ledger op) (issued by callback of persist K)
so we have to queue those persist requests added between issuing addEntry(K)
and addEntry(K) callback.
hope the explanation works for you.
> Hub server should change ledger to write, so consumed messages have chance to
> be garbage collected.
> ---------------------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-191
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-191
> Project: Bookkeeper
> Issue Type: Improvement
> Affects Versions: 4.1.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.2.0
>
> Attachments: BK-191.diff, BOOKKEEPER-191.diff, BOOKKEEPER-191.diff
>
>
> currently, hub server write entries to only one ledger, if a topic doesn't
> change ownership, all entries will be added to this ledger. so those consumed
> messages don't have chance to be garbage collected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
