[
https://issues.apache.org/jira/browse/BOOKKEEPER-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030828#comment-16030828
]
ASF GitHub Bot commented on BOOKKEEPER-944:
-------------------------------------------
Github user eolivelli commented on the issue:
https://github.com/apache/bookkeeper/pull/108
@reddycharan this patch has now some conflict,
can you rebase and address @sijie comments so that we can get this merged ?
> Multiple issues and improvements to BK Compaction.
> --------------------------------------------------
>
> Key: BOOKKEEPER-944
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-944
> Project: Bookkeeper
> Issue Type: Improvement
> Components: bookkeeper-server
> Affects Versions: 4.4.0
> Reporter: Venkateswararao Jujjuri (JV)
> Assignee: Venkateswararao Jujjuri (JV)
>
> We have identified multiple issues with BK compaction.
> This issue is to list all of them in one Jira ticket.
> 1.
> MajorCompaction and MinorCompaction are very basic. Either they do it or
> won’t do it. Proposal is to add Low Water Mark(LWM) and High Water Mark(HWM)
> to the disk space. Have different compaction frequency and re-claim %s when
> the disk space is < low water mark , > LWM < HWM, > HWM.
> 2.
> MajorCompaction and Minor Compactions are strictly frequency based. They
> should at least time of the day based, and also run during low system load,
> and if the system load raises, reduce the compaction depending on the disk
> availability
> 3.
> Current code disables compaction when disk space grows beyond configured
> threshold. There is no exit from this point. Have an option to keep reserved
> space for compaction, at least 2 entryLog file sizes when
> isForceGCAllowWhenNoSpace enabled.
> 4.
> Current code toggles READONLY status of the bookie as soon as it falls below
> the disk storage threshold. Imagine if we keep 95% as the threshold, Bookie
> becomes RW as soon as it falls below 95 % and few more writes pushes it above
> 95 and it turns back to RONLY. Use a set of defines (another set of LWM/HWM?)
> where Bookie turns RO on high end and won't become RW until it hits low end.
> 5.
> Current code never checks if the compaction is enabled or disabled once the
> major/minor compaction is started. If the bookie goes > disk threshold (95%)
> and at that compaction is going on, it never checks until it finishes but
> there may not be disk available for compaction to take place. So check if
> compaction is enabled after processing every EntryLog.
> 6.
> Current code changes the Bookie Cookie value even when new storage is added.
> When the cookie changes Bookie becomes a new one, and BK cluster treats it as
> new bookie. If we have mechanism to keep valid cookie even after adding
> additional disk space, we may have a chance to bring the bookie back to
> healthy mode and have compaction going.
> 7. Bug
> CheckPoint was never attempted to complete after once sync failure. There is
> a TODO in the code for this area.
> 8.
> When the disk is above threshold, Bookie goes to RO. If we have to restart
> the bookie, on the way back, bookie tries to create new entrylog and other
> files, which will fail because disk usage is above threshold, hence bookie
> refuses to come up.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
