[ https://issues.apache.org/jira/browse/HBASE-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack reassigned HBASE-5920: ---------------------------- Assignee: Derek Wollenstein > New Compactions Logic can silently prevent user-initiated compactions from > occurring > ------------------------------------------------------------------------------------ > > Key: HBASE-5920 > URL: https://issues.apache.org/jira/browse/HBASE-5920 > Project: HBase > Issue Type: Bug > Components: client, regionserver > Affects Versions: 0.92.1 > Reporter: Derek Wollenstein > Assignee: Derek Wollenstein > Priority: Minor > Labels: compaction > Attachments: 5290-094.txt, HBASE-5920-0.92.1-1.patch, > HBASE-5920-0.92.1-2.patch, HBASE-5920-0.92.1.patch, HBASE-5920-trunk-1.patch, > HBASE-5920-trunk.patch > > > There seem to be some tuning settings in which manually triggered major > compactions will do nothing, including loggic > From Store.java in the function > List<StoreFile> compactSelection(List<StoreFile> candidates) > When a user manually triggers a compaction, this follows the same logic as a > normal compaction check. when a user manually triggers a major compaction, > something similar happens. Putting this all together: > 1. If a user triggers a major compaction, this is checked against a max files > threshold (hbase.hstore.compaction.max). If the number of storefiles to > compact is > max files, then we downgrade to a minor compaction > 2. If we are in a minor compaction, we do the following checks: > a. If the file is less than a minimum size > (hbase.hstore.compaction.min.size) we automatically include it > b. Otherwise, we check how the size compares to the next largest size. > based on hbase.hstore.compaction.ratio. > c. If the number of files included is less than a minimum count > (hbase.hstore.compaction.min) then don't compact. > In many of the exit strategies, we aren't seeing an error message. > The net-net of this is that if we have a mix of very large and very small > files, we may end up having too many files to do a major compact, but too few > files to do a minor compact. > I'm trying to go through and see if I'm understanding things correctly, but > this seems like the bug > To put it another way > 2012-05-02 20:09:36,389 DEBUG > org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction > requested: > regionName=str,44594594594594592,1334939064521.f7aed25b55d4d7988af763bede9ce74e., > store > Name=c, fileCount=15, fileSize=1.5g (20.2k, 362.5m, 155.3k, 3.0m, 30.7k, > 361.2m, 6.9m, 4.7m, 14.7k, 363.4m, 30.9m, 3.2m, 7.3k, 362.9m, 23.5m), > priority=-9, time=3175046817624398; Because: Recursive enqueue; > compaction_queue=(59:0), split_queue=0 > When we had a minimum compaction size of 128M, and default settings for > hbase.hstore.compaction.min,hbase.hstore.compaction.max,hbase.hstore.compaction.ratio, > we were not getting a compaction to run even if we ran > major_compact > 'str,44594594594594592,1334939064521.f7aed25b55d4d7988af763bede9ce74e.' from > the ruby shell. Note that we had many tiny regions (20k, 155k, 3m, 30k,..) > and several large regions (362.5m,361.2m,363.4m,362.9m). I think the bimodal > nature of the sizes prevented us from doing a compaction. > I'm not 100% sure where this errored out because when I manually triggered a > compaction, I did not see > ' // if we don't have enough files to compact, just wait > if (filesToCompact.size() < this.minFilesToCompact) { > if (LOG.isDebugEnabled()) { > LOG.debug("Skipped compaction of " + this.storeNameStr > + ". Only " + (end - start) + " file(s) of size " > + StringUtils.humanReadableInt(totalSize) > + " have met compaction criteria."); > } > ' > being printed in the logs (and I know DEBUG logging was enabled because I saw > this elsewhere). > I'd be happy with better error messages when we decide not to compact for > user enabled compactions. > I'd also like to see some override that says "user triggered major compaction > always occurs", but maybe that's a bad idea for other reasons. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira