[jira] [Commented] (HBASE-9365) Optionally limit major compactions to off peak hours
[ https://issues.apache.org/jira/browse/HBASE-9365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753913#comment-13753913 ] Lars Hofhansl commented on HBASE-9365: -- Heh. [~jesse_yates] was saying the same thing. I'll see what I can do. > Optionally limit major compactions to off peak hours > > > Key: HBASE-9365 > URL: https://issues.apache.org/jira/browse/HBASE-9365 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > We already have off peak hours (where we set a more aggressive compaction > ratio) and periodic major compactions. > It would be nice if we could limit this to off peak hours as well. > A major compaction can be triggered in three ways: > # a periodic chore checking every ~3h (10.000.000ms) by default > # a minor compaction promoted to major because the last major compaction was > too long ago > # a minor compaction promoted to major (at the HStore level), because it > touched all HFiles anyway > For case #1 and #2 we could could optionally return false from > Store.isMajorCompaction(...) when we're not in the off peak window. Would > have a new config option to enforce that only optionally. > In case #3 we're compacting all files anyway, so not need to interfere with > that. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9365) Optionally limit major compactions to off peak hours
[ https://issues.apache.org/jira/browse/HBASE-9365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753459#comment-13753459 ] Andrew Purtell commented on HBASE-9365: --- Can we make the decision when to run major compaction pluggable? Then provide the current decisionmaking as default. Then introduce the suggested policy in this issue. > Optionally limit major compactions to off peak hours > > > Key: HBASE-9365 > URL: https://issues.apache.org/jira/browse/HBASE-9365 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > We already have off peak hours (where we set a more aggressive compaction > ratio) and periodic major compactions. > It would be nice if we could limit this to off peak hours as well. > A major compaction can be triggered in three ways: > # a periodic chore checking every ~3h (10.000.000ms) by default > # a minor compaction promoted to major because the last major compaction was > too long ago > # a minor compaction promoted to major (at the HStore level), because it > touched all HFiles anyway > For case #1 and #2 we could could optionally return false from > Store.isMajorCompaction(...) when we're not in the off peak window. Would > have a new config option to enforce that only optionally. > In case #3 we're compacting all files anyway, so not need to interfere with > that. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9365) Optionally limit major compactions to off peak hours
[ https://issues.apache.org/jira/browse/HBASE-9365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752816#comment-13752816 ] Lars Hofhansl commented on HBASE-9365: -- AFAIK, they are running major compactions manually; so probably not (haven't checked the code). The of peak stuff is in both fb and 0.94+ The patch here would be simple, actually, just a check for the already off peak hours in some more places. > Optionally limit major compactions to off peak hours > > > Key: HBASE-9365 > URL: https://issues.apache.org/jira/browse/HBASE-9365 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > We already have off peak hours (where we set a more aggressive compaction > ratio) and periodic major compactions. > It would be nice if we could limit this to off peak hours as well. > A major compaction can be triggered in three ways: > # a periodic chore checking every ~3h (10.000.000ms) by default > # a minor compaction promoted to major because the last major compaction was > too long ago > # a minor compaction promoted to major (at the HStore level), because it > touched all HFiles anyway > For case #1 and #2 we could could optionally return false from > Store.isMajorCompaction(...) when we're not in the off peak window. Would > have a new config option to enforce that only optionally. > In case #3 we're compacting all files anyway, so not need to interfere with > that. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9365) Optionally limit major compactions to off peak hours
[ https://issues.apache.org/jira/browse/HBASE-9365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752766#comment-13752766 ] stack commented on HBASE-9365: -- Is this facility out in the fb branch? > Optionally limit major compactions to off peak hours > > > Key: HBASE-9365 > URL: https://issues.apache.org/jira/browse/HBASE-9365 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl > > We already have off peak hours (where we set a more aggressive compaction > ratio) and periodic major compactions. > It would be nice if we could limit this to off peak hours as well. > A major compaction can be triggered in three ways: > # a periodic chore checking every ~3h (10.000.000ms) by default > # a minor compaction promoted to major because the last major compaction was > too long ago > # a minor compaction promoted to major (at the HStore level), because it > touched all HFiles anyway > For case #1 and #2 we could could optionally return false from > Store.isMajorCompaction(...) when we're not in the off peak window. Would > have a new config option to enforce that only optionally. > In case #3 we're compacting all files anyway, so not need to interfere with > that. > Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
