[ https://issues.apache.org/jira/browse/HBASE-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848585#comment-15848585 ]
huaxiang sun commented on HBASE-16981: -------------------------------------- Thanks [~jingcheng.du]. I was looking at the testing failure (not finished at that time) and I did not catch it. These failures are real since the logic in testing code to go back to past months are wrong, I corrected it. I also corrected testing logic for past weeks with monthly policy as it could be past month. Local run for TestHColumnDescriptor,TestMobCompactor,TestPartitionedMobCompactor passed. > Expand Mob Compaction Partition policy from daily to weekly, monthly and > beyond > ------------------------------------------------------------------------------- > > Key: HBASE-16981 > URL: https://issues.apache.org/jira/browse/HBASE-16981 > Project: HBase > Issue Type: New Feature > Components: mob > Affects Versions: 2.0.0 > Reporter: huaxiang sun > Assignee: huaxiang sun > Attachments: HBASE-16981.master.001.patch, > HBASE-16981.master.002.patch, HBASE-16981.master.003.patch, > HBASE-16981.master.004.patch, HBASE-16981.master.005.patch, > HBASE-16981.master.006.patch, HBASE-16981.master.007.patch, > HBASE-16981.master.008.patch, > Supportingweeklyandmonthlymobcompactionpartitionpolicyinhbase.pdf > > > Today the mob region holds all mob files for all regions. With daily > partition mob compaction policy, after major mob compaction, there is still > one file per region daily. Given there is 365 days in one year, at least 365 > files per region. Since HDFS has limitation for number of files under one > folder, this is not going to scale if there are lots of regions. To reduce > mob file number, we want to introduce other partition policies such as > weekly, monthly to compact mob files within one week or month into one file. > This jira is create to track this effort. -- This message was sent by Atlassian JIRA (v6.3.15#6346)