[ https://issues.apache.org/jira/browse/HBASE-23149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Busbey updated HBASE-23149: -------------------------------- Fix Version/s: (was: 1.4.11) 1.4.12 > hbase shouldPerformMajorCompaction logic is not correct > ------------------------------------------------------- > > Key: HBASE-23149 > URL: https://issues.apache.org/jira/browse/HBASE-23149 > Project: HBase > Issue Type: Bug > Components: Compaction > Affects Versions: 1.4.9 > Reporter: jackylau > Assignee: jackylau > Priority: Major > Fix For: 1.4.12 > > > we can know that the major compaction is skipping from the below > regionserver's log, but it is compacting that region. > and read the code and find it is not correct and i add mark "/*** ***/" > below > > public boolean shouldPerformMajorCompaction(final Collection<StoreFile> > filesToCompact) > throws IOException { > if (lowTimestamp > 0L && lowTimestamp < (now - mcTime)) { > if (filesToCompact.size() == 1) { > if (sf.isMajorCompaction() && (cfTTL == Long.MAX_VALUE || oldest < cfTTL)) { > float blockLocalityIndex = > sf.getHDFSBlockDistribution().getBlockLocalityIndex( > RSRpcServices.getHostname(comConf.conf, false)); > if (blockLocalityIndex < comConf.getMinLocalityToForceCompact()) > { result = true; } > else > { LOG.debug("Skipping major compaction of " + regionInfo + " because one > (major) compacted file only, oldestTime " + oldest + "ms is < TTL=" + cfTTL + > " and blockLocalityIndex is " + blockLocalityIndex + " (min " + > comConf.getMinLocalityToForceCompact() + ")"); } > } else if (cfTTL != HConstants.FOREVER && oldest > cfTTL) > { LOG.debug("Major compaction triggered on store " + regionInfo + ", because > keyvalues outdated; time since last major compaction " + (now - lowTimestamp) > + "ms"); result = true; } > } else > { LOG.debug("Major compaction triggered on store " + regionInfo + "; time > since last major compaction " + (now - lowTimestamp) + "ms"); } > result = true;{color:#de350b} */******here it is not right, it should be > move to the above *****/*{color} > } > return result; > } > > 2019-09-27 09:09:35,960 DEBUG [st129,16020,1568236573216_ChoreService_1] > compactions.RatioBasedCompactionPolicy: Skipping major compaction of > 100E_POINT_point_2ddata_z3_geom_GpsTime_v6,\x17,1568215725799.413a563092544e8df480fd601b2de71b. > because one (major) compacted file only, oldestTime 3758085589ms is < > TTL=9223372036854775807 and blockLocalityIndex is 1.0 (min 0.0) > 2019-09-27 09:09:35,961 DEBUG [st129,16020,1568236573216_ChoreService_1] > compactions.SortedCompactionPolicy: Selecting compaction from 1 store files, > 0 compacting, 1 eligible, 100 blocking > 2019-09-27 09:09:35,961 DEBUG [st129,16020,1568236573216_ChoreService_1] > regionserver.HStore: 413a563092544e8df480fd601b2de71b - d: Initiating major > compaction (all files) > 2019-09-27 09:09:35,961 DEBUG [st129,16020,1568236573216_ChoreService_1] > regionserver.CompactSplitThread: Large Compaction requested: > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4b5582f1; > Because: CompactionChecker requests major compaction; use default priority; > compaction_queue=(1:0), split_queue=0, merge_queue=0 > 2019-09-27 09:09:35,961 INFO > [regionserver/st129/10.3.72.129:16020-longCompactions-1568236575579] > regionserver.HRegion: Starting compaction on d in region > 100E_POINT_point_2ddata_z3_geom_GpsTime_v6,\x17,1568215725799.413a563092544e8df480fd601b2de71b. > 2019-09-27 09:09:35,961 INFO > [regionserver/st129/10.3.72.129:16020-longCompactions-1568236575579] > regionserver.HStore: Starting compaction of 1 file(s) in d of > 100E_POINT_point_2ddata_z3_geom_GpsTime_v6,\x17,1568215725799.413a563092544e8df480fd601b2de71b. > into > tmpdir=hdfs://st129:8020/hbase/data/default/100E_POINT_point_2ddata_z3_geom_GpsTime_v6/413a563092544e8df480fd601b2de71b/.tmp, > totalSize=5.1 G > 2019-09-27 09:09:35,961 DEBUG > [regionserver/st129/10.3.72.129:16020-longCompactions-1568236575579] > compactions.Compactor: Compacting > hdfs://st129:8020/hbase/data/default/100E_POINT_point_2ddata_z3_geom_GpsTime_v6/413a563092544e8df480fd601b2de71b/d/3b4080f9b6f149e1b0a476058c8564e6, > keycount=83914030, bloomtype=NONE, size=5.1 G, encoding=FAST_DIFF, > compression=SNAPPY, seqNum=2621061, earliestPutTs=1565788490371 -- This message was sent by Atlassian Jira (v8.3.4#803005)