[ https://issues.apache.org/jira/browse/PHOENIX-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16029614#comment-16029614 ]
Ankit Singhal commented on PHOENIX-3797: ---------------------------------------- [~jamestaylor], Yes Local Index data is in right sorted order. Compaction was failing because :- We have a RepairScanner to handle the cases when two regions are merged during the hbck run(to repair overlaps or something) ,as these merges will be fine for data regions but can corrupt the data for local index(as we use start key of the region as suffix for local index to maintain the data within the region) Due to the bug(basically the typo), the local index files are indentified always inconsistent with respect to region boundaries resulting in this repair scanner to run everytime and which is failing with above exception.(because we are creating index mutation from data store and writing directly to local index hfiles). Attaching the fix for the same too, in case we want to pursue with this repair as well. > Local Index - Compaction fails on table with local index due to > non-increasing bloom keys > ----------------------------------------------------------------------------------------- > > Key: PHOENIX-3797 > URL: https://issues.apache.org/jira/browse/PHOENIX-3797 > Project: Phoenix > Issue Type: Bug > Environment: Head of 4.x-HBase-0.98 with PHOENIX-3796 patch applied. > HBase 0.98.23-hadoop2 > Reporter: Mujtaba Chohan > Assignee: Ankit Singhal > Priority: Blocker > Fix For: 4.11.0 > > Attachments: PHOENIX-3797.patch > > > Compaction fails on table with local index. > {noformat} > 2017-04-19 16:37:56,521 ERROR > [RS:0;host:59455-smallCompactions-1492644947594] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=FHA,00Dxx0000001gES005001xx000003DGPd,1492644985470.92ec6436984981cdc8ef02388005a957., > storeName=L#0, fileCount=3, fileSize=44.4 M (23.0 M, 10.7 M, 10.8 M), > priority=7, time=7442973347247614 > java.io.IOException: Non-increasing Bloom keys: > 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01H+&\xA1(00Dxx0000001gER001001xx000003DGPb01739544DCtf > after > 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01I+\xF4\x9Ax00Dxx0000001gER001001xx000003DGPa017115434KTM > > at > org.apache.hadoop.hbase.regionserver.StoreFile$Writer.appendGeneralBloomfilter(StoreFile.java:960) > at > org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:996) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:428) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:276) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:121) > at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1154) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1559) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:502) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:540) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:722) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)