[
https://issues.apache.org/jira/browse/PHOENIX-3797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030388#comment-16030388
]
Lars Hofhansl commented on PHOENIX-3797:
----------------------------------------
Thanks [~an...@apache.org].
So if understand this correctly the issue happens when HBCK merges two regions.
Then we'll two sets of local index rows in the L#0 column family right, so as
we read and rewrite we might find a previous key following the later start key
of the 2nd region and hence get the out of order problem. Your solution is to
not have the compaction write write the local index stuff, but instead write it
in smaller batches that themselves will not overlap, directly through the
region.
Is that about right?
If would certainly be nice if we could somehow leave the compaction writer do
the work, but I do not see how we can do the scanning then. Perhaps one
solution is to wrap two scanner inside a scanner wrapper and scan from both
region start keys. In that way we could get the re-written keys in the correct
order and leave the writing to the compation writer - just thinking aloud here.
> Local Index - Compaction fails on table with local index due to
> non-increasing bloom keys
> -----------------------------------------------------------------------------------------
>
> Key: PHOENIX-3797
> URL: https://issues.apache.org/jira/browse/PHOENIX-3797
> Project: Phoenix
> Issue Type: Bug
> Environment: Head of 4.x-HBase-0.98 with PHOENIX-3796 patch applied.
> HBase 0.98.23-hadoop2
> Reporter: Mujtaba Chohan
> Assignee: Ankit Singhal
> Priority: Blocker
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3797.patch, PHOENIX-3797_v2.patch
>
>
> Compaction fails on table with local index.
> {noformat}
> 2017-04-19 16:37:56,521 ERROR
> [RS:0;host:59455-smallCompactions-1492644947594]
> regionserver.CompactSplitThread: Compaction failed Request =
> regionName=FHA,00Dxx0000001gES005001xx000003DGPd,1492644985470.92ec6436984981cdc8ef02388005a957.,
> storeName=L#0, fileCount=3, fileSize=44.4 M (23.0 M, 10.7 M, 10.8 M),
> priority=7, time=7442973347247614
> java.io.IOException: Non-increasing Bloom keys:
> 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01H+&\xA1(00Dxx0000001gER001001xx000003DGPb01739544DCtf
> after
> 00Dxx0000001gES005001xx000003DGPd\x00\x00\x80\x00\x01I+\xF4\x9Ax00Dxx0000001gER001001xx000003DGPa017115434KTM
>
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.appendGeneralBloomfilter(StoreFile.java:960)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:996)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:428)
> at
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.compact(Compactor.java:276)
> at
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:64)
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:121)
> at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1154)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1559)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.doCompaction(CompactSplitThread.java:502)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:540)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
