[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-11-10 Thread wangyongqiang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998218#comment-14998218
 ] 

wangyongqiang commented on HBASE-12959:
---

you use hbase0.98.10, but this bug is fixed in hbase0.98.14, 
u do not meet this problem?


HBaseHBASE-13945
Prefix_Tree seekBefore() does not work correctly










>  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
> 
>
> Key: HBASE-12959
> URL: https://issues.apache.org/jira/browse/HBASE-12959
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 0.98.7
> Environment: hbase 0.98.7
> hadoop 2.5.1
>Reporter: wuchengzhi
>Priority: Critical
> Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
> txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
> txtfile-part6.txt.gz, txtfile-part7.txt.gz
>
>
> I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
> 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
> gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
> time.
> in the regionserver dump , there are some logs as below:
> Tasks:
> ===
> Task: Compacting info in 
> PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
> Status: RUNNING:Compacting store info
> Running for 1097s  (on the  site running more than 3 days)
> 
> Thread 197 (regionserver60020-smallCompactions-1421954341530):
>   State: RUNNABLE
>   Blocked count: 7
>   Waited count: 3
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
> 
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
> 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
> 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
> 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
> Thread 177 (regionserver60020-smallCompactions-1421954314809):
>   State: RUNNABLE
>   Blocked count: 40
>   Waited count: 60
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
> 
> 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-11-10 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14998232#comment-14998232
 ] 

Duo Zhang commented on HBASE-12959:
---

No, I haven't met this problem yet.

>  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
> 
>
> Key: HBASE-12959
> URL: https://issues.apache.org/jira/browse/HBASE-12959
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 0.98.7
> Environment: hbase 0.98.7
> hadoop 2.5.1
>Reporter: wuchengzhi
>Priority: Critical
> Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
> txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
> txtfile-part6.txt.gz, txtfile-part7.txt.gz
>
>
> I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
> 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
> gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
> time.
> in the regionserver dump , there are some logs as below:
> Tasks:
> ===
> Task: Compacting info in 
> PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
> Status: RUNNING:Compacting store info
> Running for 1097s  (on the  site running more than 3 days)
> 
> Thread 197 (regionserver60020-smallCompactions-1421954341530):
>   State: RUNNABLE
>   Blocked count: 7
>   Waited count: 3
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
> 
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
> 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
> 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
> 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
> Thread 177 (regionserver60020-smallCompactions-1421954314809):
>   State: RUNNABLE
>   Blocked count: 40
>   Waited count: 60
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
> 
> 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-11-09 Thread Duo Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997903#comment-14997903
 ] 

Duo Zhang commented on HBASE-12959:
---

I sent an email to [~bdifn] long time ago to confirm whether HBASE-12817 solve 
this problem but he/she haven't replied yet...

And PREFIX_TREE encoding has been used widely in our project for a long time. I 
haven't find any other problem after HBASE-12817.

Hope this could help. [~wang_yongqiang]

>  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
> 
>
> Key: HBASE-12959
> URL: https://issues.apache.org/jira/browse/HBASE-12959
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 0.98.7
> Environment: hbase 0.98.7
> hadoop 2.5.1
>Reporter: wuchengzhi
>Priority: Critical
> Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
> txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
> txtfile-part6.txt.gz, txtfile-part7.txt.gz
>
>
> I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
> 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
> gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
> time.
> in the regionserver dump , there are some logs as below:
> Tasks:
> ===
> Task: Compacting info in 
> PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
> Status: RUNNING:Compacting store info
> Running for 1097s  (on the  site running more than 3 days)
> 
> Thread 197 (regionserver60020-smallCompactions-1421954341530):
>   State: RUNNABLE
>   Blocked count: 7
>   Waited count: 3
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
> 
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
> 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
> 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
> 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
> Thread 177 (regionserver60020-smallCompactions-1421954314809):
>   State: RUNNABLE
>   Blocked count: 40
>   Waited count: 60
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
> 
> 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-11-09 Thread wangyongqiang (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14996402#comment-14996402
 ] 

wangyongqiang commented on HBASE-12959:
---

 Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 does anybody know if this problem is solved?  will this problem happen in 
hbase0.98.10


HBaseHBASE-12952, Seek with prefixtree may hang
and what about this problem, will this problem happen in hbase0.98.10


many thanks









>  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
> 
>
> Key: HBASE-12959
> URL: https://issues.apache.org/jira/browse/HBASE-12959
> Project: HBase
>  Issue Type: Bug
>  Components: hbase
>Affects Versions: 0.98.7
> Environment: hbase 0.98.7
> hadoop 2.5.1
>Reporter: wuchengzhi
>Priority: Critical
> Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
> txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
> txtfile-part6.txt.gz, txtfile-part7.txt.gz
>
>
> I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
> 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
> gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
> time.
> in the regionserver dump , there are some logs as below:
> Tasks:
> ===
> Task: Compacting info in 
> PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
> Status: RUNNING:Compacting store info
> Running for 1097s  (on the  site running more than 3 days)
> 
> Thread 197 (regionserver60020-smallCompactions-1421954341530):
>   State: RUNNABLE
>   Blocked count: 7
>   Waited count: 3
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
> 
> org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
> 
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
> 
> org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
> 
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
> 
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
> 
> org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
> 
> org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
> 
> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
> Thread 177 (regionserver60020-smallCompactions-1421954314809):
>   State: RUNNABLE
>   Blocked count: 40
>   Waited count: 60
>   Stack:
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
> 
> org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
> 
> 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-02-03 Thread wuchengzhi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304589#comment-14304589
 ] 

wuchengzhi commented on HBASE-12959:


I try to use that patch just now, but it seems not fix my issue .

  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 

 Key: HBASE-12959
 URL: https://issues.apache.org/jira/browse/HBASE-12959
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 0.98.7
 Environment: hbase 0.98.7
 hadoop 2.5.1
Reporter: wuchengzhi
Priority: Critical
 Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
 txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
 txtfile-part6.txt.gz, txtfile-part7.txt.gz


 I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
 gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
 time.
 in the regionserver dump , there are some logs as below:
 Tasks:
 ===
 Task: Compacting info in 
 PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
 Status: RUNNING:Compacting store info
 Running for 1097s  (on the  site running more than 3 days)
 
 Thread 197 (regionserver60020-smallCompactions-1421954341530):
   State: RUNNABLE
   Blocked count: 7
   Waited count: 3
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 
 org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
 Thread 177 (regionserver60020-smallCompactions-1421954314809):
   State: RUNNABLE
   Blocked count: 40
   Waited count: 60
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-02-03 Thread wuchengzhi (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304659#comment-14304659
 ] 

wuchengzhi commented on HBASE-12959:


Not yet,please help to fix it,thanks.

  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 

 Key: HBASE-12959
 URL: https://issues.apache.org/jira/browse/HBASE-12959
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 0.98.7
 Environment: hbase 0.98.7
 hadoop 2.5.1
Reporter: wuchengzhi
Priority: Critical
 Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
 txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
 txtfile-part6.txt.gz, txtfile-part7.txt.gz


 I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
 gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
 time.
 in the regionserver dump , there are some logs as below:
 Tasks:
 ===
 Task: Compacting info in 
 PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
 Status: RUNNING:Compacting store info
 Running for 1097s  (on the  site running more than 3 days)
 
 Thread 197 (regionserver60020-smallCompactions-1421954341530):
   State: RUNNABLE
   Blocked count: 7
   Waited count: 3
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 
 org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
 Thread 177 (regionserver60020-smallCompactions-1421954314809):
   State: RUNNABLE
   Blocked count: 40
   Waited count: 60
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-02-03 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304644#comment-14304644
 ] 

ramkrishna.s.vasudevan commented on HBASE-12959:


Okie, are you working on a patch for this?  If not I can take it up?

  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 

 Key: HBASE-12959
 URL: https://issues.apache.org/jira/browse/HBASE-12959
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 0.98.7
 Environment: hbase 0.98.7
 hadoop 2.5.1
Reporter: wuchengzhi
Priority: Critical
 Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
 txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
 txtfile-part6.txt.gz, txtfile-part7.txt.gz


 I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
 gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
 time.
 in the regionserver dump , there are some logs as below:
 Tasks:
 ===
 Task: Compacting info in 
 PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
 Status: RUNNING:Compacting store info
 Running for 1097s  (on the  site running more than 3 days)
 
 Thread 197 (regionserver60020-smallCompactions-1421954341530):
   State: RUNNABLE
   Blocked count: 7
   Waited count: 3
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 
 org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
 Thread 177 (regionserver60020-smallCompactions-1421954314809):
   State: RUNNABLE
   Blocked count: 40
   Waited count: 60
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-02-03 Thread zhangduo (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304717#comment-14304717
 ] 

zhangduo commented on HBASE-12959:
--

[~bdifn] Have you tried 0.98.10RC2? You can download it here.
http://people.apache.org/~apurtell/0.98.10RC2/

HBASE-12817 have fixed a prefix tree decoding issue and will be released with 
0.98.10.

Thanks~

  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 

 Key: HBASE-12959
 URL: https://issues.apache.org/jira/browse/HBASE-12959
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 0.98.7
 Environment: hbase 0.98.7
 hadoop 2.5.1
Reporter: wuchengzhi
Priority: Critical
 Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
 txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
 txtfile-part6.txt.gz, txtfile-part7.txt.gz


 I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
 gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
 time.
 in the regionserver dump , there are some logs as below:
 Tasks:
 ===
 Task: Compacting info in 
 PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
 Status: RUNNING:Compacting store info
 Running for 1097s  (on the  site running more than 3 days)
 
 Thread 197 (regionserver60020-smallCompactions-1421954341530):
   State: RUNNABLE
   Blocked count: 7
   Waited count: 3
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 
 org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
 Thread 177 (regionserver60020-smallCompactions-1421954314809):
   State: RUNNABLE
   Blocked count: 40
   Waited count: 60
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
 
 

[jira] [Commented] (HBASE-12959) Compact never end when table's dataBlockEncoding using PREFIX_TREE

2015-02-03 Thread ramkrishna.s.vasudevan (JIRA)

[ 
https://issues.apache.org/jira/browse/HBASE-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303077#comment-14303077
 ] 

ramkrishna.s.vasudevan commented on HBASE-12959:


Is this related to HBASE-12952?

  Compact never end when table's dataBlockEncoding using  PREFIX_TREE
 

 Key: HBASE-12959
 URL: https://issues.apache.org/jira/browse/HBASE-12959
 Project: HBase
  Issue Type: Bug
  Components: hbase
Affects Versions: 0.98.7
 Environment: hbase 0.98.7
 hadoop 2.5.1
Reporter: wuchengzhi
Priority: Critical
 Attachments: PrefixTreeCompact.java, txtfile-part1.txt.gz, 
 txtfile-part2.txt.gz, txtfile-part4.txt.gz, txtfile-part5.txt.gz, 
 txtfile-part6.txt.gz, txtfile-part7.txt.gz


 I upgraded the hbase from 0.96.1.1 to 0.98.7 and hadoop from 2.2.0 to 
 2.5.1,some table encoding using prefix-tree was abnormal for compacting,  the 
 gui shows the table's Compaction status is MAJOR_AND_MINOR(MAJOR) all the 
 time.
 in the regionserver dump , there are some logs as below:
 Tasks:
 ===
 Task: Compacting info in 
 PREFIX_NOT_COMPACT,,1421954285670.41ef60e2c221772626e141d5080296c5.
 Status: RUNNING:Compacting store info
 Running for 1097s  (on the  site running more than 3 days)
 
 Thread 197 (regionserver60020-smallCompactions-1421954341530):
   State: RUNNABLE
   Blocked count: 7
   Waited count: 3
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.followFan(PrefixTreeArrayScanner.java:329)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:149)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToOrBeforeUsingPositionAtOrAfter(PrefixTreeSeeker.java:199)
 
 org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeSeeker.seekToKeyInBlock(PrefixTreeSeeker.java:162)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$EncodedScannerV2.loadBlockAndSeekToKey(HFileReaderV2.java:1172)
 
 org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:573)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:173)
 
 org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:313)
 
 org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:257)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:697)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:683)
 
 org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:533)
 
 org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:222)
 
 org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:77)
 
 org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:110)
 org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1099)
 org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1482)
 Thread 177 (regionserver60020-smallCompactions-1421954314809):
   State: RUNNABLE
   Blocked count: 40
   Waited count: 60
   Stack:
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.column.ColumnReader.populateBuffer(ColumnReader.java:81)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateQualifier(PrefixTreeArrayScanner.java:471)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.populateNonRowFields(PrefixTreeArrayScanner.java:452)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.nextRow(PrefixTreeArrayScanner.java:226)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArrayScanner.advance(PrefixTreeArrayScanner.java:208)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtQualifierTimestamp(PrefixTreeArraySearcher.java:244)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.positionAtOrAfter(PrefixTreeArraySearcher.java:123)
 
 org.apache.hadoop.hbase.codec.prefixtree.decode.PrefixTreeArraySearcher.seekForwardToOrAfter(PrefixTreeArraySearcher.java:183)