[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13970: Component/s: regionserver > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang > Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2 > > Attachments: HBASE-13970-v1.patch, HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this on what is the reason behind it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-13970: --- Affects Version/s: 2.0.0 Fix Version/s: 2.0.0 > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this on what is the reason behind it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-13970: -- Attachment: HBASE-13970.patch A simple patch that add an increment AtomicInteger as the suffix of compaction name. [~ram_krish] Could you please test whether this patch works? Thanks. And also, thanks for the great digging. > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0 > > Attachments: HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this on what is the reason behind it. -- This
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-13970: -- Fix Version/s: 1.1.2 1.2.0 0.98.14 Affects Version/s: 1.1.1 1.2.0 0.98.13 Status: Patch Available (was: Open) > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.13, 2.0.0, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2 > > Attachments: HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-13970: -- Attachment: HBASE-13970-v1.patch Overflow is OK as what [~anoopsamjohn] said, we only need unique Strings. But yes negative counter is bit ugly. I rotated the value to 0 if we reached Integer.MAX_VALUE in the new patch. Thanks. > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2 > > Attachments: HBASE-13970-v1.patch, HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13970: Fix Version/s: (was: 1.2.0) 1.2.1 1.3.0 1.0.2 > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0, 0.98.14, 1.0.2, 1.1.2, 1.3.0, 1.2.1 > > Attachments: HBASE-13970-v1.patch, HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this on what is the reason behind it. -- This message was sent by Atlassian JIRA
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-13970: -- Resolution: Fixed Assignee: Duo Zhang (was: ramkrishna.s.vasudevan) Hadoop Flags: Reviewed Fix Version/s: (was: 1.0.2) Status: Resolved (was: Patch Available) Pushed to all branches except branch-1.0(HBASE-8329 has not applied to branch-1.0). Thanks [~anoopsamjohn] and [~ram_krish]. > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang > Fix For: 2.0.0, 0.98.14, 1.1.2, 1.3.0, 1.2.1 > > Attachments: HBASE-13970-v1.patch, HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMember
[jira] [Updated] (HBASE-13970) NPE during compaction in trunk
[ https://issues.apache.org/jira/browse/HBASE-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13970: Fix Version/s: (was: 1.2.1) (was: 1.3.0) 1.2.0 > NPE during compaction in trunk > -- > > Key: HBASE-13970 > URL: https://issues.apache.org/jira/browse/HBASE-13970 > Project: HBase > Issue Type: Bug >Affects Versions: 2.0.0, 0.98.13, 1.2.0, 1.1.1 >Reporter: ramkrishna.s.vasudevan >Assignee: Duo Zhang > Fix For: 2.0.0, 0.98.14, 1.2.0, 1.1.2 > > Attachments: HBASE-13970-v1.patch, HBASE-13970.patch > > > Updated the trunk.. Loaded the table with PE tool. Trigger a flush to ensure > all data is flushed out to disk. When the first compaction is triggered we > get an NPE and this is very easy to reproduce > {code} > 015-06-25 21:33:46,041 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,051 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Flushing 1/1 column families, memstore=76.91 MB > 2015-06-25 21:33:46,159 ERROR > [regionserver/stobdtserver3/10.224.54.70:16040-longCompactions-1435248183945] > regionserver.CompactSplitThread: Compaction failed Request = > regionName=TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4., > storeName=info, fileCount=3, fileSize=343.4 M (114.5 M, 114.5 M, 114.5 M), > priority=3, time=7536968291719985 > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController$ActiveCompaction.access$700(PressureAwareCompactionThroughputController.java:79) > at > org.apache.hadoop.hbase.regionserver.compactions.PressureAwareCompactionThroughputController.finish(PressureAwareCompactionThroughputController.java:238) > at > org.apache.hadoop.hbase.regionserver.compactions.Compactor.performCompaction(Compactor.java:306) > at > org.apache.hadoop.hbase.regionserver.compactions.DefaultCompactor.compact(DefaultCompactor.java:106) > at > org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext.compact(DefaultStoreEngine.java:112) > at > org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1202) > at > org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1792) > at > org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:524) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2015-06-25 21:33:46,745 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.DefaultStoreFlusher: Flushed, sequenceid=1534, memsize=76.9 M, > hasBloomFilter=true, into tmp file > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/.tmp/942ba0831a0047a08987439e34361a0c > 2015-06-25 21:33:46,772 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HStore: Added > hdfs://stobdtserver3:9010/hbase/data/default/TestTable/028fb0324cd6eb03d5022eb8c147b7c4/info/942ba0831a0047a08987439e34361a0c, > entries=68116, sequenceid=1534, filesize=68.7 M > 2015-06-25 21:33:46,773 INFO > [rs(stobdtserver3,16040,1435248182301)-flush-proc-pool3-thread-1] > regionserver.HRegion: Finished memstore flush of ~76.91 MB/80649344, > currentsize=0 B/0 for region > TestTable,283887,1435248198798.028fb0324cd6eb03d5022eb8c147b7c4. > in 723ms, sequenceid=1534, compaction requested=true > 2015-06-25 21:33:46,780 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/reached/TestTable > 2015-06-25 21:33:46,790 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received created > event:/hbase/flush-table-proc/abort/TestTable > 2015-06-25 21:33:46,791 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > 2015-06-25 21:33:46,803 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure start children changed > event: /hbase/flush-table-proc/acquired > 2015-06-25 21:33:46,818 INFO [main-EventThread] > procedure.ZKProcedureMemberRpcs: Received procedure abort children changed > event: /hbase/flush-table-proc/abort > {code} > Will check this on what is the reason behind it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)