[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267612#comment-14267612 ] Cosmin Lehene commented on HBASE-5532: -- [~terry_zhang], [~apurtell] this may not apply anymore. The code has changed a bit, and it seems we check for null readers. I'm not sure about the locking semantics. The logic has moved to StoreFile.java and there's related logic in StripeStoreFileManager isMajorCompaction() is in HStore.java > get NPE during MajorCompactionChecker > -- > > Key: HBASE-5532 > URL: https://issues.apache.org/jira/browse/HBASE-5532 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: terry zhang > Labels: delete > Attachments: HBASE-5532-v2.patch, HBASE-5532.patch > > > We found error log (NullPointerException) below on our online cluster: > 2012-03-05 00:17:09,592 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) > After Check the code we found although it already check whether store files > has null reader at the begin of the function(isMajorCompaction), but it still > has some possibility the reader is closed before it return(eg mini > compaction). So we need to check store file reader before we use it to avoid > this NPE -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226622#comment-13226622 ] stack commented on HBASE-5532: -- This looks like patch for 0.90 only? > get NPE during MajorCompactionChecker > -- > > Key: HBASE-5532 > URL: https://issues.apache.org/jira/browse/HBASE-5532 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: terry zhang > Attachments: HBASE-5532-v2.patch, HBASE-5532.patch > > > We found error log (NullPointerException) below on our online cluster: > 2012-03-05 00:17:09,592 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) > After Check the code we found although it already check whether store files > has null reader at the begin of the function(isMajorCompaction), but it still > has some possibility the reader is closed before it return(eg mini > compaction). So we need to check store file reader before we use it to avoid > this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[
https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224955#comment-13224955
]
terry zhang commented on HBASE-5532:
yes,Nicolas. This is a race condition and we can use lock to avoid this issue.
And now in the function toDetermines if Store should be split also didn't use
lock to protect.
{code:title=Store.java|borderStyle=solid}
/**
* Determines if Store should be split
* @return byte[] if store should be split, null otherwise.
*/
public byte[] getSplitPoint() {
this.lock.readLock().lock();
try {
// sanity checks
if (this.storefiles.isEmpty()) {
return null;
}
StoreFile.Reader r = sf.getReader(); * <= check if the reader is null*
if (r == null) {
LOG.warn("Storefile " + sf + " Reader is null");
continue;
}
long size = r.length();
if (size > maxSize) {
// This is the largest one so far
maxSize = size;
largestSf = sf;
}
}
StoreFile.Reader r = largestSf.getReader(); *<= check if the reader is
null*
if (r == null) {
LOG.warn("Storefile " + largestSf + " Reader is null");
return null;
}
// Get first, last, and mid keys. Midkey is the key that starts block
// in middle of hfile. Has column and timestamp. Need to return just
// the row we want to split on as midkey.
byte [] midkey = r.midkey();
.
return null;
}
{code}
> get NPE during MajorCompactionChecker
> --
>
> Key: HBASE-5532
> URL: https://issues.apache.org/jira/browse/HBASE-5532
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Reporter: terry zhang
> Attachments: HBASE-5532.patch
>
>
> We found error log (NullPointerException) below on our online cluster:
> 2012-03-05 00:17:09,592 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker:
> Caught exception
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878)
> at
> org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172)
> at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
> After Check the code we found although it already check whether store files
> has null reader at the begin of the function(isMajorCompaction), but it still
> has some possibility the reader is closed before it return(eg mini
> compaction). So we need to check store file reader before we use it to avoid
> this NPE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224761#comment-13224761 ] Nicolas Spiegelberg commented on HBASE-5532: Sounds like this is a race condition, so this patch wouldn't be sufficient. We need some way to lock the reader during this query. > get NPE during MajorCompactionChecker > -- > > Key: HBASE-5532 > URL: https://issues.apache.org/jira/browse/HBASE-5532 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: terry zhang > Attachments: HBASE-5532.patch > > > We found error log (NullPointerException) below on our online cluster: > 2012-03-05 00:17:09,592 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) > After Check the code we found although it already check whether store files > has null reader at the begin of the function(isMajorCompaction), but it still > has some possibility the reader is closed before it return(eg mini > compaction). So we need to check store file reader before we use it to avoid > this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223909#comment-13223909 ] terry zhang commented on HBASE-5532: Hi, Ted. Our is base on 0.90.2 (including many patches from 90.3 till 90.5). > get NPE during MajorCompactionChecker > -- > > Key: HBASE-5532 > URL: https://issues.apache.org/jira/browse/HBASE-5532 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: terry zhang > Attachments: HBASE-5532.patch > > > We found error log (NullPointerException) below on our online cluster: > 2012-03-05 00:17:09,592 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) > After Check the code we found although it already check whether store files > has null reader at the begin of the function(isMajorCompaction), but it still > has some possibility the reader is closed before it return(eg mini > compaction). So we need to check store file reader before we use it to avoid > this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5532) get NPE during MajorCompactionChecker
[ https://issues.apache.org/jira/browse/HBASE-5532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223291#comment-13223291 ] Zhihong Yu commented on HBASE-5532: --- Which version of HBase were you using ? > get NPE during MajorCompactionChecker > -- > > Key: HBASE-5532 > URL: https://issues.apache.org/jira/browse/HBASE-5532 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: terry zhang > Attachments: HBASE-5532.patch > > > We found error log (NullPointerException) below on our online cluster: > 2012-03-05 00:17:09,592 ERROR > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: > Caught exception > java.lang.NullPointerException > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:878) > at > org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:857) > at > org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3017) > at > org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1172) > at org.apache.hadoop.hbase.Chore.run(Chore.java:66) > After Check the code we found although it already check whether store files > has null reader at the begin of the function(isMajorCompaction), but it still > has some possibility the reader is closed before it return(eg mini > compaction). So we need to check store file reader before we use it to avoid > this NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
