[
https://issues.apache.org/jira/browse/HBASE-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037561#comment-15037561
]
Yu Li commented on HBASE-14906:
-------------------------------
>From the HadoopQA report, observe below failures (errors) although it says +1:
{noformat}
Tests in error:
org.apache.hadoop.hbase.regionserver.TestBulkLoad.bulkHLogShouldThrowErrorWhenFamilySpecifiedAndHFileExistsButNotInTableDescriptor(org.apache.hadoop.hbase.regionserver.TestBulkLoad)
Run 1:
TestBulkLoad.bulkHLogShouldThrowErrorWhenFamilySpecifiedAndHFileExistsButNotInTableDescriptor
�
Run 2:
TestBulkLoad.bulkHLogShouldThrowErrorWhenFamilySpecifiedAndHFileExistsButNotInTableDescriptor
�
Run 3:
TestBulkLoad.bulkHLogShouldThrowErrorWhenFamilySpecifiedAndHFileExistsButNotInTableDescriptor
�
org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath(org.apache.hadoop.hbase.regionserver.TestBulkLoad)
Run 1: TestBulkLoad.shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath �
Unexpected ex...
Run 2: TestBulkLoad.shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath �
Unexpected ex...
Run 3: TestBulkLoad.shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath �
Unexpected ex...
{noformat}
And detailed exception:
{noformat}
shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath(org.apache.hadoop.hbase.regionserver.TestBulkLoad)
Time elapsed: 0.043 sec <<< ERROR!
java.lang.Exception: Unexpected exception,
expected<org.apache.hadoop.hbase.DoNotRetryIOException> but
was<java.lang.ArithmeticException>
at
org.apache.hadoop.hbase.regionserver.FlushLargeStoresPolicy.configureForRegion(FlushLargeStoresPolicy.java:59)
at
org.apache.hadoop.hbase.regionserver.FlushPolicyFactory.create(FlushPolicyFactory.java:52)
at
org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:845)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:786)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6195)
at
org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6204)
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamiliesAndSpecifiedTableName(TestBulkLoad.java:239)
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.testRegionWithFamilies(TestBulkLoad.java:249)
at
org.apache.hadoop.hbase.regionserver.TestBulkLoad.shouldThrowErrorIfBadFamilySpecifiedAsFamilyPath(TestBulkLoad.java:207)
{noformat}
Since these are Errors not Failures, the test stop at the middle phase.
The issue is caused by the patch here since it doesn't handle the case that
column family number is zero, although this won't happen in real world, it's
possible in our unit test case like TestBulkload.
> Improvements on FlushLargeStoresPolicy
> --------------------------------------
>
> Key: HBASE-14906
> URL: https://issues.apache.org/jira/browse/HBASE-14906
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Yu Li
> Assignee: Yu Li
> Fix For: 2.0.0
>
> Attachments: HBASE-14906.patch, HBASE-14906.v2.patch
>
>
> When checking FlushLargeStoragePolicy, found below possible improving points:
> 1. Currently in selectStoresToFlush, we will do the selection no matter how
> many actual families, which is not necessary for one single family
> 2. Default value for hbase.hregion.percolumnfamilyflush.size.lower.bound
> could not fit in all cases, and requires user to know details of the
> implementation to properly set it. We propose to use
> "hbase.hregion.memstore.flush.size/column_family_number" instead:
> {noformat}
> <property>
> <name>hbase.hregion.percolumnfamilyflush.size.lower.bound</name>
> <value>16777216</value>
> <description>
> If FlushLargeStoresPolicy is used and there are multiple column families,
> then every time that we hit the total memstore limit, we find out all the
> column families whose memstores exceed a "lower bound" and only flush them
> while retaining the others in memory. The "lower bound" will be
> "hbase.hregion.memstore.flush.size / column_family_number" by default
> unless value of this property is larger than that. If none of the families
> have their memstore size more than lower bound, all the memstores will be
> flushed (just as usual).
> </description>
> </property>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
