[
https://issues.apache.org/jira/browse/HBASE-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120819#comment-14120819
]
Hadoop QA commented on HBASE-11882:
-----------------------------------
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12666379/HBASE-11882-master-v3.patch
against trunk revision .
ATTACHMENT ID: 12666379
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/10707//console
This message is automatically generated.
> Row level consistency may not be maintained with bulk load and compaction
> -------------------------------------------------------------------------
>
> Key: HBASE-11882
> URL: https://issues.apache.org/jira/browse/HBASE-11882
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.99.0, 2.0.0
> Reporter: Jerry He
> Assignee: Jerry He
> Priority: Critical
> Fix For: 0.99.0, 2.0.0
>
> Attachments: HBASE-11882-master-v1.patch,
> HBASE-11882-master-v2.patch, HBASE-11882-master-v3.patch,
> TestHRegionServerBulkLoad.java.patch
>
>
> While looking into the TestHRegionServerBulkLoad failure for HBASE-11772, I
> found the root cause is that row level atomicity may not be maintained with
> bulk load together with compation.
> TestHRegionServerBulkLoad is used to test bulk load atomicity. The test uses
> multiple threads to do bulk load and scan continuously and do compactions
> periodically.
> It verifies row level data is always consistent across column families.
> After HBASE-11591, we added readpoint checks for bulkloaded data using the
> seqId at the time of bulk load. Now a scanner will not see the data from a
> bulk load if the scanner's readpoint is earlier than the bulk load seqId.
> Previously, the atomic bulk load result is visible immediately to all
> scanners.
> The problem is with compaction after bulk load. Compaction does not lock the
> region and it is done one store (column family) at a time. It also compact
> away the seqId marker of bulk load.
> Here is an event sequence where the row level consistency is broken.
> 1. A scanner is started to scan a region with cf1 and cf2. The readpoint is
> 10.
> 2. There is a bulk load that loads into cf1 and cf2. The bulk load seqId is
> 11. Bulk load is guarded by region write lock. So it is atomic.
> 3. There is a compaction that compacts cf1. It compacts away the seqId marker
> of the bulk load.
> 4. The scanner tries to next to row-1001. It gets the bulk load data for cf1
> since there is no seqId preventing it. It does not get the bulk load data
> for cf2 since the scanner's readpoint (10) is less than the bulk load seqId
> (11).
> Now the row level consistency is broken in this case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
