[
https://issues.apache.org/jira/browse/HBASE-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186595#comment-15186595
]
stack commented on HBASE-15398:
-------------------------------
The storeHeap/joinedHeap came in here:
{code}
commit 39954ddcd4b6e825012f394072d38a92b2171fa1
Author: Zhihong Yu <te...@apache.org>
Date: Wed Jan 9 21:37:35 2013 +0000
HBASE-5416 Improve performance of scans with some kind of filters (Max
Lapan and Sergey)
git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1431103
13f79535-47bb-0310-9956-ffa450edef68
{code}
A.K.A "Filter on one CF and if a match, then load and return full row (WAS:
Improve performance of scans with some kind of filters)"
bq. We scan storeHeap first, then joinedHeap, and merge the results and sort
and return to client.
Everywhere we merge sort except in this case? And because of this one case, we
have to add a sort to Result (previous we relied on Cells coming up out of the
server in order IIRC)?
So, if the SCVF filter is in place, we won't do partials? And we'll not do
partials silently? i.e. we'll just not go the partial route if SCVF is in place?
Good find [~yangzhe1991]
> Cells loss or disorder when using family essential filter and partial
> scanning protocol
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-15398
> URL: https://issues.apache.org/jira/browse/HBASE-15398
> Project: HBase
> Issue Type: Bug
> Components: dataloss, Scanners
> Affects Versions: 1.2.0, 1.1.3
> Reporter: Phil Yang
> Assignee: Phil Yang
> Priority: Critical
> Attachments: 15398-test.txt, HBASE-15398.v1.txt
>
>
> In RegionScannerImpl, we have two heaps, storeHeap and joinedHeap. If we have
> a filter and it doesn't apply to all cf, the stores whose families needn't be
> filtered will be in joinedHeap. We scan storeHeap first, then joinedHeap,
> and merge the results and sort and return to client. We need sort because the
> order of Cell is rowkey/cf/cq/ts and a smaller cf may be in the joinedHeap.
> However, after HBASE-11544 we may transfer partial results when we get
> SIZE_LIMIT_REACHED_MID_ROW or other similar states. We may return a larger cf
> first because it is in storeHeap and then a smaller cf because it is in
> joinedHeap. Server won't hold all cells in a row and client doesn't have a
> sorting logic. The order of cf in Result for user is wrong.
> And a more critical bug is, if we get a LIMIT_REACHED_MID_ROW on the last
> cell of a row in storeHeap, we will break scanning in RegionScannerImpl and
> in populateResult we will change the state to SIZE_LIMIT_REACHED because next
> peeked cell is next row. But this is only the last cell of one and we have
> two... And SIZE_LIMIT_REACHED means this Result is not partial (by
> ScannerContext.partialResultFormed), client will see it and merge them and
> return to user with losing data of joinedHeap. On next scan we will read next
> row of storeHeap and joinedHeap is forgotten and never be read...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
