[
https://issues.apache.org/jira/browse/HBASE-5569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233845#comment-13233845
]
Lars Hofhansl commented on HBASE-5569:
--------------------------------------
@Stack:
So HBASE-2856 has logic that prevents KVs from being removed in a flush or
compaction when it has expired (due to TTL or too many version) but there is
still a scanner open with a readpoint <= the KV's memstoreTS. (which means
these KVs were created after the scanner was opened)
Say you have set your store set 3 versions. Now you create 10 versions of a KV,
the extra 7 versions are not removed during a flush or compaction when a
scanner that was opened before the KVs were created.
This patch adds the same for deleted KVs (IMHO that is something that
HBASE-2856 missed). So now expired and deleted KVs are not collected if a
scanner could still access them.
It means that a flush or compaction needs to copy these KVs to the new store
file instead of skipping them. This only happens for KVs that were created (or
now deleted) after the scanner(s) were openened.
The output I added is the memstoreTS. The ts is already part toString.
I don't think that needs to be part of the release notes (at least we did not
add this to the release notes for HBASE-2856 or its backport).
> Do not collect deleted KVs when they are still in use by a scanner.
> -------------------------------------------------------------------
>
> Key: HBASE-5569
> URL: https://issues.apache.org/jira/browse/HBASE-5569
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 5569-v2.txt, 5569-v3.txt, 5569-v4.txt, 5569.txt,
> TestAtomicOperation-output.trunk_120313.rar
>
>
> I noticed this because TestAtomicOperation.testMultiRowMutationMultiThreads
> fails rarely.
> The solution is similar to HBASE-2856, where expired KVs are not collected
> when in use by a scanner.
> ---
> What I pieced together so far is that it is the *scanning* side that has
> problems sometimes.
> Every time I see a assertion failure in the log I see this before:
> {quote}
> 2012-03-12 21:48:49,523 DEBUG [Thread-211] regionserver.StoreScanner(499):
> Storescanner.peek() is changed where before =
> rowB/colfamily11:qual1/75366/Put/vlen=6,and after =
> rowB/colfamily11:qual1/75203/DeleteColumn/vlen=0
> {quote}
> The order of if the Put and Delete is sometimes reversed.
> The test threads should always see exactly one KV, if the "before" was the
> Put the thread see 0 KVs, if the "before" was the Delete the threads see 2
> KVs.
> This debug message comes from StoreScanner to checkReseek. It seems we still
> some consistency issue with scanning sometimes :(
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
