[
https://issues.apache.org/jira/browse/HBASE-12363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated HBASE-12363:
---------------------------------
Labels: Phoenix (was: )
> KEEP_DELETED_CELLS considered harmful?
> --------------------------------------
>
> Key: HBASE-12363
> URL: https://issues.apache.org/jira/browse/HBASE-12363
> Project: HBase
> Issue Type: Sub-task
> Components: regionserver
> Reporter: Lars Hofhansl
> Labels: Phoenix
>
> Brainstorming...
> This morning in the train (of all places) I realized a fundamental issue in
> how KEEP_DELETED_CELLS is implemented.
> The problem is around knowing when it is safe to remove a delete marker (we
> cannot remove it unless all cells affected by it are remove otherwise).
> This was particularly hard for family marker, since they sort before all
> cells of a row, and hence scanning forward through an HFile you cannot know
> whether the family markers are still needed until at least the entire row is
> scanned.
> My solution was to keep the TS of the oldest put in any given HFile, and only
> remove delete markers older than that TS.
> That sounds good on the face of it... But now imagine you wrote a version of
> ROW 1 and then never update it again. Then later you write a billion other
> rows and delete them all. Since the TS of the cells in ROW 1 is older than
> all the delete markers for the other billion rows, these will never be
> collected... At least for the region that hosts ROW 1 after a major
> compaction.
> I don't see a good way out of this. In parent I outlined these four solutions:
> So there are three options I think:
> # Only allow the new flag set on CFs with TTL set. MIN_VERSIONS would not
> apply to deleted rows or delete marker rows (wouldn't know how long to keep
> family deletes in that case). (MAX)VERSIONS would still be enforced on all
> rows types except for family delete markers.
> # Translate family delete markers to column delete marker at (major)
> compaction time.
> # Change HFileWriterV* to keep track of the earliest put TS in a store and
> write it to the file metadata. Use that use expire delete marker that are
> older and hence can't affect any puts in the file.
> # Have Store.java keep track of the earliest put in internalFlushCache and
> compactStore and then append it to the file metadata. That way HFileWriterV*
> would not need to know about KVs.
> And I implemented #4.
> I'd love to get input on ideas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
