[
https://issues.apache.org/jira/browse/PHOENIX-7214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kadir Ozdemir updated PHOENIX-7214:
-----------------------------------
Description:
HBase minor compaction does not remove deleted or expired cells since the minor
compaction works on a subset of HFiles. However, it is safe to remove expired
rows for immutable tables. For immutable tables, rows are inserted but not
updated. This means a given row will have only one version.This means we can
safely remove expired rows during minor compaction using CompactionScanner in
Phoenix.
CompactionScanner currently runs only for major compaction. We can introduce an
new table attribute called MINOR_COMPACT_TTL. Phoenix can run
CompactionScanner for minor compaction too for the tables with
MINOR_COMPACT_TTL = TRUE. By doing so, the expired rows will be purged during
minor compaction for these tables. This will be useful when TTL is less than 7
days, say 2 days, as major compaction typically runs only once a week.
was:
HBase minor compaction does not remove deleted or expired cells since the minor
compaction works on a subset of HFiles. However, it is safe to remove expired
rows for immutable tables without delete markers. For immutable tables, rows
are inserted but not updated. This means a given row will have only one
version. If these are are not deleted by using delete markers but purged with
TTL then, we can safely remove expired rows during minor compaction using
CompactionScanner in Phoenix.
CompactionScanner currently runs only for major compaction. We can introduce an
new table attribute called MINOR_COMPACT_TTL. Phoenix can run
CompactionScanner for minor compaction too for the tables with
MINOR_COMPACT_TTL = TRUE. By doing so, the expired rows will be purged during
minor compaction for these tables. This will be useful when TTL is less than 7
days, say 2 days, as major compaction typically runs only once a week.
> Purging expired rows during minor compaction for immutable tables
> -----------------------------------------------------------------
>
> Key: PHOENIX-7214
> URL: https://issues.apache.org/jira/browse/PHOENIX-7214
> Project: Phoenix
> Issue Type: New Feature
> Reporter: Kadir Ozdemir
> Priority: Major
>
> HBase minor compaction does not remove deleted or expired cells since the
> minor compaction works on a subset of HFiles. However, it is safe to remove
> expired rows for immutable tables. For immutable tables, rows are inserted
> but not updated. This means a given row will have only one version.This means
> we can safely remove expired rows during minor compaction using
> CompactionScanner in Phoenix.
> CompactionScanner currently runs only for major compaction. We can introduce
> an new table attribute called MINOR_COMPACT_TTL. Phoenix can run
> CompactionScanner for minor compaction too for the tables with
> MINOR_COMPACT_TTL = TRUE. By doing so, the expired rows will be purged during
> minor compaction for these tables. This will be useful when TTL is less than
> 7 days, say 2 days, as major compaction typically runs only once a week.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
