Kadir Ozdemir created PHOENIX-6888:
--------------------------------------
Summary: Fixing TTL and Max Lookback Issues for Phoenix Tables
Key: PHOENIX-6888
URL: https://issues.apache.org/jira/browse/PHOENIX-6888
Project: Phoenix
Issue Type: Bug
Affects Versions: 5.1.3
Reporter: Kadir Ozdemir
In HBase, the unit of data is a cell and data retention rules are executed at
the cell level. These rules are defined at the column family level. Phoenix
leverages the data retention features of HBase and exposes them to its users to
provide its TTL feature at the table level. However, these rules (since they
are defined at the cell level instead of the row level) results in partial row
retention that in turn creates data integrity issues at the Phoenix level.
Similarly, Phoenix’s max lookback feature leverages HBase deleted data
retention capabilities to preserve deleted cells within a configurable max
lookback. This requires two data retention windows, max lookback and TTL. One
end of these windows is the current time and the end is a moment in the past
(i.e., current time minus the window size). Typically, the max lookback window
is shorter than the TTL window. In the max lookback window, we would like to
preserve the complete history of mutations regardless of how many cell versions
these mutations generated. In the remaining TTL window outside the max
lookback, we would like to apply the data retention rules defined above.
However, HBase provides only one data retention window. Thus, the max lookback
window had to be extended to become TTL window and the max lookback feature
results in unwantedly retaining deleted data for the maximum of max lookback
and TTL periods.
This Jira is to fix both of these issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
