[
https://issues.apache.org/jira/browse/IGNITE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko reassigned IGNITE-19267:
----------------------------------------
Assignee: Kirill Tkalenko
> Implement local Low Watermark propagation
> -----------------------------------------
>
> Key: IGNITE-19267
> URL: https://issues.apache.org/jira/browse/IGNITE-19267
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Kirill Tkalenko
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> According to the IEP-91, we can delete old data, once it becomes older than
> the certain threshold. At the moment, we can consider this threshold to be
> shared between different tables, but be unique on individual nodes. It's
> called a Low Watermark (LW).
> The way the value is chosen is the following:
> * There's the {_}data availability time{_}, that can be configured by the
> user. This is a cluster configuration. It has a value of, for example, 45
> minutes. Valid values - {{{}[0, +INF){}}}.
> * There's a {_}GC frequency{_}, that's also a cluster configuration. For
> example, 5 minuter. Range of valid values should be more strict.
> * Last LW value is persisted in Vault.
> * Every 5 minutes, we assign a new "lwCandidate{{{} = now() - 45min -
> maxClockSkew"{}}}.
> ** If there are no running transactions with timestamp below
> {{{}lwCandidate{}}}, we promote the candidate into a real LW value.
> ** Otherwise, we trigger GC with timestamp of the transaction with the
> oldest timestamp (and promoting LW to that timestamp), and raising the bar
> every time* that transaction is completed. Eventually, we will reach the
> point where there are no running transactions with timestamp below
> {{{}lwCandidate{}}}.
> * it's not necessary to do it every time. But, once the timestamp of the
> oldest RO transaction is above or equal to {{{}lwCandidate{}}}, we must
> guarantee its promotion. Everything else is optimization.
> * If there's a new RO transaction with timestamp below {{{}lwCandidate{}}},
> we fail it.
> Promoted LW value cannot become smaller no matter what. All data below LW is
> considered to be invalid, maybe broken and completely invisible to user.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
