[
https://issues.apache.org/jira/browse/IGNITE-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iurii Gerzhedovich updated IGNITE-26067:
----------------------------------------
Description:
To implement staleness check for query plan auto-reoptimization, it's proposed
to introduce modification counter on per-partition basis. We don’t really need
a fair modification counter, but rather a timestamp when we detected staleness
for the last time. Hence, for every partition we need to maintain a triplet of
volatile attributes *{{(modificationCounter, nextMilestone,
lastMilestoneReachedTs)}}*. This triplet should be initialized at the start of
the partition with values *{{(0, computeNextMilestone(currentPartitionSize),
HybridClock.now())}}*. To implement *{{computeNextMilestone}}*, we need to
introduce 2 new configuration parameters:
{code:java}
sql.statistics.staleRowsFraction = 0.2 (default) // fraction of table required
to be modified to trigger statistics recollection, i.e. currentSize *
staleRowsFraction to compute actual number of rows required
sql.statistics.minStaleRowsCount = 500 (default) // minimal number of updates
required to trigger statistics recollection
{code}
These are cluster wide parameters, later we will introduce overrides on a
per-table basis.
A *{{modificationCounter}}* should be incremented by 1 per any RowId from
update command, i.e. it should be incremented by 1 for *{{UpdateCommand}}*, and
by *{{size(batch)}}* for *{{UpdateAllCommand}}*.
When *{{modificationCounter}}* exceeds *{{nextMilestone}}*, a new milestone
should be computed, and *{{lastMilestoneReachedTs}}* is set with *{{commitTs}}*.
The counters should be exposed as metrics
was:
To implement staleness check for query plan auto-reoptimization, it's proposed
to introduce modification counter on per-partition basis. We don’t really need
a fair modification counter, but rather a timestamp when we detected staleness
for the last time. Hence, for every partition we need to maintain a triplet of
volatile attributes *{{(modificationCounter, nextMilestone,
lastMilestoneReachedTs)}}*. This triplet should be initialized at the start of
the partition with values *{{(0, computeNextMilestone(currentPartitionSize),
HybridClock.now())}}*. To implement *{{computeNextMilestone}}*, we need to
introduce 2 new configuration parameters:
{code:java}
sql.statistics.staleRowsFraction = 0.2 (default) // fraction of table required
to be modified to trigger statistics recollection, i.e. currentSize *
staleRowsFraction to compute actual number of rows required
sql.statistics.minStaleRowsCount = 500 (default) // minimal number of updates
required to trigger statistics recollection
{code}
These are cluster wide parameters, later we will introduce overrides on a
per-table basis.
A *{{modificationCounter}}* should be incremented by 1 per any RowId from
update command, i.e. it should be incremented by 1 for *{{UpdateCommand}}*, and
by *{{size(batch)}}* for *{{UpdateAllCommand}}*.
When *{{modificationCounter}}* exceeds *{{nextMilestone}}*, a new milestone
should be computed, and *{{lastMilestoneReachedTs}}* is set with *{{commitTs}}*.
> Add modification counter to partition
> -------------------------------------
>
> Key: IGNITE-26067
> URL: https://issues.apache.org/jira/browse/IGNITE-26067
> Project: Ignite
> Issue Type: Improvement
> Components: tables ai3
> Reporter: Konstantin Orlov
> Priority: Major
> Labels: ignite-3
>
> To implement staleness check for query plan auto-reoptimization, it's
> proposed to introduce modification counter on per-partition basis. We don’t
> really need a fair modification counter, but rather a timestamp when we
> detected staleness for the last time. Hence, for every partition we need to
> maintain a triplet of volatile attributes *{{(modificationCounter,
> nextMilestone, lastMilestoneReachedTs)}}*. This triplet should be initialized
> at the start of the partition with values *{{(0,
> computeNextMilestone(currentPartitionSize), HybridClock.now())}}*. To
> implement *{{computeNextMilestone}}*, we need to introduce 2 new
> configuration parameters:
> {code:java}
> sql.statistics.staleRowsFraction = 0.2 (default) // fraction of table
> required to be modified to trigger statistics recollection, i.e. currentSize
> * staleRowsFraction to compute actual number of rows required
> sql.statistics.minStaleRowsCount = 500 (default) // minimal number of updates
> required to trigger statistics recollection
> {code}
> These are cluster wide parameters, later we will introduce overrides on a
> per-table basis.
> A *{{modificationCounter}}* should be incremented by 1 per any RowId from
> update command, i.e. it should be incremented by 1 for *{{UpdateCommand}}*,
> and by *{{size(batch)}}* for *{{UpdateAllCommand}}*.
> When *{{modificationCounter}}* exceeds *{{nextMilestone}}*, a new milestone
> should be computed, and *{{lastMilestoneReachedTs}}* is set with
> *{{commitTs}}*.
> The counters should be exposed as metrics
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
