[
https://issues.apache.org/jira/browse/IGNITE-20210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17789942#comment-17789942
]
Denis Chudov commented on IGNITE-20210:
---------------------------------------
After some discussions, the scope of this ticket was reduced. The main reason
is that table start (and API futures of starting tables as well) relies on
versioned values, and all events related to table start happen withing one
revision of meta storage. Other components, such as indexes, also rely on this
fact while processing table related events. Assuming that the partitions should
start only by assignments' change event, it means that the process of a table
start should be distributed between multiple revisions (rev. _x_ starts
TableImpl and writes assignments, other revision _x + y_ processes assignments
and starts partitions, after that indexes start and table creation completes).
This work is also closely related to the colocation epic IGNITE-19170 in which
the partition start should be reworked and the responsibility for partitions
should be moved from tables to zones and inevitably the table start and the
partitions start will happen on different meta storage revisions.
So, in this ticket I mostly reworked the partition start, removing similar code
related to this process in table start and assignments' changes processing.
Table start process is still responsible for starting partitions, but it used
the code processing the assignment changes to do this, which will make further
improvements easier. Table stop process remained responsible for the partitions
stop as well. It will be also required to start partitions written to stable
assignments on the node recovery, but now the tables are started on node
recovery, and this triggers the start of partitions.
> Start partitions on corresponding assignments.stable, calculate if missing,
> cleanup obsolete resources (part 1)
> ---------------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-20210
> URL: https://issues.apache.org/jira/browse/IGNITE-20210
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Assignee: Denis Chudov
> Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Please check https://issues.apache.org/jira/browse/IGNITE-20187 and
> https://issues.apache.org/jira/browse/IGNITE-20209 for more details. This
> ticket is about assignments stable catch-up. Obviously there are the
> following possibilities:
> # Assignments.stable are present - just start table locally. Basically it is
> IGNITE-20187 but not for assignments pending but stable.
> # Assignemnts stable are missing. Well it's the same as IGNITE-20209 but for
> table creation triggers and not rebalance ones.
> Besides that it's nessessary to cleanup obsolete resourves e.g. raft and
> partitions storages.
> Currently, all that stuff is implemented incorrectly through:
> {code:java}
> if (partitionAssignments(vaultManager, tableId, 0) != null) {
> assignmentsFuture = completedFuture(tableAssignments(vaultManager,
> tableId, zoneDescriptor.partitions()));
> } else {
> assignmentsFuture =
> distributionZoneManager.dataNodes(ctx.storageRevision(),
> tableDescriptor.zoneId())
> .thenApply(dataNodes -> AffinityUtils.calculateAssignments(
> dataNodes,
> zoneDescriptor.partitions(),
> zoneDescriptor.replicas()
> ));
> } {code}
> h3. Definition of Done
> * Assignments.stable update is properly catched-up on top of corresponding
> table creation triggers.
> * Partitions start up is implemented trough assignments.stable instead of
> table cfg triggers along with assignments recalculation, like it's
> implemented now.
> * Obsolete partition storages are removed on node restart.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
