[
https://issues.apache.org/jira/browse/IGNITE-18963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mirza Aliev updated IGNITE-18963:
---------------------------------
Description:
{*}Motivation{*}:
Altering filters form the SQL must lead to data nodes recalculation.
*Definition of done:*
* Altering filters form the SQL leads to data nodes recalculation.
* Any scale up timers must be canceled and corresponding nodes must be included
in data nodes if filter were altered and data nodes recalculation was
triggered.
*Implementation details:*
Proper algorithm must be implemented under the ticket, currently is unclear how
we guarantee atomic cancel of existing scale up timers and data nodes
recalculation.
The list of open questions that we have to address to implement this feature
properly:
1) How should we handle situation, when alter filter event and scale up event
happen at the same time?
1.1) We should be able to handle the situation, when we had concurrent scale
up and alter filter event, and if we decide to cancel scale up, it is possible,
that new scale up with the greater revision has already rescheduled this timer
concurrently, so if we cancel that scale up, we could loose the latest scale up
event.
Scenario would look like this:
Topology = [A,B,C], filter is (A, B), data nodes = [A,B]
# Node D is added, timers for scale up are set to 5 minutes
# Node E is added, timers for scale up are rescheduled
# Change filter to filter(A, B, D, E) before scale up timer is up, revision of
event is 10
# Node F is added concurrently, with revision 11, which is also fits the
filter, timer is rescheduled
#* Here we come to the situation, where change filter event must cancel scale
up, but it cannot cancel it, because node F was added and scale up for that
must be triggered later.
#* Event about node F adding is not visible for change filter (in topology
augmentation map), because it has higher revision
# Nodes [A, B, D, E] will be as data nodes after filter change event
# Scale Up is triggered and [A, B, D, E, F] is a new data nodes.
2) Currently, we have topologyAugmentationMap and data nodes are stored as map
node -> counter. When we apply filter, we don't want to remove corresponding
node from the map in metastore, if node do not pass filter. The reason is
because filter could be changed in the future, and node could pass filter after
that, meaning that we must be able to understand, do we need to include this
node according to the scale up/scale down events.
was:
{*}Motivation{*}:
Altering filters form the SQL must lead to data nodes recalculation.
*Definition of done:*
* Altering filters form the SQL leads to data nodes recalculation.
* Any scale up timers must be canceled and corresponding nodes must be included
in data nodes if filter were altered and data nodes recalculation was
triggered.
*Implementation details:*
Proper algorithm must be implemented under the ticket, currently is unclear how
we guarantee atomic cancel of existing scale up timers and data nodes
recalculation.
The list of open questions that we have to address to implement this feature
properly:
1) How should we handle situation, when alter filter event and scale up event
happen at the same time?
1.1) We should be able to handle the situation, when we had concurrent scale
up and alter filter event, and if we decide to cancel scale up, it is possible,
that new scale up with the greater revision has already rescheduled this timer
concurrently, so if we cancel that scale up, we could loose the latest scale up
event.
Scenario would look like this:
Topology = [A,B,C], filter is (A, B), data nodes = [A,B]
# Node D is added, timers for scale up are set to 5 minutes
# Node E is added, timers for scale up are rescheduled
# Change filter to filter(A, B, D, E) before scale up timer is up, revision of
event is 10
# Node F is added concurrently, with revision 11, which is also fits the
filter, timer is rescheduled
#* Here we come to the situation, where change filter event must cancel scale
up, but it cannot cancel it, because node F was added and scale up for that
must be triggered later.
#* Event about node F adding is not visible for change filter (in topology
augmentation map), because it has higher revision
# Nodes [A, B, D, E] will be as data nodes after filter change event
# Scale Up is triggered and [A, B, D, E, F] is a new data nodes.
2) Currently, we have topologyAugmentationMap and data nodes are stored as map
node -> counter. When we apply filter, we don't want to remove corresponding
node from the map in metastore, if node do not pass filter. The reason is
because filter could be changed in the future, and node could pass filter after
that, meaning that we must be able to understand, do we need to include this
node according to the scale up/scale down events.
> Altering filters must recalculate data nodes
> --------------------------------------------
>
> Key: IGNITE-18963
> URL: https://issues.apache.org/jira/browse/IGNITE-18963
> Project: Ignite
> Issue Type: Improvement
> Reporter: Mirza Aliev
> Priority: Major
> Labels: ignite-3
>
> {*}Motivation{*}:
> Altering filters form the SQL must lead to data nodes recalculation.
> *Definition of done:*
>
> * Altering filters form the SQL leads to data nodes recalculation.
> * Any scale up timers must be canceled and corresponding nodes must be
> included in data nodes if filter were altered and data nodes recalculation
> was triggered.
> *Implementation details:*
> Proper algorithm must be implemented under the ticket, currently is unclear
> how we guarantee atomic cancel of existing scale up timers and data nodes
> recalculation.
> The list of open questions that we have to address to implement this feature
> properly:
> 1) How should we handle situation, when alter filter event and scale up event
> happen at the same time?
> 1.1) We should be able to handle the situation, when we had concurrent
> scale up and alter filter event, and if we decide to cancel scale up, it is
> possible, that new scale up with the greater revision has already rescheduled
> this timer concurrently, so if we cancel that scale up, we could loose the
> latest scale up event.
> Scenario would look like this:
> Topology = [A,B,C], filter is (A, B), data nodes = [A,B]
> # Node D is added, timers for scale up are set to 5 minutes
> # Node E is added, timers for scale up are rescheduled
> # Change filter to filter(A, B, D, E) before scale up timer is up, revision
> of event is 10
> # Node F is added concurrently, with revision 11, which is also fits the
> filter, timer is rescheduled
> #* Here we come to the situation, where change filter event must cancel scale
> up, but it cannot cancel it, because node F was added and scale up for that
> must be triggered later.
> #* Event about node F adding is not visible for change filter (in topology
> augmentation map), because it has higher revision
> # Nodes [A, B, D, E] will be as data nodes after filter change event
> # Scale Up is triggered and [A, B, D, E, F] is a new data nodes.
> 2) Currently, we have topologyAugmentationMap and data nodes are stored as
> map node -> counter. When we apply filter, we don't want to remove
> corresponding node from the map in metastore, if node do not pass filter. The
> reason is because filter could be changed in the future, and node could pass
> filter after that, meaning that we must be able to understand, do we need to
> include this node according to the scale up/scale down events.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
