[
https://issues.apache.org/jira/browse/IOTDB-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070771#comment-17070771
]
Tian Jiang commented on IOTDB-460:
----------------------------------
That seems to work, too. The first one tracks the state of each slot, making
the cluster more monitorable, with the cost of potentially more meta group
communication. The second one seems to have fewer overheads.
But I have one question:
How are you going to define the finishing of data pulling? As we are doing a
group-to-group data transfer, there may be three ways to define this: a) when
the new leader has pulled data from the old leader, b) when over half of the
new members have pulled data from the old members, c) when all new members have
pulled data from old members. This choice will influence how you process the
cases when some of the members are down, either old or new.
> [Distributed]Remove data of outdated slots
> ------------------------------------------
>
> Key: IOTDB-460
> URL: https://issues.apache.org/jira/browse/IOTDB-460
> Project: Apache IoTDB
> Issue Type: Improvement
> Reporter: Tian Jiang
> Priority: Major
> Labels: data-transfer, distributed
>
> In node addition/removal, the slots managed by a node will change. However,
> the data of corresponding slots cannot be removed yet because the new holders
> of the slots will pull such data to themselves. As the data pulling is issued
> by the new holders randomly, it is hard for the previous holders to find out
> when will the data be needless. As a result, the previous holder cannot
> delete the local data with confidence.
> It will be necessary to find a way for the previous holders to know when the
> data has been replicated to the new holders so that they will be able to
> remove the local data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
