Guys, I would do the following:
1. Map all my partitions to
nodes: org.apache.ignite.cache.affinity.Affinity#mapPartitionsToNodes
2. Send jobs (with its list of partitions) to each node using map returned
on step1
3. Job may be like:
new Runnable() {
@Override public void run() {
for (Integer part : parts) {
Iterator<Cache.Entry<Object, Object>> it = cache.query(new
ScanQuery<>(part)).iterator();
// do the stuff...
}
}
};
This may result in network calls for some worst cases when topology
changes under your feet, but even in this case this should work.
--Yakov
2015-12-11 2:13 GMT+03:00 Andrey Kornev <[email protected]>:
> Dmitriy,
>
> Given the approach you suggested below, what would be your recommendation
> for dealing with cluster topology changes while the iteration is in
> progress? An obvious one I can think of is to
> - somehow detect the change,
> - cancel the tasks on all the nodes
> - wait until the rebalancing is finished and
> - restart the computation.
>
> Are there any other ways? Ideally, I'd like to have the "exactly-once"
> execution semantics.
>
> Thanks
> Andrey
>
