Hi.
We are currently working on a feature to track the rebalancing of our Ignite
cluster listening all events: when it starts and end, how many partitions
are sent, etc.
We implemented a singleton Ignite Service and deploy it on the cluster.
When the service starts it listens the following remote events:
EVT_NODE_JOINED
EVT_NODE_LEFT
EVT_NODE_FAILED
EVT_NODE_SEGMENTED
EVT_CACHE_REBALANCE_STARTED
EVT_CACHE_REBALANCE_STOPPED
EVT_CACHE_REBALANCE_PART_SUPPLIED
EVT_CACHE_REBALANCE_PART_LOADED
EVT_CACHE_REBALANCE_PART_UNLOADED
EVT_CACHE_REBALANCE_PART_DATA_LOST
EVT_CACHE_REBALANCE_PART_MISSED
Some questions about our approach:
1. Is the following sequence of events correct?
* Baseline: A,B
* setBaselineTopology: A, B, C, D
* C and D send event EVT_CACHE_REBALANCE_STARTED
* A and B send multiple EVT_CACHE_REBALANCE_PART_SUPPLIED with the
partitions to rebalance
* C and D receive data from A and B
* C and D send multiple CACHE_ENTRY_CREATED, CACHE_ENTRY_DESTROYED,
CACHE_REBALANCE_OBJECT_LOADED to create new entries
* C and D send multiple CACHE_REBALANCE_PART_LOADED when the data is
loaded into the new nodes
* C and D send event EVT_CACHE_REBALANCE_STOPPED
* A and B send multiple CACHE_ENTRY_CREATED, CACHE_ENTRY_DESTROYED,
CACHE_REBALANCE_OBJECT_UNLOADED to remove old entries
* A and B send multiple EVT_CACHE_REBALANCE_PART_UNLOADED when the
partitions are removed
2. As you can see, I receive some CACHE_ENTRY_DESTROYED when C and D
are receiving the data. Why are they destroying entries? Also, something
similar happens when A and B are removing partitions, I receive some
CACHE_ENTRY_CREATED.
3. Sometimes I did not receive an EVT_CACHE_REBALANCE_STARTED from a
node that is supposed to start a rebalancing, how is this possible?
4. About the use of services, when the node that contains a service is
removed from the cluster, it is translated to another node. Is there a way
to do it manually? A mean, for example I know that the node is processing a
lot of data, and I want to transfer this service to a different node, is it
possible to do it?
5. We want to know if a rebalancing is in course, is there a way to ask
the cluster about this? We currently listen the events and maintain a
variable with this state, but if the service dies. the variable is lost.
Thank you.
