Hi there Nadine,
This is an interesting scenario.... As you have already mentioned, the
locators property is recommended to be static and defined up front.
From what I understand the list of locators is build up dynamically at
runtime. i.e there must be some process that runs before the locators
property is set, that checks if there is an existing locator running and
populates its credentials into the property.
If I "walk through" the start up process you have defined, it makes
sense that it could work. (so please don't take this as a blessing) It
seems that it would work for locators... but I'm not sure about what it
means for the servers or clients.
Also are you intending to us WAN replication? If so, how is that
handled? As the locators need a well-defined list of "remote-locators"
at start up.
It also brings up a whole bunch of other questions for servers, clients
and HA.
How does the central registry handle locators failing/stopping?
How does it handle the case if locator2 were to fail? I mean, locator3
would still validly reference it, but it might not exist any more.
(which is fine). But if you now start locator4 (due to HA procedures),
neither locator1 or locator3 would know about locator4. (or locator5 or
locator6)
Does the central registry monitor the locators to confirm that they
still work and if not does it removes them from the list? What about
connectivity issues (split brain, etc...)
Is the start up procedure for the clusters ALWAYS the same? i.e all
locators are started before the servers are started. Providing the
"full" list of locators to the servers at startup.
How do servers handle the failure of locators? What happens if all
locators were to fail? The servers would still work, but reference
incorrect locator host names. So the servers would not know about any of
the "new" locators joining.
How do clients get the full list of locators?
How are the pools defined? i.e do clients follow the same process? Check
the central registry for running locators and populate the pool locators
list with only the registered locators?
What happens when clients start before all the locators have been
started? The ability for clients to connect/reconnect to clusters in
case of failure is through the defined list of locators/servers. If it
detects failure in the one, it will try the next in the list....
transparent failover.
How does a running client handle the failure of all the locators? In
case of all locators failing, does it mean that all servers and clients
have to be shutdown and the whole eco-system has to be restarted?
So maybe this approach is not advisable.
If I may ask, why would you not be able to determine a set a servers
that are used for the cluster? Could you not somehow reserve some hosts
for the Geode cluster, without dynamic name allocation? Are you running
in a virtualized environment like AWS, VMware?
--Udo
On 22/06/2016 10:24 PM, Nadine Daccache wrote:
Hello everyone,
I have a problem related to the locators property used to start the
members of a cluster, I thought you might help.
In my case, the IP addresses of the locators hosts could not be
statically configured, and are only known at runtime. The idea is to
register the locator host and port in a central registry, and whenever
a locator is started, to check the previously started locators.
For instance:
-locator1 will be started with no locators property, and registers
host1[port1]
-locator2 will be started with --locators=host1[port1], and registers
host2[port2]
-locator3 will be started with --locators=host1[port1],host2[port2],
and registers host3[port3]
I have tried it, and it seems to work fine. The 3 locators join the
same distributed system. But I haven’t performed more advanced testing.
The documentation clearly states that every locator should have the
complete locators list, even if they have not been started.
Do you know what would be the limitations (if any) if the locators are
started in this way? Do you have any idea that could help?
Regards,
Nadine
