Howard Chu wrote:
Emmanuel Lecharny wrote:
Hi guys,
the current implementation of replication, AFAICT, is based on a pull
system. So to speak, each time you modify some entry, a message is sent
to all the replicas, waiting for an ack to be returned.
The main advantage is that we can't be faster: we replicate as soon as
possible.
The main issue is that if a replica is not connected, we will try and
try and try, until the remote server is back.
That can become a serious PITA; it certainly was with OpenLDAP slurpd.
It is already a PITA for Mitosis ;)
Here are some ideas I rehashed in the train on my way to office those
last two days...
- We should ask the replicas to register to the other servers using a
LDAP extended request
- then the server will push the modifications in a blocking queue for
each replicas
- the blocking queue is read by a thread and the modifications are
stored in a base, and sent to the replicas using a LDAP request, with a
control (Replication)
- the replica receives the modifications as simple LDAP requests, plus a
control, and deal with those, and send back a Ldap response with a
status, allowing the modification to be removed from the store.
- if the replica is disconnected (for any reason), the server does not
send anymore modifications to the replica, until the replica connects
again.
- in this case, we simply restart the thread and send all the pending
modifications found in the store and in the queue to the replica.
Likewise, a queue of pending mods can become a serious PITA if the
replica is offline for a long time.
The queue will be limited in size, obviously. I can even be empty, all
the modification being stored on disk.
However, we don't necessarily have to keep a track of pending modify for
disconnect replicas, as it's easy to know which entries has not been
replicated since the last time the replica was on. The connecting
replica could send the last entryCSN received, and then a search can be
done on the server for every CSN with a higher CSN. Then you don't need
to keep any modification on disk, as they are already stored in the DiT.
See section 17.2.4.1 "Replacing Slurpd" here
http://www.openldap.org/doc/admin24/replication.html
Damn, _know_, I see that my previous sentence is exactly what OpenLDAP
does... Either I'm a smart guy, or -more probably - I already read this
page and it remains somewhere in my fuzzy memory :)
I have to get some sleep, and start reading more instead of reinventing
the wheel, I guess :).
Thanks Howard !
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
