Timo Sirainen writes:
Why not go with a pure log replication scheme?
this way you basically have 3 processes.
1- The normal, currently existing programs. Add logs to the process
2- A Master replication process which listens for clients requesting for
info.
3- The slave processes that request infomation and write it to the slave
machines.
With this approach you can basically break it down into logical units of
code which can be tested and debugged. Also helps when you need to worry
about security and the level at which each component needs to work.
I'm not completely sure what you mean by these. Basically the same as
what I said, except just have imap/deliver simply send the changes
without any waiting?
I am basically suggesting to log all the changes to a log(s) and have a
separate program handle passing on the information to the slaves.
But isn't the point of the master/slave that the slave would switch on
if the master dies?
Yes.
If you switch slave to be the new master, it doesn't
matter if the logs were written to master's disk. Sure the message could
come back when the master is again brought back
I was thinking that once a master dies, next it it comes back it would be a
slave and no longer master. This would, unfortunately imply, that any
transactions that were not copied over would be lost.
I don't understand what you mean. Sure the logs could timeout at some
point (or shouldn't there be some size limits anyway?), but you'd still
need to keep track of what different slaves have seen.
I was thinking that it would be the slaves job to ask for data.
Example pseudo transactions.
Master gets transactions 1 through 100
Slave(s) start from scratch and ask from transactions starting at 1.
Say one slave, let's call it A, dies at transaction 50 and another slave,
say B, continues and goes all the way to 100.
More transactions come and now we are up to 150.
Slave B will ask for anything after 100.
When Slave A comes back it would ask for anything after 50.
The master simply gets a request for transactions after a given number so it
doesn't need to keep track the status of each slave.
possible. It would be easier to implement much simpler master-slave
replication, but in error conditions that would almost guarantee that
some messages get lost. I want to avoid that.
If you think Synchronous replication is doable.. go for it. :-)
it is a tradeoff of reliability vs speed.
Specially as the number of slaves grow the communication will grow.
Sure. I wasn't planning on implementing multi-slave or multi-master
before the master/slave was fully working and stress testing showing
that no mails get lost even if master is killed every few seconds (and
each crash causing master/slave to switch roles randomly).
Great idea.
