On Dec 1, 2006, at 2:28 PM, Jeff Davis wrote: > On Fri, 2006-12-01 at 18:29 +0100, Andreas Kostyrka wrote: >>> Alrighty. The problem at hand is, "how do I get 1024 copies of a >>> database?" Do you have some insight into how to do this with >>> Slony-I? >> Shipout the sql statements needed via slon -a >> Distribute them yourself. >> > > If this works, why can't this be done automatically with slony? > Couldn't > the confirmations just be sent to the provider and the provider > send the > confirmations to it's provider?
It cannot be done automatically because by design slony does not implement a transport mechanism from one node to another when using log shipping. It is designed to leave that up to the user. As the user you are free to use anything from scp to carrier pigeon to get the data to the offline secondaries. Also when using log shipping, you would not need a cascade of providers, just one primary and one secondary with log shipping enabled, and the offline secondaries do not communicate whatsoever back to the cluster. Assuming you use log shipping (and I would highly recommend it), a problem is ensuring that all 1024 secondaries stay in reasonable sync. You either "ship" the logs by actually sending the files to the hosts and applying them there with 1024 separate remote processes, or use a centralized process (or processes) to apply the logs (from say the real slony secondary server). In the former config you have the issue of tending 1024 processes on 1024 different boxes, but since they all work independently, they could fall quite out of sync with varying load, etc. (where "quite" is defined by the user of course). In the latter config, the central process can keep things synchronized more easily, but you'll need to build in some allowance for when boxes inevitably go down or are generally unavailable. Either way, you have a bit of a garbage collection issue. How long do you keep the logs around at their origin, and at what point given a failed secondary do you give up on synchronizing it and force a rebuild, since you cannot keep the old logs forever? How do you best keep track of who has received logs and who hasn't? The answer to these questions depends a lot on the data volume. With a light enough volume, you might simply ignore the problem altogether and let the logs stack up indefinitely when one of the offline secondaries is unavailable. Under heavy (write) load things would pile up rather quickly and that wouldn't be an option. Another (related) problem is rebuilding (or adding) offline secondaries. To do this you'd need to dump the secondary and load it. At this point--unless you suspend replication to all other offline secondaries--the replication process will need to be smart enough to know which log to apply next (this is not too hard). And you'll need to keep the logs around long enough in this case as well. -Casey _______________________________________________ Slony1-general mailing list [email protected] http://gborg.postgresql.org/mailman/listinfo/slony1-general
