On Mon, 2005-05-30 at 20:11 +0000, Aaron Stone wrote: > DBMail 2.0.x currently inserts a message with ownership set to an internal > user, then performs the delivery resolution and uid generation, and copies > the message over to any user scheduled to receive it. > > In a clustered setting, we add one more part to the behind-the-scenes > phase: getting a token from the cluster. Once the machine has the token, > it can generate uid's for all of the messages it has queued, pass off the > token to the next machine, and then deliver all of messages.
Kind of, yeah. > Each machine would thus queue incoming messages which were otherwise ready > for delivery, wait for the token, then deliver them. Sort-of. I don't suggest dbmail should implement the queue itself, but write it to the DB with an activation state field: 1. write the message - state=0 2. wait for token - generate uid, state -> 1 3. wait for token to prove validity, state -> 2 (now visible) > Database replication would be handled by the database, and would be > lockless multimaster because we have already done the work of guaranteeing > that uid's won't conflict. Sure. > As you mention, however, the APPEND command would also have to be > rewritten to wait for a token. Is that a problem from a user interactivity > perspective? I figure it might sometimes take more than a few seconds to > receive a token. Will this cause a problem if the OK response is delayed > until a token is received? The client can get an OK- the message just won't be visible yet. RFC2060 doesn't say that a message APPENDed has to be visible to a resulting SEARCH/FETCH immediately :) However I suspect clients are foolish here, so it might be wise to take an APPEND- set a flag, and if a SEARCH occurs on that mailbox with any state<2 (when APPEND was done recently!) that it block while state <2 in the box. > Also, for the token algorithm, is there a mechanism to ask for the token? > In order to prevent token-storming, each machine holds the token until it > either 1) processes a queue of messages or 2) waits for a timeout. If you > don't have a timeout, then you have tokens flying about eating up all > bandwidth and doing nothing. But if there's a machine on the other side of > the cluster that has a messages, but the token is several seconds away on > the other side of the cluster, perhaps it should be requested for with a > packet that flies full speed backwards, telling each machine that it > should pass the token forwards ASAP. Right? That's why the token-passing algorithm has a regenerate-token operation built-in. You can simply attempt to regenerate the token every time you "want" the token. If a machine still has the token, it'll stop it :) Now that timeout should only be a couple (max: 5) seconds. [[ if this doesn't clear it up, let me know. suffice it to say, a group message "i need the token" is not a good idea for many reasons and the regeneration protocol should be used instead. ]] -- Internet Connection High Quality Web Hosting http://www.internetconnection.net/
