On Fri, 2005-05-27 at 23:50 +0000, Aaron Stone wrote: > Under the assumption that mail clients really do believe the UIDNEXT > "optimization" -- this basically sucks the life out of any hope for > lockless multimaster :-(
Yeah, that's the only conclusion I've come to. > *takes Mark Crispin's name in vain; several times* Yeah, that's pretty much what I did :) To those of you just tuning in, UID/UIDVALIDITY was about the dumbest idea ever in the history of modern systems. Originally, UW/IMAP could make an optimization of making the UID the binary offset within the mbox file and UIDVALIDITY the inode number. Even if it were never implemented as such because, it can be represented as such, it's going to be the same kind of problem. That means any solution to one problem will work for the other. So, think of it this way: Can you build a file system that distributes across multiple machines, and IF A MACHINE IS DOWN still guarantee that they increase the length of the file at the same time? I didn't think so. So, my idea is to defer updates to the filesystem when group-communication is impossible. It looks like Cyrus is torn between doing nothing, and updating UIDVALIDITY each time the communication between the two breaks. It may very well be that communication breaks infrequently, but these guys are right: Your users will hate it. UIDVALIDITY is wrong in general is because (using the network filesystem analogy) updating it is akin to copying over everything to a new network disk and expiring the old one. So remember kids, don't assume that because you don't understand the problem that someone in another field will solve it for you one day. I can't solve this RFC2060 problem. I don't think anyone can. If I'm wrong, I'll be happy because this problem shows up all time time in my field. > So, token passing? :) > Also, we might consider using the MUPDATE protocol to do the requisite > locking, but then also use the databases to do the actual data > replication. I don't think this is a good idea. I think the replication should be left up to the database itself. Trying to add replication to dbmail itself means that it's all the harder to write programs that interface with the db directly. And if we really want to give that up, why aren't we all working on cyrus? I mean, any sqlite/mysql/pgsql is going to be slower (overhead) than a special-purpose mail store :) No, I don't think MUPDATE is a good idea for dbmail's replication, but it might be useful to support it to replicate with Cyrus. It might make switching Cyrus users over to DBMAIL easier because they just set up dbmail as a "cyrus peer" that has no data and Cyrus will use MURDER/MUPDATE to "repair" the dbmail instance. I think getting native replication down first is more important. > http://www.faqs.org/rfcs/rfc3656.html > > Actually, after skimming the RFC, scratch that idea. (But I'll leave the > thought in this email in case someone else can make something of it.) :) > On Fri, May 27, 2005, Geo Carncross <[EMAIL PROTECTED]> > said: > > > On Fri, 2005-05-27 at 10:51 -0700, Kevin Baker wrote: > >> Thanks for the explination I totally understand now. > >> > >> That being said, what is the actual harm in one message id > >> being even a second off for the UID. They will both be > >> entered in the mailstore, even if they are slightly out of > >> order. > > > > It matters because if a client sees a mailbox and the highest UID is 287 > > and UID 286 comes in later it will NEVER KNOW about UID 286 which means > > mail is lost and the user is confused. > > > > RFC2060 actually _encourages_ IMAP client authors to make this > > optimization. > > > > > >> Ultimately the Mail client will sort by the header > >> information and if the clocks are different then they will > >> be out of order. > > > > No. You're confusing Received-time and UID-time. > > > > You also assume clients perform an IMAP SEARCH ALL NOT DELETED operation > > whenever a new message comes in. There's no such guarantee. If a client > > saw "LAST UID=287" twice in a raw, then it'll never download UID 286. > > It'll just never be fetched. > > > > Here's the roadmap: > > > > UID 287 inserted > > client sees highest UID number is 287 > > UID 286 inserted > > client NEVER DOWNLOADS UID 286 > > > > It never does because it has no reason whatsoever to think it exists. > > Some IMAP clients will detect it later because they do an IMAP SEARCH > > operation and treat the mailbox as a cache (Netscape, Thunderbird), but > > others will never notice it until UIDVALIDITY changes. > > > > > >> I should note that I've been running a Cyrus system for > >> some time now that occasionally has messages arrive one or > >> two spots out of order, due to latency with mailfilters. > >> It really isn't a problem. > > > > Cyrus _NEVER_ inserts UIDs before the highest number. Received time has > > nothing to do with it. > > > >> I realize this might bend the RFC slightly... but the > >> added functionality is great and it will not break any > >> mail clients. I am all for complete RFC compliance, but if > >> we are talking about a matter of a second here and there > >> due to processor lag I think we are probably going > >> overboard. > > > > No. It's a very serious violation of RFC2060. This "optimization" breaks > > client optimizations. It breaks _most_ mail clients in very subtle ways. > > > > People will complain. > > > > -- > > Internet Connection High Quality Web Hosting > > http://www.internetconnection.net/ > > > > _______________________________________________ > > Dbmail-dev mailing list > > Dbmail-dev@dbmail.org > > http://twister.fastxs.net/mailman/listinfo/dbmail-dev > > -- Internet Connection High Quality Web Hosting http://www.internetconnection.net/
