[Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Mordechai T. Abzug
So, I'm interested in setting up a mail server with multimaster replication that supports POP, IMAP, and SMTP. It looks like dbmail might fit the bill, except for this: http://mailman.fastxs.net/pipermail/dbmail-dev/2004-June/004018.html Has this problem been solved yet? If not, since the mess

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Aaron Stone
We've got one decent plan proposed already, but no implementation. There are quite a few more recent threads on how to achieve multimaster replication. Nothing is as easy as anybody thinks because of how the IMAP specification is written, but thinking about the problem is always appreciated! I ca

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Mordechai T. Abzug
On Fri, Jul 15, 2005 at 02:16:02AM -, Aaron Stone wrote: > I can't find the thread at the moment, but the best plan we have so > far is a token passing system. There was a proposed API posted to > the list, which is a good starting point for anybody who'd like to > begin implementing it. Woul

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Geo Carncross
On Thu, 2005-07-14 at 23:46 -0400, Mordechai T. Abzug wrote: > On Fri, Jul 15, 2005 at 02:16:02AM -, Aaron Stone wrote: > > > I can't find the thread at the moment, but the best plan we have so > > far is a token passing system. There was a proposed API posted to > > the list, which is a good

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Geo Carncross
On Thu, 2005-07-14 at 19:02 -0400, Mordechai T. Abzug wrote: > So, I'm interested in setting up a mail server with multimaster > replication that supports POP, IMAP, and SMTP. It looks like dbmail > might fit the bill, except for this: > > http://mailman.fastxs.net/pipermail/dbmail-dev/2004-June/

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Aaron Stone
On Fri, Jul 15, 2005, Geo Carncross <[EMAIL PROTECTED]> said: > This is the grandfather of the thread: > > <[EMAIL PROTECTED]> > > Demonstrating unsolvability of "IMAP problem" by comparing to unsolvable > clock problem: > > <[EMAIL PROTECTED]> > > Demonstrating unsolvability of "IMAP problem"

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-15 Thread Aaron Stone
On Fri, Jul 15, 2005, Geo Carncross <[EMAIL PROTECTED]> said: > On Thu, 2005-07-14 at 23:46 -0400, Mordechai T. Abzug wrote: >> On Fri, Jul 15, 2005 at 02:16:02AM -, Aaron Stone wrote: >> >> > I can't find the thread at the moment, but the best plan we have so >> > far is a token passing syst

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-16 Thread Geo Carncross
On Fri, 2005-07-15 at 18:52 +, Aaron Stone wrote: > On Fri, Jul 15, 2005, Geo Carncross <[EMAIL PROTECTED]> > said: > > > This is the grandfather of the thread: > > > > <[EMAIL PROTECTED]> > > > > Demonstrating unsolvability of "IMAP problem" by comparing to unsolvable > > clock problem: > >

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-20 Thread Mordechai T. Abzug
On Fri, Jul 15, 2005 at 02:02:27PM -0400, Geo Carncross wrote: > Won't work. As much as it seems like this would be a good idea (and > believe me: about half a dozen people on this list have had it, so > it certainly is a good idea. better still, don't believe me, check > the archive yourself :) )

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-21 Thread Mordechai T. Abzug
Following up to myself with clarifications and corrections: On Wed, Jul 20, 2005 at 12:23:24AM -0400, Mordechai T. Abzug wrote: > (S5) Each time an email arrives at a server (via SMTP or IMAP, not via > replication), the server generates a new UID using scheme from > part (S1) that is

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Geo Carncross
On Wed, 2005-07-20 at 00:23 -0400, Mordechai T. Abzug wrote: > On Fri, Jul 15, 2005 at 02:02:27PM -0400, Geo Carncross wrote: > > > Won't work. As much as it seems like this would be a good idea (and > > believe me: about half a dozen people on this list have had it, so > > it certainly is a good

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Geo Carncross
On Thu, 2005-07-21 at 17:55 -0400, Mordechai T. Abzug wrote: > Following up to myself with clarifications and corrections: > > On Wed, Jul 20, 2005 at 12:23:24AM -0400, Mordechai T. Abzug wrote: > > > (S5) Each time an email arrives at a server (via SMTP or IMAP, not via > > replication), th

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Mordechai T. Abzug
On Fri, Jul 22, 2005 at 10:46:50AM -0400, Geo Carncross wrote: > > (A2) Users should have an affinity to a particular server, and should > > only switch/be switched to another server in the event of a > > failure. > > Incorrect. Many people want load-balancing. High-availability access

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Christian G. Warden
On Fri, Jul 22, 2005 at 01:19:47PM -0400, Mordechai T. Abzug wrote: > > Won't work. A client that performs the following operations will lose > > email: > > > > * Client (C) connects to host (A) sees uidvalidity mismatch; gets 1,2,4 > > * C connects to host (B) sees exists "4", tries to fetch uid

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Mordechai T. Abzug
On Fri, Jul 22, 2005 at 11:54:33AM -0400, Geo Carncross wrote: > > Clarification: the new UID must be greater than any value currently > > KNOWN LOCALLY in high_saved. Because of race conditions or network > > outages, the local high_saved table might be out-of-date with respect > > to other serv

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Mordechai T. Abzug
On Fri, Jul 22, 2005 at 10:43:17AM -0700, Christian G. Warden wrote: > > The design actually ensures, in this exact scenario, that the client > > does see the mail. What happens is that when B creates email UID 3, B > > sends A a message saying "I made an email with UID 3". When A > > receives t

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Geo Carncross
On Fri, 2005-07-22 at 14:18 -0400, Mordechai T. Abzug wrote: > On Fri, Jul 22, 2005 at 10:43:17AM -0700, Christian G. Warden wrote: > > > > The design actually ensures, in this exact scenario, that the client > > > does see the mail. What happens is that when B creates email UID 3, B > > > sends

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-22 Thread Mordechai T. Abzug
On Fri, Jul 22, 2005 at 02:27:24PM -0400, Geo Carncross wrote: > If you have per-user host affinities that are _guaranteed_ then > you've gotten load-balancing (again) and not > high-availability. You've just moved the load-balancing onto an > extra point of failure. No, you have high availabilit

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-25 Thread Geo Carncross
On Fri, 2005-07-22 at 15:43 -0400, Mordechai T. Abzug wrote: > > Nevertheless the problem still exists: > > Client is fresh > > Client connects to A and gets UID 1,2,3,4,5,7 > > Client connects to B and sees EXISTS=6 > > because B sees UID 1,2,3,4,6,7 > > Client will NEVER

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-25 Thread Mordechai T. Abzug
On Mon, Jul 25, 2005 at 10:56:38AM -0400, Geo Carncross wrote: > I understand what you're saying about duplicates now. You're saying, > worst case scenario, UID 6 is downloaded twice- once as UID 6 and > the second time as UID 9. Yes. In particular, we can't cheat the gods of mathematics -- we'r

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-25 Thread Geo Carncross
On Mon, 2005-07-25 at 11:30 -0400, Mordechai T. Abzug wrote: > > Note: "B" however cannot ignore that message. It _MUST_ rewrite > > it. It's not acceptable for UID 6 to refer to a different message on > > each host. After that replication, a request for UID 6 must fail. > > No; my terminology ("

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-26 Thread Mordechai T. Abzug
On Mon, Jul 25, 2005 at 12:36:43PM -0400, Geo Carncross wrote: > It is. That's okay- the idea is that "B" will simulate a delete of > uid "6" as part of the replication, otherwise a client connecting to > "B" will see two copies of one message, and connecting to "A" it > will see two copies of a d

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-26 Thread Geo Carncross
On Mon, 2005-07-25 at 19:47 -0400, Mordechai T. Abzug wrote: > On Mon, Jul 25, 2005 at 12:36:43PM -0400, Geo Carncross wrote: > > > It is. That's okay- the idea is that "B" will simulate a delete of > > uid "6" as part of the replication, otherwise a client connecting to > > "B" will see two copi

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-27 Thread Mordechai T. Abzug
On Tue, Jul 26, 2005 at 10:30:21AM -0400, Geo Carncross wrote: > If a flag changes on UID 9 on host A and host B at nearly the same > time (between replications) which flag change wins? What if they're > not incompatible (one marking +FOOBaz and the other marking > \Answered ?) In order for that

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-27 Thread Geo Carncross
On Tue, 2005-07-26 at 22:46 -0400, Mordechai T. Abzug wrote: > On Tue, Jul 26, 2005 at 10:30:21AM -0400, Geo Carncross wrote: > > > If a flag changes on UID 9 on host A and host B at nearly the same > > time (between replications) which flag change wins? What if they're > > not incompatible (one m

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-27 Thread Mordechai T. Abzug
On Wed, Jul 27, 2005 at 11:40:07AM -0400, Geo Carncross wrote: > Because that's not what IMAP flags are. They're individual > annotations to a message, not a single annotation - as all the flags > and keywords combined. How does dbmail store flags? If it stored each flag as a separate entry in a

Re: [Dbmail-dev] multimaster replication and IMAP

2005-07-27 Thread Mordechai T. Abzug
On Wed, Jul 27, 2005 at 04:16:54PM -0400, Mordechai T. Abzug wrote: > The semantics of flags in a multimaster world are subtle. BTW: my main point from that whole email was not well-expressed. The key point is that there probably is no correct answer to what the user expects, in that it is likel