Re: looking for Cyrus mail format documentation

2003-02-06 Thread John Alton Tamplin
Phil Howard wrote: That would result in doubling the bandwidth on the inside server connection since it would be dealing with the mail first coming in to the MX, then being replicated back out to the other server. By delivering outside mail to the outside server first, the only bandwidth usage

Re: looking for Cyrus mail format documentation

2003-02-06 Thread Phil Howard
On Thu, Feb 06, 2003 at 09:58:30AM -0500, John Alton Tamplin wrote: | Phil Howard wrote: | | That would result in doubling the bandwidth on the inside server connection | since it would be dealing with the mail first coming in to the MX, then | being replicated back out to the other server. By

Re: looking for Cyrus mail format documentation

2003-02-05 Thread Patrick Welche
On Sat, Feb 01, 2003 at 11:31:13AM -0500, Rob Siemborski wrote: On Fri, 31 Jan 2003, Phil Howard wrote: | Of course replicating some things such as seen state will be quite | painful, and you may need to do some hacks to keep uids unique between | the machines. How does Cyrus manage

Re: looking for Cyrus mail format documentation

2003-02-05 Thread John Alton Tamplin
Patrick Welche wrote: All this sounds remarkably similar to the postgres-r database replication problem cf nice paper by Bettina Kemme http://www.cs.mcgill.ca/~kemme/papers/vldb00.html Here it would be client connects to imap server A and says APPEND. Server A then sends APPEND to server A and

Re: looking for Cyrus mail format documentation

2003-02-05 Thread Phil Howard
On Tue, Feb 04, 2003 at 10:49:21AM -0500, Rob Siemborski wrote: | Well, for one, this requires change to Cyrus, which Phil doesn't seem to | want to do. As long as the change is simple, I would not mind doing so. Making the UID so UID % NumberOfServers == ServerID holds true should be easy and

Re: looking for Cyrus mail format documentation

2003-02-05 Thread Phil Howard
On Wed, Feb 05, 2003 at 11:41:12AM +0900, Mark Keasling wrote: | It sounds like you may need to design a distributed mailstore that will | satisfy both your requirements and those of IMAP and then implement a | server around that mailstore. That was my original plan to do on top of Maildir. But

Re: looking for Cyrus mail format documentation

2003-02-05 Thread Phil Howard
On Tue, Feb 04, 2003 at 09:35:34PM -0800, David Lang wrote: | you stated that you want to have the outside box act as a secondary MX for | the inside one, if you do this and accept the extra bandwidth used then | you could still do this and have the mail only delivered to the inside box | and

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Phil Howard
On Tue, Feb 04, 2003 at 02:16:36AM -0500, Rob Siemborski wrote: | On Tue, 4 Feb 2003, Phil Howard wrote: | | Does the RFC say that the IMAP UIDs have to be the file name? | | No, of course not. | | Do the IMAP UIDs have to be the same between different sessions? | | They cannot change

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Mike Brodbelt
Rob Siemborski wrote: On Sat, 1 Feb 2003, Phil Howard wrote: | Doing replicated IMAP stores (espeically geographicly distanct ones) is | not an easy problem. It's easy if every message is a separate file. This is not true. It has nothing to do with the implementation of the mailstore. I've

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Phil Howard
On Tue, Feb 04, 2003 at 10:19:27AM +, Mike Brodbelt wrote: | Rob Siemborski wrote: | On Sat, 1 Feb 2003, Phil Howard wrote: | | Doing replicated IMAP stores (espeically geographicly distanct ones) is | | not an easy problem. | | It's easy if every message is a separate file. | | This is

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Phil Howard
On Tue, Feb 04, 2003 at 07:20:57PM +0900, Mark Keasling wrote: | Hi, | | On Tue, 4 Feb 2003 03:19:12 -0600, Phil Howard [EMAIL PROTECTED] wrote... | On Tue, Feb 04, 2003 at 02:16:36AM -0500, Rob Siemborski wrote: | | | On Tue, 4 Feb 2003, Phil Howard wrote: | | | | Does the RFC say that

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Henrique de Moraes Holschuh
On Tue, 04 Feb 2003, Phil Howard wrote: I wonder how well that method of replication works when both nodes cannot reach each other, and both are doing updates. And I wonder They don't. If they cannot reach each other, at most one of them must allow updates. -- One disk to rule them all,

Re: looking for Cyrus mail format documentation

2003-02-04 Thread John Alton Tamplin
Henrique de Moraes Holschuh wrote: On Tue, 04 Feb 2003, Phil Howard wrote: I wonder how well that method of replication works when both nodes cannot reach each other, and both are doing updates. And I wonder They don't. If they cannot reach each other, at most one of them must allow

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Phil Howard
On Tue, Feb 04, 2003 at 09:51:21AM -0500, John Alton Tamplin wrote: | Henrique de Moraes Holschuh wrote: | | On Tue, 04 Feb 2003, Phil Howard wrote: | | | I wonder how well that method of replication works when both nodes | cannot reach each other, and both are doing updates. And I wonder |

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Kendrick Vargas
I've been sorta following this thread, and I don't claim to know a whole lot about programming, but I'm wondering why something simple like what I'm about to suggest wouldn't work... Here goes: If a group of servers are gonna be in constant communication, why not just have each server assign

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Jason Fesler
However, had message IDs been the RFC822 message ID, then it would have been possible to receive new mail on each node. Since the IDs would not collide, that would be OK. If you did get the same ID delivered to both somehow, it should be the same mail. If not, it violates the RFC, so then

Re: looking for Cyrus mail format documentation

2003-02-04 Thread Mark Keasling
Hi, On Tue, 4 Feb 2003 05:57:53 -0600, Phil Howard [EMAIL PROTECTED] wrote... One thing I was thinking of would be a hack to make one server always use only odd UIDs, and the other always use only even UIDs, and to do catchups while they are reachable with each other. But this is getting into

Re: looking for Cyrus mail format documentation

2003-02-04 Thread David Lang
of the limitation more then the generic database two-way-sync problem David Lang On Tue, 4 Feb 2003, Phil Howard wrote: Date: Tue, 4 Feb 2003 03:19:12 -0600 From: Phil Howard [EMAIL PROTECTED] To: Rob Siemborski [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: looking for Cyrus mail format

Re: looking for Cyrus mail format documentation

2003-02-03 Thread Phil Howard
On Mon, Feb 03, 2003 at 09:18:47AM -0500, Rob Siemborski wrote: | On Sun, 2 Feb 2003, Phil Howard wrote: | | Apparently the way Cyrus does it, there are problems. But that does | not mean it cannot be done in general. By keeping a sequential number | and naming the files by that number

Re: looking for Cyrus mail format documentation

2003-02-03 Thread Rob Siemborski
On Tue, 4 Feb 2003, Phil Howard wrote: Does the RFC say that the IMAP UIDs have to be the file name? No, of course not. Do the IMAP UIDs have to be the same between different sessions? They cannot change without also chanigng the UIDVALIDITY of the mailbox, which is an expensive operation

Re: looking for Cyrus mail format documentation

2003-02-02 Thread Henrique de Moraes Holschuh
On Sat, 01 Feb 2003, Phil Howard wrote: | Doing replicated IMAP stores (espeically geographicly distanct ones) is | not an easy problem. It's easy if every message is a separate file. Unless the UIDs are UUIDs, it is NOT simple. And Cyrus does not use a UUID for every message, but rather an

Re: looking for Cyrus mail format documentation

2003-02-02 Thread Rob Siemborski
On Sat, 1 Feb 2003, Phil Howard wrote: So this new message was be appended to the same FILE? That sounds more like the old UNIX mailbox format. No. Same mailbox. Two servers are in sync, both with a UIDNEXT of 1000 for a particular mailbox. They suffer a netsplit and both have an APPEND

Re: looking for Cyrus mail format documentation

2003-02-02 Thread Phil Howard
On Sun, Feb 02, 2003 at 08:20:03PM -0500, Rob Siemborski wrote: | On Sat, 1 Feb 2003, Phil Howard wrote: | | So this new message was be appended to the same FILE? That sounds | more like the old UNIX mailbox format. | | No. Same mailbox. | | Two servers are in sync, both with a UIDNEXT of

Re: looking for Cyrus mail format documentation

2003-02-01 Thread Rob Siemborski
On Fri, 31 Jan 2003, Phil Howard wrote: | Of course replicating some things such as seen state will be quite | painful, and you may need to do some hacks to keep uids unique between | the machines. How does Cyrus manage uids? I hope these are not uids in /etc/passwd. No, they're the unique

Re: looking for Cyrus mail format documentation

2003-02-01 Thread Phil Howard
On Sat, Feb 01, 2003 at 11:31:13AM -0500, Rob Siemborski wrote: | On Fri, 31 Jan 2003, Phil Howard wrote: | | | Of course replicating some things such as seen state will be quite | | painful, and you may need to do some hacks to keep uids unique between | | the machines. | | How does Cyrus

looking for Cyrus mail format documentation

2003-01-31 Thread Phil Howard
A couple people have suggested to me that I use Cyrus-IMAP as opposed to Courier-IMAP, and have given some good arguments for that decision direction. However, I have still have one show stopper for that switch: some external programs that work directly with the storage space of all the mail.

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Earl R Shannon
Hello, One of the disadvantage of using Cyrus might be that there is no API to the mail store other than the IMAP protocol. You simply cannot go mucking around the mail store with external programs without the potential to cause problems. That said, mail is stored in directories that map unto

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Adam Tauno Williams
A couple people have suggested to me that I use Cyrus-IMAP as opposed to Courier-IMAP, and have given some good arguments for that decision direction. However, I have still have one show stopper for that switch: some external programs that work directly with the storage space of all the mail.

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Rob Siemborski
On Fri, 31 Jan 2003, Earl R Shannon wrote: Cyrus documentation calls the IMAP server a black box. This is defined to mean that the users do not have access to the account/data accept through the well defined ( :-/ ) IMAP protocol. This black box concept also extends to a certain extent to the

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Rob Siemborski
On Fri, 31 Jan 2003, Adam Tauno Williams wrote: SASL documentation?! There is some floating about. I love SASL, but the shreds of documentation are universally terrible. If you have specific suggestions as to what needs to be added, please let us know. If you actually have text, that'd be

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Brian
Adam Tauno Williams said: SASL documentation?! There is some floating about. I love SASL, but the shreds of documentation are universally terrible. AFAIK, this is also true for much of the software that CMU produces, however the quality of the software has been so good that no one seems to

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Earl R Shannon
Hello, You are correct on all counts. I was simply trying to make a point, and IMAP is the major protocol used to access the mail store, at least it is here. Nor did I mean to imply that any server that one set's up to see how things worked should be a production machine. In fact, because one

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Rob Siemborski
On Fri, 31 Jan 2003, Earl R Shannon wrote: But that does now beg the question. There must be some form of coordination between the various processes as they access the mail store. Can this not be abstracted out and put in an API to make it easier for people to write their own applications? I

Re: looking for Cyrus mail format documentation

2003-01-31 Thread John Alton Tamplin
Earl R Shannon wrote: But that does now beg the question. There must be some form of coordination between the various processes as they access the mail store. Can this not be abstracted out and put in an API to make it easier for people to write their own applications? I would venture a guess

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Rob Siemborski
On Fri, 31 Jan 2003, John Alton Tamplin wrote: The point is if you expose the internal API for accessing the mailstore you are stuck with it and can't make changes. I can't imagine there is a big need for this or other people wanting to write code to implement that API, so if you really want

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Phil Howard
On Fri, Jan 31, 2003 at 08:57:50AM -0500, Adam Tauno Williams wrote: | It does not use maildir. It actually can use several storage backends, flat | file to sleepcat and some others. Rumor of an SQL backend, that might be what | your looking for. SQL would be harder to do for what I'm doing.

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Phil Howard
On Fri, Jan 31, 2003 at 09:57:40AM -0500, John Alton Tamplin wrote: | Earl R Shannon wrote: | | But that does now beg the question. There must be some form of | coordination between the various processes as they access the | mail store. Can this not be abstracted out and put in an API to |

Re: looking for Cyrus mail format documentation

2003-01-31 Thread John A. Tamplin
Phil Howard wrote: One of the needs I have is to build a two-way mail store replica. Either node may be delivered to, and either node may be accessed by the user but only one at a time. The two nodes are topologically and geographically far apart, and bandwidth between them is to be considered

Re: looking for Cyrus mail format documentation

2003-01-31 Thread Phil Howard
On Fri, Jan 31, 2003 at 08:33:41PM -0500, John A. Tamplin wrote: | Phil Howard wrote: | | One of the needs I have is to build a two-way mail store replica. Either | node may be delivered to, and either node may be accessed by the user but | only one at a time. The two nodes are topologically