Re: Funding Cyrus High Availability

2004-09-20 Thread Paul Dekkers
David Carter wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to implementing the replication needed for #1 modify all

Re: Funding Cyrus High Availability

2004-09-20 Thread David Carter
On Sun, 19 Sep 2004, David Lang wrote: here is the problem. you have a new message created on both servers at the same time. how do you allocate the UID without any possibility of stepping on each other? With a new UIDvalidity you can choose any ordering you like. Of course one of the two

Re: Funding Cyrus High Availability

2004-09-20 Thread David Carter
On Sun, 19 Sep 2004, David Lang wrote: assiming that the simplest method would cost ~$3000 to code I would make a wild guess that the ballpark figures would be 1. active/passive without automatic failover $3k 2. active/passive with automatic failover (limited to two nodes or withing a murder

Re: Funding Cyrus High Availability

2004-09-20 Thread David Lang
On Mon, 20 Sep 2004, David Carter wrote: On Sun, 19 Sep 2004, David Lang wrote: assiming that the simplest method would cost ~$3000 to code I would make a wild guess that the ballpark figures would be 1. active/passive without automatic failover $3k 2. active/passive with automatic failover

Re: Funding Cyrus High Availability

2004-09-20 Thread David Lang
On Mon, 20 Sep 2004, Paul Dekkers wrote: David Carter wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to implementing the

Re: Funding Cyrus High Availability

2004-09-20 Thread David Carter
On Mon, 20 Sep 2004, David Lang wrote: Thanks, this is exactly the type of feedback that I was hopeing to get. so you are saying that #5 is more like $50k-100k and #6 goes up from there If anyone could implement Active-Active for Cyrus from scratch in 100 to 150 hours it would be Ken, but I

RE: Funding Cyrus High Availability

2004-09-19 Thread David Lang
On Fri, 17 Sep 2004 [EMAIL PROTECTED] wrote: From: David Lang [mailto:[EMAIL PROTECTED] Mike, one of the problems with this is that different databases have different interfaces and capabilities. if you design it to work on Oracle then if you try to make it work on MySQL there are going to be

Re: Funding Cyrus High Availability

2004-09-19 Thread David Lang
There are many ways of doing High Availability. This is an attempt to outline the various methods with the advantages and disadvantages. Ken and David (and anyne else who has thoughts on this) please feel free to add to this. I'm attempting to outline them roughly in order of complexity. 1.

Re: Funding Cyrus High Availability

2004-09-19 Thread David Carter
On Sun, 19 Sep 2004, David Lang wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to implementing the replication needed for #1

Re: Funding Cyrus High Availability

2004-09-19 Thread David Lang
On Sun, 19 Sep 2004, David Carter wrote: On Sun, 19 Sep 2004, David Lang wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to

Re: Funding Cyrus High Availability

2004-09-19 Thread Jure Pe_ar
On Sun, 19 Sep 2004 00:52:08 -0700 (PDT) David Lang [EMAIL PROTECTED] wrote: Nice review of replication ABC :) Here are my thoughts: 1. Active-Slave replication with manual failover This is really the simplest way to do it. Rsync (and friends) does 90% of the required job here; the only thing

Re: Funding Cyrus High Availability

2004-09-19 Thread David Lang
please don't misunderstand my posts. it's not that I don't think that active/active/active is possible, it's just that I think it's far more complicated. assiming that the simplest method would cost ~$3000 to code I would make a wild guess that the ballpark figures would be 1. active/passive

Re: Funding Cyrus High Availability

2004-09-19 Thread Michael Loftis
--On Monday, September 20, 2004 00:43 +0200 Jure Pe ar [EMAIL PROTECTED] wrote: On Sun, 19 Sep 2004 00:52:08 -0700 (PDT) David Lang [EMAIL PROTECTED] wrote: Nice review of replication ABC :) Here are my thoughts: 1. Active-Slave replication with manual failover This is really the simplest way

Re: Funding Cyrus High Availability

2004-09-18 Thread Paul Dekkers
Hi, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to be to teach murder about it so that

Re: Funding Cyrus High Availability

2004-09-18 Thread David Carter
On Fri, 17 Sep 2004, Ken Murchison wrote: Actually what I was really asking, is are people looking for an active-active config and an active-passive config? I'm not sure that IMAP is ameniable to active-active: the prevalence of UIDs in the protocol means that it would be very hard to resolve

Re: Funding Cyrus High Availability

2004-09-18 Thread Derrick J Brashear
On Sat, 18 Sep 2004, Ken Murchison wrote: I'm not sure that IMAP is ameniable to active-active: the prevalence of UIDs in the protocol means that it would be very hard to resolve the inconsistencies that would occur if a pair of machines ever lost touch. Right, I was assuming that active-passive

Re: Funding Cyrus High Availability

2004-09-17 Thread Paul Dekkers
Hi, Eric S. Pulley wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or

Re: Funding Cyrus High Availability

2004-09-17 Thread Lee
mysql does not have multi-master functionality, and it's replication, is quite honestly, a joke. You may have mis-spoken and are talking about the up-and-coming mysql cluster or the mysql max product (both of which i'm much less familiar with). Indeed i was talking about mysql cluster (which

Re: Funding Cyrus High Availability

2004-09-17 Thread Paul Dekkers
Hi, Ken Murchison wrote: I think this would cause performance to suffer greatly. I think what we want is lazy replication, where the client gets instant results from the machine its connected to, and the replication is done in the background. I believe this is what David's implementation

Re: Funding Cyrus High Availability

2004-09-17 Thread David Lang
On Fri, 17 Sep 2004, Paul Dekkers wrote: Date: Fri, 17 Sep 2004 08:25:26 +0200 From: Paul Dekkers [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Funding Cyrus High Availability Hi, Eric S. Pulley wrote: Question: Are people looking at this as both redundancy

Re: Funding Cyrus High Availability

2004-09-17 Thread Paul Dekkers
David Lang wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or date/time.

Re: Funding Cyrus High Availability

2004-09-17 Thread David Carter
On Fri, 17 Sep 2004, Paul Dekkers wrote: Isn't it possible to have equal roles? If all changes are put in some backlog, and a synchroniser process runs on both machines and pushes the backlog (as soon as there is any) to another machine... then you can have the some process on both (equal)

Re: Funding Cyrus High Availability

2004-09-17 Thread Jure Pe_ar
On Fri, 17 Sep 2004 08:25:26 +0200 Paul Dekkers [EMAIL PROTECTED] wrote: I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with

Re: Funding Cyrus High Availability

2004-09-17 Thread David Carter
On Fri, 17 Sep 2004, Jure [ISO-8859-2] Pe_ar wrote: So how does this cyrus in a raid view sound? It should probalby be called raims for redundand array of inexpensive mail servers anyway ;) We call it RAIN: Redundant Array of Inexpensive Nodes. Really cheap Intel servers in our case :) -- David

Re: Funding Cyrus High Availability

2004-09-17 Thread Earl R Shannon
Hello, All that you say is true. But for performance one either buys bigger and better or multiple machines to spread the load. Murder allows one to buy multiple machines. All I am saying is that improving perforance may already be done. I believe redundancy in the application is more important at

Re: Funding Cyrus High Availability

2004-09-17 Thread Ken Murchison
David Lang wrote: On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to

Re: Funding Cyrus High Availability

2004-09-17 Thread Lee
My vote would be for active/active, its usually more reliable and of course it builds in better scaleability. I imagine the the main question of everyone will be how the choice of active/active or active/passive will effect cost/time of implementation. L On Sep 17, 2004, at 1:16 PM, Ken

Re: Funding Cyrus High Availability

2004-09-17 Thread David Lang
On Fri, 17 Sep 2004, Ken Murchison wrote: David Lang wrote: On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have

RE: Funding Cyrus High Availability

2004-09-17 Thread szii
My biggest question here is, simply, why recreate what's already out there? There are a number of projects (LVM, PVFS) which do this kind of replication/distribution/virtulization for filesystems. There are a number of databases which have active/active clustering (mysql, DB2, Oracle, et al) and

Re: Funding Cyrus High Availability

2004-09-17 Thread Jure Pe_ar
On Fri, 17 Sep 2004 13:28:08 -0700 [EMAIL PROTECTED] wrote: My biggest question here is, simply, why recreate what's already out there? Because none of the existing solutions does not fit our needs well enough. There are a number of projects (LVM, PVFS) which do this kind of

RE: Funding Cyrus High Availability

2004-09-17 Thread David Lang
On Fri, 17 Sep 2004 [EMAIL PROTECTED] wrote: My biggest question here is, simply, why recreate what's already out there? There are a number of projects (LVM, PVFS) which do this kind of replication/distribution/virtulization for filesystems. There are a number of databases which have active/active

RE: Funding Cyrus High Availability

2004-09-17 Thread szii
-Original Message- From: David Lang [mailto:[EMAIL PROTECTED] Sent: Friday, September 17, 2004 2:25 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: Funding Cyrus High Availability Mike, one of the problems with this is that different databases have different

Re: Funding Cyrus High Availability

2004-09-16 Thread Ken Murchison
Lee wrote: What do people think about a bounty program like horde's: http://www.horde.org/bounties/ Basically people can make paypal donations to fund certain features. For something like the high availability support, Im guessing that ALOT of people would donate small to large amounts of cash

Re: Funding Cyrus High Availability

2004-09-16 Thread [EMAIL PROTECTED]
Hello All, I would be willing to pay for this function. Though I am just a startup, and have very little capital. Most I could prolly do is $100 to $200. Not much. My fear, which maybe the fear of others is the risk of putting money in, but there not being enough support by others to reach the

Re: Funding Cyrus High Availability

2004-09-16 Thread Lee
I imagine for a big project like this, refunds could be given. I think its more a matter of finding someone to deal with this. Id be happy to do it, but i think it would be best if Ken or another core developer that everyone knows and already trusts is in charge of holding the cash. Any Ideas

Re: Funding Cyrus High Availability

2004-09-16 Thread Michael Loftis
--On Thursday, September 16, 2004 18:13 -0400 Lee [EMAIL PROTECTED] wrote: cut P.S. Ken, not sure if this would be easier or more complex, but another alternative here might be to write a mysql backend to cyrus, which would eliminate the need to worry about redundancy given mysql's multimaster

Re: Funding Cyrus High Availability

2004-09-16 Thread Ken Murchison
Lee wrote: I imagine for a big project like this, refunds could be given. I think its more a matter of finding someone to deal with this. Id be happy to do it, but i think it would be best if Ken or another core developer that everyone knows and already trusts is in charge of holding the cash.

Re: Funding Cyrus High Availability

2004-09-16 Thread David Lang
On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to be to teach murder

Re: Funding Cyrus High Availability

2004-09-16 Thread Eric S. Pulley
--On Thursday, September 16, 2004 6:56 PM -0400 Ken Murchison [EMAIL PROTECTED] wrote: [SNIP] Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control

Re: Funding Cyrus High Availability

2004-09-16 Thread Earl Shannon
Hello, Question: Are people looking at this as both redundancy and performance, or just redundance? My $0.02 worth. Performance gains can be found the traditional way, ie, faster hardware, etc.Our biggest need is for redundance. If something goes wrong on one machine we still need to be

Re: Funding Cyrus High Availability

2004-09-16 Thread Michael Loftis
--On Thursday, September 16, 2004 22:14 -0400 Earl Shannon [EMAIL PROTECTED] wrote: Hello, Question: Are people looking at this as both redundancy and performance, or just redundance? My $0.02 worth. Performance gains can be found the traditional way, ie, faster hardware, etc.Our biggest

Re: Funding Cyrus High Availability

2004-09-16 Thread Fabian Fagerholm
On Thu, 2004-09-16 at 18:56 -0400, Ken Murchison wrote: I think this would cause performance to suffer greatly. I think what we want is lazy replication, where the client gets instant results from the machine its connected to, and the replication is done in the background. I believe this