Re: Funding Cyrus High Availability
Hi, Eric S. Pulley wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or date/time. I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_. Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating... Paul --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
mysql does not have multi-master functionality, and it's replication, is quite honestly, a joke. You may have mis-spoken and are talking about the up-and-coming mysql cluster or the mysql max product (both of which i'm much less familiar with). Indeed i was talking about mysql cluster (which is now included with teh distro). Im pretty convinced having talked with some mysql peeps, that cluster will eventually (not too distant future) be VERY bullet proof. I just figured that writing cyrus to use mysql (or SQL SPEC) as a backend might kill two birds with one stone, and create a better general platforms for growth. None the less, id would love to see just replication is everyone if mysql back is out. L cut --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
Hi, Ken Murchison wrote: I think this would cause performance to suffer greatly. I think what we want is lazy replication, where the client gets instant results from the machine its connected to, and the replication is done in the background. I believe this is what David's implementation does. Yes, but if I understood it well it is per action, and not long after the action was performed on one of the machines. (It should at least not take long, but get in queue/backlog or something for the background process? I'm not sure how it's done in David's patch, and neither if that is really what we should go for, but that's up to you developers :-)) In my other reply: I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_. It plays a role that in our situation there is also spamassassin running on the servers: if that could be distributed because one mail can be delivered to one box and another one to the other that would already mean quite some load-balancing: and then we have not taken the load of cyrus into account :-) Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating... I'm not sure about that, btw: I'm no good programmer, but I can imagine that this is a something you want. If one server is down it should mean that all tasks can be performed at the other one. I 'm curious how this would look if both servers are still running but cannot reach eachother. If there is indeeed a UUID: what if there are doubles... but I guess that has been taken into account. Paul --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004, Paul Dekkers wrote: Date: Fri, 17 Sep 2004 08:25:26 +0200 From: Paul Dekkers [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Funding Cyrus High Availability Hi, Eric S. Pulley wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or date/time. I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_. Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating... right, but there are already tools freely available on most platforms to do the election and changing of the role (by switching between config files and restarting the master) what is currently lacking is any ability to do the master/slave role. once we have that it's just a little scripting to tie just about any failover software in to make it automatic. one thing we need to watch out for here is that we don't set an impossible/unreasonable goal. don't try to solve every problem and add every availablity feater you can imagine all at once. instead let's look at the building blocks that are needed and identify what's currently not available. currently we have murder which will spread the load across multiple machines. currently we have many tools available to detect a server failure and run local scripts to reconfigure machines (HACMP on AIX, hearbeat for Linux, *BSD, Solaris, etc) what we currently do not have is any ability to have one mailstore updated to match changes in another one. once we have that ability there are many things that can be built by glueing togeather existing code. once we have a bit of experiance with people actually useing these features it will then be obvious which features need better integration with Cyrus and which make sense to remain seperate. I also would not be really satisfied with interval synchronisation as the only choice. I think we need something where the primary mailstore pushes a record of it's changes to the secondary mailstore This can then be tweaked in several directions. 1. locking can be added so that the primary doesn't complete it's command until the secondary says it has a permanent record of the change (two-phase commit or a reasonable facimily of such) 2. batch up the changes until they hit some threshold (size or time or combination) and then send a batch of changes all at once 3. recongnise it's own changes to gain the ability to push updates in both directions at the same time (true two-phase commit with bi-directional replication, some horribile performance pathalogical cases, but attractive in some cases) or other varients but these all share the same common need the ability for the master to output all it's changes and the ability for a slave to read such changes and update itself to match the nice thing is that with IMAP much of the data needed is already output, you could do a first approximation of this with a client that opened a seperate connection to every folder on the primary server and just sat watching for server messages and whenever it saw an update send the matching command to the slave (fetching the full data as needed to get all the info). this obviously won't scale to any reasonalbe size, but this means that most of what's needed is already identified so the core could be just a common output of the exisitng messages with a little more data (mailbox and folder in most cases, message contents in a few) let's get these small, but critical pieces done and then we can grow and experiment from there. David Lang Paul --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
No lmtp / pop authentication on backend
Hello, as Ken Murchison told me you could help me on my authentication problems, I send you the scenario. My main problem is to make Frontend services like LMTP and POP authenticate against backend. Users are capable to authenticate either as SASLDB or as LDAP users. And at the bottom of the present messages an architecture overview of what I want to reach. Thank you if you can help !! Laurent GAUTHIER wrote: Laurent GAUTHIER wrote: Hello, I have an architecture of Murder Cyrus Imap Servers with Postfix. The design makes Postfix Frontend deliver SMTP to local LMTP service. My problem is when the Cyrus LMTP service (unix socket) on Frontend tries to connect to backend LMTP (TCP/IP socket) , I have messages bellow : Frontend: Sep 13 19:25:29 localhost cyrus/lmtpproxyd[19904]: lmtpengine do_auth: failed to authenticate Backend: Sep 13 19:29:55 localhost cyrus/lmtpd[2210]: connection from cyrfro.mydomain.com [182.69.69.44] Sep 13 19:29:58 localhost cyrus/lmtpd[2210]: badlogin: 182.69.69.44 DIGEST-MD5 SASL(-13): user not found: no secret in database Is there any parameter as the ones for mupdate like # mupdate_username # mupdate_authname # mupdate_realm # mupdate_password that would be dedicated to LMTP? In other words, how to configure the frontend lmtp service auth-name when it intents to connect to the back-end? From doc/install-murder.html: Additionally, you will need entries in imapd.conf to indicate the proxy auth name and passwords (if you are using a SASL mechanism that requires them) to the backends, for example, if your backends are mail1.andrew.cmu.edu and mail2.andrew.cmu.edu with passwords of foo and bar, and an auth name of murder: mail1_password: foo mail2_password: bar proxy_authname: murder -- Kenneth Murchison Oceana Matrix Ltd. I'd already configured these entries. In order for you to have a good idea of my configuration, I join you my conf files and additionnal piece of information. For the moment I use sasldb, but my real need is to authenticate against LDAP. I tried that with a single Cyrus IMAP and worked with SASL configured on LDAP. As with the murder configuration it didn't work as I expected, I went back to sasldb. Maybe I mistake between common users and specific rights Cyrus services users. What is the typical configuration on a murder, as I should have in my final architecture several fronts and backs in order to accept between 60 000 and 100 000 users? If your trained eyes can see any mistake ... it would permit me to implement the LDAP authentication. I don't use LDAP authentication at all, so I probably won't be much help. You should post to the list and SASL/LDAP experts such as Igor can probably help. By the way, I will next test Horde IMP to access backend mailboxes. Where would you install it, as you know prety well Cyrus imap architecture? On front ends ? On independant servers configured to talk with front ends? Either way would work. You just need to figure out if the frontends can handle the load of doing both. You might want to look into using an IMAP proxy (http://www.imapproxy.org/) for webmail. I'd recommend installing this on the same machine as IMP. -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp frontback-imapd.conf Description: Binary data
Re: No lmtp / pop authentication on backend
Am Freitag, 17. September 2004 09:13 schrieb Laurent GAUTHIER: as Ken Murchison told me you could help me on my authentication problems, I send you the scenario. My main problem is to make Frontend services like LMTP and POP authenticate against backend. Users are capable to authenticate either as SASLDB or as LDAP users. And at the bottom of the present messages an architecture overview of what I want to reach. Backend: Sep 13 19:29:55 localhost cyrus/lmtpd[2210]: connection from cyrfro.mydomain.com [182.69.69.44] Sep 13 19:29:58 localhost cyrus/lmtpd[2210]: badlogin: 182.69.69.44 DIGEST-MD5 SASL(-13): user not found: no secret in database Cyrus-SASL starts a query to your auxprop-database with $username and $realm. The $realm is equal with your servername or if configured with $servername in /etc/imapd.conf. If there is no dataset equal to both, you will see user not found. Check your sasldblistusers on the Backend-Server if the Username and Realm is equal. I would guess the realm is wrong. Change your servername in imapd.conf to $realm to satisfy sasl. sasldblistusers: [EMAIL PROTECTED] userPassword -- Andreas --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
David Lang wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? Cyrus performs pretty well already. Background redundancy would be awesome. Especially if we had control over when the syncing process occurred either via time interval or date/time. I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_. Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating... right, but there are already tools freely available on most platforms to do the election and changing of the role (by switching between config files and restarting the master) what is currently lacking is any ability to do the master/slave role. once we have that it's just a little scripting to tie just about any failover software in to make it automatic. There are indeed tools available for that, but they're not always working as they're supposed to do and are often very OS limited. With FreeBSD I had no luch with heartbeat (wouldn't compile under FreeBSD-5), (U)CARP was not available and FreeVRRP was buggy (at least in my case, sometimes I had two masters). Also I wouldn't like it when restarting the cyrus-process with a different config-file is necessary (or there must be a seperate process for synchronising that needs restarting, that would make it better). That would still kill connections to that cyrus-process, I'd rather see a software switch between that role. Isn't it possible to have equal roles? If all changes are put in some backlog, and a synchroniser process runs on both machines and pushes the backlog (as soon as there is any) to another machine... then you can have the some process on both (equal) servers... Of course there needs to be some more intelligence, but that's basicly what I would expect. one thing we need to watch out for here is that we don't set an impossible/unreasonable goal. I agree that we'll have to define properly what we expect and what is reasonable, but I think that at this moment Ken (as developer) has the best overview in this. We offer our wishlist, and I suppose he translates that to code in his head ;-) I suppose that's why he came up with the question about performance versus redundancy/availability. don't try to solve every problem and add every availablity feater you can imagine all at once. instead let's look at the building blocks that are needed and identify what's currently not available. I don't agree there completely: I don't want to depend on yet another tool that defines what the master or slave is. Sometimes they don't work at all, work only at the same LAN, ... I'm not sure if you can count on that. (Hmm, you're the first that mentions the clustering software for defining roles, and I didn't read about this on your website either. This is new to me.) currently we have murder which will spread the load across multiple machines. Yes, that's indeed something we don't need looking at :-) (Although there is a posibility now to spread load as well of course, with two machines available at the same time...) currently we have many tools available to detect a server failure and run local scripts to reconfigure machines (HACMP on AIX, hearbeat for Linux, *BSD, Solaris, etc) what we currently do not have is any ability to have one mailstore updated to match changes in another one. I would combine these two, and I think that can be done by just well-designing the last thing you mention. I also would not be really satisfied with interval synchronisation as the only choice. In my sketch above (really not sure if it works of course) where both have something like a backlog you can like tail that backlog and push the update as soon as possible to the second machine. You solve the thing you mention with delays while pushing updates to two servers at the same time. I think we need something where the primary mailstore pushes a record of it's changes to the secondary mailstore Why not also vise versa?! We want the two servers to be accessible at the same time, right? If one server is down it should mean that all tasks can be performed at the other one. I 'm curious how this would look if both servers are still running but cannot reach eachother. If there is indeeed a UUID: what if there are doubles... but I guess that has been taken into account. In cluster terminology this situation is known as being 'split-brained' and is generally viewed as a 'VERY BAD THING' that each cluster software solves in a slightly
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004, Paul Dekkers wrote: Isn't it possible to have equal roles? If all changes are put in some backlog, and a synchroniser process runs on both machines and pushes the backlog (as soon as there is any) to another machine... then you can have the some process on both (equal) servers... Of course there needs to be some more intelligence, but that's basicly what I would expect. We have 16 servers: half the accounts on each system are master copies and half are replicas. Each machine has a small database (a CDB lookup file) to tell it whether a given account is master or slave. The replication engine (which runs independently from the normal master spawned jobs) bails out rapidly if the replica copy of an account is updated: it would proceed to transform the master into a copy of the replica, but that's probably not what you wanted :). I have a tool which allows me to switch the master and replica copy for any (inactive) account without having to shut anything down. This tool also lets me migrate data off onto a third system and immediately create a replica of that. This makes upgrading operating systems a much less fraught task. In my sketch above (really not sure if it works of course) where both have something like a backlog you can like tail that backlog and push the update as soon as possible to the second machine. You solve the thing you mention with delays while pushing updates to two servers at the same time. Yes, that's exactly how my code works. Asynchronous replication (which Ken called lazy replication) is fairly easy to do in Cyrus. Synchronous replication, where you only get a response to an IMAP/POP/LMTP command when the data is safely committed to the replica, would involve a much more substantial rewrite of the Cyrus code. That's where block based replication schemes like DRDB have a big advantage: the state that they have to track is much less involved. I'm currently running with a replication cycle of one second on my live servers for rolling replication (that's just a name I made up, its not an official term), so on average we would lose of half a second of update traffic for 1/16th of our user base if a single system failed. Further safeguards are possible by keeping copies of incoming mail for a short time on the MTA systems, but that's not really a Cyrus concern. We also replicate to a tape backup spooling engine overnight. The replication engine is rather useful for fast incremental updates. If one server is down it should mean that all tasks can be performed at the other one. I 'm curious how this would look if both servers are still running but cannot reach eachother. If there is indeeed a UUID: what if there are doubles... but I guess that has been taken into account. UUIDs are just a convenient representation of message text, so that you can pass messages by reference rather than value. Duplicates don't matter (though I don't believe that they actual occur given my allocation scheme) so long as the message text is the same. I maintain databases of MD5 checksums for messages and cache text just to be on the safe side. UUIDs were originally just Mailbox UniqueID + Message UID. Unfortunately, UniqueID isn't very Unique: its just a simple hash of the mailbox name. I ended up allocating UUIDs in large chunks from the master process on each machine. If a process runs out of UUIDS (which would take some going as they are allocated in chunks of 2**24), it falls back to call by value. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004 08:25:26 +0200 Paul Dekkers [EMAIL PROTECTED] wrote: I would say not at an interval but as soon as there is an action performed on one mailbox, the other one would be pushed to do something. I believe that is called rolling replication. I would not be really happy with a interval synchronisation. It would make it harder to use both platforms at the same time, and that is what I want as well. So there is a little-bit of load-balancing involved, but more and more _availability_. Being able to use both platforms at the same time maybe implies that there is either no master/slave role or that this is auto-elected between the two and that this role is floating... Paul I'm jumping back into this thread a bit late ... My feeling is that most of cyrus instalations run one or a few domains with many users; at least that is my case. That's why i'd base any kind of replication we come up with on the mailbox as the base unit. As raid uses disk block for its unit, so would we use mailbox (with all its subfolders). In a way that one would be able to take care of the whole domains on the higher level, if needed. Today we have the option of using murder (or perdition, with some added logic) when more than one backend machine is needed. This brings us a kind of raid linear (linux md speak) or concatenation of space into a single mailstore. With all the 'features' of such setup: if you lose one machine(disk), all users(data) on that machine(disk) are not available. So what i'm thinking is we need is a kind of raid1 or mirroring of mailboxes. Imagine user1 having its mailbox on server1 and server2, user2 on server2 and server3, user3 on server3 and server1 ... for example. Murder is already a central point with a knowledge of where a certain mailbox is and how to proxy pop, imap and lmtp to it and in my way of seeing things, it would be best to teach it how to handle this 'mirroring' too. Let say one of the two mailboxes is primary, and the other is secondary; murder connects to the primary, lets the client do whatever it wants and then replays the exact same actions to the secondary mailbox. If this is done after the primary disconnects or while the client is still talking to the primary, is implementation detail. Performance bonus: connect to both mailboxes at once and pronounce as primary the one that responds faster :) Murder would have to know how to record and playback the whole client-server dialogues. Considering that there's already a system in cyrus that lets admin see the 'telemetry' of the imap conversation, i guess this could be extended and tied into murder. So far this is just how clients would talk to our system. What else would we need? Certanly a mechanism to manually move mailboxes between servers in a way that murder knows about the changes. Thinking of it, mupdate protocol already knows how to push metadatas around; why not extend it so it can also move mailboxes? Or should perl mupdate module be born and then some scripts should be written with it and imap? Then maybe some mechanism for murder to deceide on which servers to put newly created mailboxes on. Ideally this would be plugin based with different policies (load, disk space, responsiveness, combination of those, something else), but a simple round robin would do for a start. For those that do not want to have mailboxes in sync, a mechanism to delay updates to the secondary mailbox. (In this case, which mailbox is primary and which is secondary should not change) Also a way of handling huge piles of backlogs in case one of the machines is down for a longer period of time. Maybe a mechanism to sync the mailbox from the other server and discarding the backlogs would be handy in such case. And a way to manually trigger such resync on a specific mailbox. Probalby something else i can't think of right now. So how does this cyrus in a raid view sound? It should probalby be called raims for redundand array of inexpensive mail servers anyway ;) This way all the logic is done in one place and you only have to take good care (in a HA sense) of the mupdate master machine. Others can remain cheap and relatively dumb than can be pulled offline at will. Given fast enough and reliable links, this could also work in a geographycally distributed manner. Ken, is something like this reasonable? Oh, i'd like to know what fastmail.fm folks think about all this HA thing. I'm sure they have some interesting insights :) -- Jure Pear --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004, Jure [ISO-8859-2] Pe_ar wrote: So how does this cyrus in a raid view sound? It should probalby be called raims for redundand array of inexpensive mail servers anyway ;) We call it RAIN: Redundant Array of Inexpensive Nodes. Really cheap Intel servers in our case :) -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
Hello, All that you say is true. But for performance one either buys bigger and better or multiple machines to spread the load. Murder allows one to buy multiple machines. All I am saying is that improving perforance may already be done. I believe redundancy in the application is more important at this point. Regards, Earl Shannon Michael Loftis wrote: --On Thursday, September 16, 2004 22:14 -0400 Earl Shannon [EMAIL PROTECTED] wrote: Hello, Question: Are people looking at this as both redundancy and performance, or just redundance? My $0.02 worth. Performance gains can be found the traditional way, ie, faster hardware, etc.Our biggest need is for redundance. If something goes wrong on one machine we still need to be able to let users use email. Cyrus already has this solved via MURDER, but FWIW, more smaller boxes isolate failures more effectively than one big box, also price/performance is still better at a certain size for any platform, and going up higher on the performance curve has HUGE price jumps. There's also the cost of administering multiple separate boxes to think about but carefully planned, this can be managed rather easily. The whole 'throw bigger and bigger boxen' at it method of 'scaling' doesn't scale. You hit the wall. One box can only do so much, granted you can spend LOTS of money and get pretty big boxes, but at some point it becomes ludicrous -- who would use a Sun E10k/E15k and a whole Symmetrix DMX for just mail? (and I'm excluding companies like AOL and IBM who actually can afford it and would maybe have a reason to scale to that size)... Price/Performance has a curve associated with it, most of us can't afford to always stay at the top end of the curve, and have to be at the middle. Further, does it make sense to re-invest in equipment every year to maintain growth? No, you should be able to expand, add another box, or two, and that scales fairly well. Better than the single big box approach. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Systems Programmer ,Information Technology Division NC State University. http://www.earl.ncsu.edu Anonymous child Some people can tell the time by looking at the sun, but I have trouble seeing the numbers. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Is there a limit for the long size of mailbox name in cyrus-IMAPd?
Wang Penghui wrote: Hi all: Does cyrus-imapd server have the limit of long size of mailbox name? For example: A virtual domain named: my.example.com And the login name is [EMAIL PROTECTED] Is there any limit with the login name? If yes, what's it? I believe this is MAX_MAILBOX_NAME which is set to 490. -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
David Lang wrote: On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to be to teach murder about it so that it can failover to the backup box(s) as needed, but for now simply having the full data at the backup location would be so far ahead of where we are now that the need to reconfigure murder for a failover is realitivly trivial by comparison. Actually what I was really asking, is are people looking for an active-active config and an active-passive config? -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Permissions change on /var/imap/quota
Hello! I'm trying to read /var/imap/quota/letter/user.letttersurname and is impossible. I changed permissions to give access to all users in the mail group, but [i don't know why] the permissions are resotred to: cyrus.mail 600 then, mail users cannot read. How can i change that? [i think is a umask problem, but how can i configure that] thanks a lot, Félix -- Felix Cuello [EMAIL PROTECTED] - 1493 - Seeing that death, a necessary end, Will come when it will come. -- William Shakespeare, Julius Caesar --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
My vote would be for active/active, its usually more reliable and of course it builds in better scaleability. I imagine the the main question of everyone will be how the choice of active/active or active/passive will effect cost/time of implementation. L On Sep 17, 2004, at 1:16 PM, Ken Murchison wrote: David Lang wrote: On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to be to teach murder about it so that it can failover to the backup box(s) as needed, but for now simply having the full data at the backup location would be so far ahead of where we are now that the need to reconfigure murder for a failover is realitivly trivial by comparison. Actually what I was really asking, is are people looking for an active-active config and an active-passive config? -- Kenneth Murchison Oceana Matrix Ltd. Software Engineer 21 Princeton Place 716-662-8973 x26 Orchard Park, NY 14127 --PGP Public Key--http://www.oceana.com/~ken/ksm.pgp --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004, Ken Murchison wrote: David Lang wrote: On Thu, 16 Sep 2004, Ken Murchison wrote: Question: Are people looking at this as both redundancy and performance, or just redundance? for performance we already have murder, what we currently lack is redundancy. once we have redundancy then the next enhancement is going to be to teach murder about it so that it can failover to the backup box(s) as needed, but for now simply having the full data at the backup location would be so far ahead of where we are now that the need to reconfigure murder for a failover is realitivly trivial by comparison. Actually what I was really asking, is are people looking for an active-active config and an active-passive config? I think that everyone would love to have the active-active option, the problem I have with this is that the active-passive config will solve many peoples problems and I believe that is will be far simpler to do so I don't want the ideal goal of active-active to end up side tracking the huge progress that would be achieved by active-passive. active-active also requires significantly different choices if the nodes are seperated by significant distances. I'd hate to end up with an active-active solution that works only with the machines all local and still have no solution to the disaster recovery senerio. David Lang -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Funding Cyrus High Availability
My biggest question here is, simply, why recreate what's already out there? There are a number of projects (LVM, PVFS) which do this kind of replication/distribution/virtulization for filesystems. There are a number of databases which have active/active clustering (mysql, DB2, Oracle, et al) and master/slave. Personally, I would LOVE to see a full RDBMS-backed system. You define your database(s) in the config file ... and that is all. All configuration options are stored on the central RDBMS. All mailboxes are stored there. You can then rely 100% on the RDBMS systems for clustering/failover/scalability/backing up ... all datastorage domain problems which they have already addressed/solved. If you want to scale out it's a matter of 1) install the cyrus software 2) Point the config file at the database server 3) Entry in the database server/cluster to allow the new frontend/proxy. 4) Fire up the daemons 5) Enjoy. The other advantages would be very nice integration with other applications which can query against databases. (ex: postfix directly supports mysql lookups.) But then, I can't afford to really help with this myself so take my thoughts with a big hope pill. =D -Mike/Szii --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Fri, 17 Sep 2004 13:28:08 -0700 [EMAIL PROTECTED] wrote: My biggest question here is, simply, why recreate what's already out there? Because none of the existing solutions does not fit our needs well enough. There are a number of projects (LVM, PVFS) which do this kind of replication/distribution/virtulization for filesystems. We're discussing replication on the application level. Block level replication is nice for many things, but doesn't really take care of consistency, which cyrus relies on pretty much. There are a number of databases which have active/active clustering (mysql, DB2, Oracle, et al) and master/slave. Personally, I would LOVE to see a full RDBMS-backed system. You define your database(s) in the config file ... and that is all. You can go with dbmail and one of the existing well established databases anytime. This can solve the issue we're having here, but brings lots of other problems that cyrus is keeping away. Just ask any Exchange admin :) The other advantages would be very nice integration with other applications which can query against databases. (ex: postfix directly supports mysql lookups.) For mail routing auth, yes ... many of us are already doing this. However, storing mail in a db gives you about 20% of db overhead (straight from the Oralce sales rep) and i/o is already a very valuable resource ... But then, I can't afford to really help with this myself so take my thoughts with a big hope pill. =D Yup :) -- Jure Pear --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Funding Cyrus High Availability
On Fri, 17 Sep 2004 [EMAIL PROTECTED] wrote: My biggest question here is, simply, why recreate what's already out there? There are a number of projects (LVM, PVFS) which do this kind of replication/distribution/virtulization for filesystems. There are a number of databases which have active/active clustering (mysql, DB2, Oracle, et al) and master/slave. Personally, I would LOVE to see a full RDBMS-backed system. You define your database(s) in the config file ... and that is all. All configuration options are stored on the central RDBMS. All mailboxes are stored there. You can then rely 100% on the RDBMS systems for clustering/failover/scalability/backing up ... all datastorage domain problems which they have already addressed/solved. SNIP The other advantages would be very nice integration with other applications which can query against databases. (ex: postfix directly supports mysql lookups.) But then, I can't afford to really help with this myself so take my thoughts with a big hope pill. =D Mike, one of the problems with this is that different databases have different interfaces and capabilities. if you design it to work on Oracle then if you try to make it work on MySQL there are going to be quite a few things you need to change. if you start on MySQL and then port to Oracle then you either ignore a large chunk of Oracle functionality that you could use or you end up having to re-write a bunch of stuff to take advantage of it. I also would love this option (I would choose postgres as the back-end) but this is significantly more complicated then a master-slave replication modification to Cyrus. As such it would cost more to get written and you would have fewer people willing to pay for any particular version. another issue in all this is the maintainance of the resulting code. If this code can be used in many different situations then more people will use it (probably including CMU) and it will be maintained as a side effect of any other changes. however if it's tailored towards a very narrow situation then only the people who have that particular problem will use it and it's likly to have issues with new changes. David Lang -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Funding Cyrus High Availability
-Original Message- From: David Lang [mailto:[EMAIL PROTECTED] Sent: Friday, September 17, 2004 2:25 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: RE: Funding Cyrus High Availability Mike, one of the problems with this is that different databases have different interfaces and capabilities. if you design it to work on Oracle then if you try to make it work on MySQL there are going to be quite a few things you need to change. --snip another issue in all this is the maintainance of the resulting code. If this code can be used in many different situations then more people will use it (probably including CMU) and it will be maintained as a side effect of any other changes. however if it's tailored towards a very narrow situation then only the people who have that particular problem will use it and it's likly to have issues with new changes. I'd actually figured something like ODBC would be used, with prepared statements. /shrug. Abstract the whole interface issue. Just some thoughts. =) -Mike/Szii --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
[PATCH] 2.3 %qu vs. %llu on *BSD
FreeBSD 4.X printf() and friends support the %llu format. scanf()
and friends do not. I suspect this is true of most BSD-4.4 derived
systems. I don't know what ANSI says about this issue.
One way to fix it is attached. My autoconf foo is lacking so I
just jammed a #define into configure.in where O_DSYNC is defined
for BSD systems. I doubt this will work in all cases.
John Capo
Index: imap/quota.h
===
RCS file: /cvs/src/cyrus/imap/quota.h,v
retrieving revision 1.1.2.3
diff -u -r1.1.2.3 quota.h
--- imap/quota.h9 Aug 2004 18:51:21 - 1.1.2.3
+++ imap/quota.h18 Sep 2004 00:36:46 -
@@ -60,7 +60,12 @@
typedef long long int quota_t;
#define UQUOTA_T_FMT %llu
#define QUOTA_T_FMT %lld
-#define QUOTA_REPORT_FMT %8llu
+# ifdef HAVE_QUAD_SCAN_FMT
+# define UQUOTA_T_SCAN_FMT %qu
+# else
+# define UQUOTA_T_SCAN_FMT %llu
+# endif
+#define QUOTA_REPORT_FMT %8lu
#else
typedef unsigned long uquota_t;
typedef long quota_t;
Index: imap/quota_db.c
===
RCS file: /cvs/src/cyrus/imap/quota_db.c,v
retrieving revision 1.1.2.3
diff -u -r1.1.2.3 quota_db.c
--- imap/quota_db.c 9 Aug 2004 18:51:21 - 1.1.2.3
+++ imap/quota_db.c 18 Sep 2004 00:36:46 -
@@ -89,7 +89,7 @@
switch (r) {
case CYRUSDB_OK:
- sscanf(data, UQUOTA_T_FMT %d, quota-used, quota-limit);
+ sscanf(data, UQUOTA_T_SCAN_FMT %d, quota-used, quota-limit);
break;
case CYRUSDB_AGAIN:
Index: imap/quota.c
===
RCS file: /cvs/src/cyrus/imap/quota.c,v
retrieving revision 1.48.2.7
diff -u -r1.48.2.7 quota.c
--- imap/quota.c9 Aug 2004 18:51:20 - 1.48.2.7
+++ imap/quota.c18 Sep 2004 00:36:46 -
@@ -251,7 +251,7 @@
}
memset(quota[quota_num], 0, sizeof(struct quotaentry));
quota[quota_num].quota.root = xstrndup(key, keylen);
-sscanf(data, UQUOTA_T_FMT %d,
+sscanf(data, UQUOTA_T_SCAN_FMT %d,
quota[quota_num].quota.used, quota[quota_num].quota.limit);
quota_num++;
Index: configure.in
===
RCS file: /cvs/src/cyrus/configure.in,v
retrieving revision 1.268.2.15
diff -u -r1.268.2.15 configure.in
--- configure.in13 Sep 2004 19:59:19 - 1.268.2.15
+++ configure.in18 Sep 2004 01:14:04 -
@@ -1079,6 +1079,7 @@
#define O_DSYNC O_SYNC /* POSIX */
# else
#define O_DSYNC O_FSYNC /* BSD */
+#define HAVE_QUAD_SCAN_FMT
# endif
#endif
