Re: Funding Cyrus High Availability
David Carter wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to implementing the replication needed for #1 modify all functions that need to update these critical pieces of data to update them on the master and let the master update the other box. We may be talking at cross purposes (and its entirely likely that I've got the wrong end of the stick!), but I consider active-active to be the case where there is no primary: users can make changes to either system, and if the two systems lose touch with each other they have to resolve their differences when contact is reestablished. I'd go for #5 as well: Since this is a setup where there is no primary at all, I suppose this is quite some different design then the #1-4 solutions. And because of that, I would think that it's rather useless to have these steps done in order to get #5 right, but I might as well be wrong. I would be most happy when the work would start on #5. Personally I don't care that much at this moment for #6, but I can imagine that this is different for others. But well; if the design is that every machine tracks changes and they have them propagated (actively or passively) to n hosts (it's not so hard to keep track of that, all hosts had this change; remove it) there is no risk of missing things or not recovering I guess. (It's only possible that a slave is out of sync for a very short time, and well - why would that be so wrong? And if that is so wrong, then maybe fix that later since this would make the work easier?) This could be the task of the cyrus daemon, but it can as well be the work of murder as Jure suggests. (Or both?) I'm not entirely sure that that is what we want, but it could be done if that fits nicely (and it can be asured that there is always a murder to talk to). If there is a problem with UID selection, I don't see a problem in that one of the servers is responsible for that task. We don't even need an election system for that, you could define a sequence for the servers; if a server with the highest preference is down, then take over its job. It's just that for the users the machines should appear all active. (And that in case of failover the remaining machines remain active, and not readonly or only active after manual intervention.) Paul --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Sun, 19 Sep 2004, David Lang wrote: here is the problem. you have a new message created on both servers at the same time. how do you allocate the UID without any possibility of stepping on each other? With a new UIDvalidity you can choose any ordering you like. Of course one of the two servers has to make that choice, and the potential for race conditions here and elsewhere in an active-active solution is amusing. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Problems with cyradm - Solaris 9
Hello All. I've installed the latest version of cyrus and cyrus SASL and everything seems to be working Ok with regard to the server. Imtest allows me to use the plain login mechanism, which is then authenticated against an LDAP directory using saslauthd. I've not yet configured cyrus to use SSL, but that will be done once I'm confident everything is working. The problem I'm having is that when I attempt to run cyradm I *always* get an error of the form below. roche# cyradm localhost imclient_connect: unknown host localhost at /usr/local/lib/perl5/site_perl/5.8.5/sun4-solaris/Cyrus/IMAP/Admin.pm line 71. Having done quite a lot of googleing and trawling through archives I've seen a few people suggest that this is a bug that is limited to Solaris 9, and one patched solution that recommends reverting to parts of a much older cyrus codebase. Just wondering if anybody can clarify what the issue is and what the solution might be as to be honest I'm still inclined to think I'm doing something stupid :) TYIA Nigel Cass Systems Engineer University of Hull --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Sun, 19 Sep 2004, David Lang wrote: assiming that the simplest method would cost ~$3000 to code I would make a wild guess that the ballpark figures would be 1. active/passive without automatic failover $3k 2. active/passive with automatic failover (limited to two nodes or withing a murder cluster) $4k 3. active/passive with updates pushed to the master $5k 4. #3 with auto failover (failover not limited to two nodes or a single murder cluster) $7k 5. active/active (limited to a single geographic location) $10k 6. active/active/active (no limits) $30k in addition to automaticly re-merge things after a split-brin has happened would probably be another $5k I think that you are missing a zero (or at least a fairly substantial multipler!) from 5. 1 - 4 can be done without substantial changes to the Cyrus core code, and Ken would be able to use my code as a reference implementation, even if he wanted to recode everything from scratch. 5 and 6 would require a much more substantial redesign and I suspect quite a lot of trial and error as this is unexplored territory for IMAP servers. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Problems with cyradm - Solaris 9
Hi, in the cyrus-imapd-2.2.8/lib/imclient.c comment out the line 238 /* hints.ai_flags = AI_CANONNAME; */ and rebuild your imapd Nigel Cass wrote: Hello All. I've installed the latest version of cyrus and cyrus SASL and everything seems to be working Ok with regard to the server. Imtest allows me to use the plain login mechanism, which is then authenticated against an LDAP directory using saslauthd. I've not yet configured cyrus to use SSL, but that will be done once I'm confident everything is working. The problem I'm having is that when I attempt to run cyradm I *always* get an error of the form below. roche# cyradm localhost imclient_connect: unknown host localhost at /usr/local/lib/perl5/site_perl/5.8.5/sun4-solaris/Cyrus/IMAP/Admin.pm line 71. Having done quite a lot of googleing and trawling through archives I've seen a few people suggest that this is a bug that is limited to Solaris 9, and one patched solution that recommends reverting to parts of a much older cyrus codebase. Just wondering if anybody can clarify what the issue is and what the solution might be as to be honest I'm still inclined to think I'm doing something stupid :) TYIA Nigel Cass Systems Engineer University of Hull --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: raw access to imap quotas, with mail user
On Sun, Sep 19, 2004 at 09:26:42PM -0700, Carl P. Corliss wrote: Better yet, only update it when you absolute need to (meaning: only when you are checking mail or making a change to your mailbox by deleting, moving or renaming). That should work - of course providing your web portal is functioning as a mail client (checking mail/etc) and not -only- interacting with imap to retrieve the quota. You are talking about some kind of triggers... but that's difficult because i don't have source code of the Webmail imap program [my boss bought that a few years ago]. regards, Félix -- Felix Cuello [EMAIL PROTECTED] - 1512 - Always do right. This will gratify some people and astonish the rest. -- Mark Twain --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyradm setinfo shutdown
Hi, Could anyone answer my question please. We recently had to take our mailserver offline for routine maintenance, and we decided to use the cyradm setinfo shutdown facility to facilitate this. From the documentation it would appear that this should stop any new users connecting by presenting them with a message. However, in practise it prevented pine from seeing any mailboxes, and killed evolution stone dead. Not once did I see any sight of the error message I had set. The only time I saw the message was I telnetted to port 143 and tried logging in manually. Is this normal behaviour or did I set something up incorrectly? Also, is there a command within cyradm to unset the information set with the setinfo command. I could not find one, and when it was time to bring the mailserver back up I could not remove the shutdown message, I had to root about manually in the file structure until I found the relevant file to delete. Is there an easy way to do this? Any help much appreciated. Gareth Ansell --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [PATCH] 2.3 %qu vs. %llu on *BSD
Linux printf page: q (quad. BSD 4.4 and Linux libc5 only. Dont use.) This is a synonym for ll. ... and further down under 'Conforming to'... Linux libc5 knows about the five C standard flags and the flag, locale, %m$ and *m$. It knows about the length modifiers h,l,L,Z,q, but accepts L and q both for long doubles and for long long integers (this is a bug). It no longer recognizes FDOU, but adds a new conversion character m, which outputs strerror(errno). and finally the linux scanf: q equivalent to L. This flag does not exist in ANSI C. ...and there you have it. I was curious, so i thought i'd post my findings for linux boxen (FC2). -Brenden On Fri, 2004-09-17 at 21:30, John Capo wrote: FreeBSD 4.X printf() and friends support the %llu format. scanf() and friends do not. I suspect this is true of most BSD-4.4 derived systems. I don't know what ANSI says about this issue. One way to fix it is attached. My autoconf foo is lacking so I just jammed a #define into configure.in where O_DSYNC is defined for BSD systems. I doubt this will work in all cases. John Capo -- Brenden Conte System Programmer, CCT Rensselaer Polytechnic Institute (518)276-2540 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
[SOLVED] raw access to imap quotas, with mail user
On Sun, Sep 19, 2004 at 09:21:29PM -0300, Felix Cuello wrote: [...] As Derrick said, I'm executing the program with setuid and that's working. I don't know if there is another way to do that. The PERL script just control malicious user string and user string size and C program open /var/imap/quota and prints mailbox usage (in percentage). Thanks everyone for each help, Félix -- Felix Cuello [EMAIL PROTECTED] - 1513 - Q: What do they call the alphabet in Arkansas? A: The impossible dream. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Mon, 20 Sep 2004, David Carter wrote: On Sun, 19 Sep 2004, David Lang wrote: assiming that the simplest method would cost ~$3000 to code I would make a wild guess that the ballpark figures would be 1. active/passive without automatic failover $3k 2. active/passive with automatic failover (limited to two nodes or withing a murder cluster) $4k 3. active/passive with updates pushed to the master $5k 4. #3 with auto failover (failover not limited to two nodes or a single murder cluster) $7k 5. active/active (limited to a single geographic location) $10k 6. active/active/active (no limits) $30k in addition to automaticly re-merge things after a split-brin has happened would probably be another $5k I think that you are missing a zero (or at least a fairly substantial multipler!) from 5. 1 - 4 can be done without substantial changes to the Cyrus core code, and Ken would be able to use my code as a reference implementation, even if he wanted to recode everything from scratch. 5 and 6 would require a much more substantial redesign and I suspect quite a lot of trial and error as this is unexplored territory for IMAP servers. Thanks, this is exactly the type of feedback that I was hopeing to get. so you are saying that #5 is more like $50k-100k and #6 goes up from there Ok folks, how much are you really willing to pay for this and since the amount of work involved translates fairly directly into both cost and time how long are you willing to go with nothing? David Lang -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Mon, 20 Sep 2004, Paul Dekkers wrote: David Carter wrote: 5. Active/Active designate one of the boxes as primary and identify all items in the datastore that absolutly must not be subject to race conditions between the two boxes (message UUID for example). In addition to implementing the replication needed for #1 modify all functions that need to update these critical pieces of data to update them on the master and let the master update the other box. We may be talking at cross purposes (and its entirely likely that I've got the wrong end of the stick!), but I consider active-active to be the case where there is no primary: users can make changes to either system, and if the two systems lose touch with each other they have to resolve their differences when contact is reestablished. I'd go for #5 as well: Since this is a setup where there is no primary at all, I suppose this is quite some different design then the #1-4 solutions. And because of that, I would think that it's rather useless to have these steps done in order to get #5 right, but I might as well be wrong. actually I think most of the work nessasary for #1 is also needed for #5-6. for #1 you need to have the ability for a system report all it's changes to a daemon and the ability for a system to read in changes and implement them. #5 needs the same abilities plus the ability to resolve conflicts. the HA steps of #2 and #4 don't gain that much, but they can also be done external to cyrus so it's not a problem to skip them. #3 involves changes to the update code to have cyrus take special actions with soem types of updates. there would need to be changes in the same area for #5, but they would be different. David Lang -- There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Funding Cyrus High Availability
On Mon, 20 Sep 2004, David Lang wrote: Thanks, this is exactly the type of feedback that I was hopeing to get. so you are saying that #5 is more like $50k-100k and #6 goes up from there If anyone could implement Active-Active for Cyrus from scratch in 100 to 150 hours it would be Ken, but I think that its a tall order. Sorry. -- David Carter Email: [EMAIL PROTECTED] University Computing Service,Phone: (01223) 334502 New Museums Site, Pembroke Street, Fax: (01223) 334679 Cambridge UK. CB2 3QH. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Sieve vacation/redirection and SMTP authentication
Hello I'm setting up sieve, but our setup will probably make things a bit more difficult: Our SMTP servers (Postfix) only allow authenticated users to send messages, and checks if the sender login matches with the mail from line of the SMTP protocol. The SMTP servers and IMAP/POP3 servers are not installed on the same machines. This setup, as is, would make it impossible for sieve to send redirection or vacation messages, as there would be no way for it to do the authentication, so some kind of modification would be necessary to make it work. I could think of two solutions: - Run an extra postifx instance that allows unauthenticated messages from the IMAP/POP servers - Run Postfix's qmqpd. This would be nice to avoid the overhead of more smtp connections, but I couldn't really find any qmqp client and would end up having to write some script with the Net::QMQP perl module. So, finally, my qustion :) Is anyone running on a similar setup that only allows for authenticated SMTP? Maybe you can suggest me some simpler workaround for redirection and vacation to work (or are the solutions listed above the only ones I'd have available?) Thanks in advance, Andre --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
the amazing disappearing quota
I'm running an older (2.2.6) version of cyrus imapd, and i keep running into the situation where a users quota disappears. Is this a known (and/or fixed) bug? Or am i stumbling onto something stranger? The only thing i can say is that there are no errors in the logs, (except LOSTQUOTA entries when i delete an account) and the quotaroot files for the quotaless users are completely msising. Thanks, -- Brenden Conte System Programmer, CCT Rensselaer Polytechnic Institute (518)276-2540 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: the amazing disappearing quota
Nevermind. I little deeper digging in the list revealed the issue. Regards, -Brenden On Mon, 2004-09-20 at 15:50, Brenden Conte wrote: I'm running an older (2.2.6) version of cyrus imapd, and i keep running into the situation where a users quota disappears. Is this a known (and/or fixed) bug? Or am i stumbling onto something stranger? The only thing i can say is that there are no errors in the logs, (except LOSTQUOTA entries when i delete an account) and the quotaroot files for the quotaless users are completely msising. Thanks, -- Brenden Conte System Programmer, CCT Rensselaer Polytechnic Institute (518)276-2540 --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Virtual domains broken?
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi, all.
When setting up imapd, I've become accustomed to setting the imapd.conf
servername option to the name of the SASL realm that I am serving for,
regardless of the server's actual hostname. For example, if I'm running
imapd on machine.x.com and my realm is x.com, I would configure
servername to match the latter and live with it.
However, in the process of performing a fresh 2.2.8 install, I got a bug
in my ear to experiment a bit. My environment consists of a FreeBSD
5.2.1-RELEASE-p9 machine with one NIC (192.168.0.1) configured with
multiple IP aliases (192.168.0.2, 192.168.0.3, etc.).
Based on my understanding of the documentation, if I were to enable
virtual domains, imapd would report a different FQDN for each
*interface* connection (e.g., mail.x.com for 192.168.0.2 or mail2.x.com
for 192.168.0.3), regardless of the actual *hostname* of that machine
(machine.x.com/192.168.0.1) and have unqualified usernames use a default
domain that is formed by truncating the FQDN (e.g., to x.com).
I've configured servername, virtdomains, and defaultdomain in the
following combinations:
servername: mail.x.com
virtdomains: yes
defaultdomain: x.com
servername: mail.x.com
virtdomains: yes
defaultdomain: x.com
servername: mail.x.com
defaultdomain: x.com
I've tried running imtest with the following options for each of the
above combinations. Neither have worked unless servername matches the
SASL realm and the other two options are unset.
[EMAIL PROTECTED]:~ imtest -m login -a acc mail.x.com
[EMAIL PROTECTED]:~ imtest -m login -a acc -r x.com mail.x.com
Also worth mentioning is the fact that I refrained from using cyradm and
instead created the user with saslpasswd2. Could this be my problem?
It would seem that either I am doing something wrong, imapd doesn't
support interface aliases, or virtual domain support is broken. Before
submitting a bug report, however, I'd like to know that I'm not just
missing something.
Here is a diff, showing my edits on the imapd.conf shipped with the
FreeBSD port, followed by my edits on cyrus.conf and non-default
settings found in the shipped imapd.conf. Not much has been changed.
[EMAIL PROTECTED]:~ diff /usr/ports/mail/cyrus-imapd22/files/imapd.conf
/usr/local/etc/imapd.conf
49c49
#servername: result returned by gethostname(2)
- ---
#servername: mail.x.com
236c236
#sendmail: /usr/sbin/sendmail
- ---
sendmail: /usr/local/sbin/sendmail
320c320
#sasl_mech_list: cram-md5 digest-md5
- ---
sasl_mech_list: digest-md5 cram-md5 plain
412a413,415
virtdomains: yes
defaultdomain: x.com
[EMAIL PROTECTED]:~ diff
/usr/ports/mail/cyrus-imapd22/work/cyrus-imapd-2.2.8/master/conf/normal.conf
/usr/local/etc/cyrus.conf
14,18c14,18
imapcmd=imapd listen=imap prefork=0
imaps cmd=imapd -s listen=imaps prefork=0
pop3cmd=pop3d listen=pop3 prefork=0
pop3s cmd=pop3d -s listen=pop3s prefork=0
sieve cmd=timsieved listen=sieve prefork=0
- ---
imapcmd=imapd listen=mail.x.com:imap prefork=0
# imaps cmd=imapd -s listen=mail.x.com:imaps prefork=0
# pop3 cmd=pop3d listen=pop3 prefork=0
# pop3s cmd=pop3d -s listen=pop3s prefork=0
# sieve cmd=timsieved listen=sieve prefork=0
[EMAIL PROTECTED]:~ grep '^[^#]' /usr/ports/mail/cyrus-imapd22/files/imapd.conf
configdirectory: /var/imap
partition-default: /var/spool/imap
sieveusehomedir: false
sievedir: /var/imap/sieve
sasl_pwcheck_method: auxprop
[EMAIL PROTECTED]:~
And here is the output from a few commands under this configuration.
Note that although the user exists and the interface reverse-resolves to
a name other than the hostname of the machine itself, an imtest session
reveals the machine hostname rather than the interface name.
[EMAIL PROTECTED]:~ sudo sasldblistusers2
Password:
[EMAIL PROTECTED]: userPassword
[EMAIL PROTECTED]:~ host 192.168.0.2
2.0.168.192.IN-ADDR.ARPA domain name pointer mail.x.com
[EMAIL PROTECTED]:~ imtest -m login mail.x.com
S: * OK machine.x.com Cyrus IMAP4 v2.2.8 server ready
C: C01 CAPABILITY
S: * CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS NAMESPACE UIDPLUS
ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT THREAD=ORDEREDSUBJECT
THREAD=REFERENCES ANNOTATEMORE IDLE AUTH=DIGEST-MD5 AUTH=CRAM-MD5 SASL-IR
S: C01 OK Completed
Please enter your password:
C: L01 LOGIN acc {3}
S: + go ahead
C: omitted
S: L01 NO Login failed: user not found
Authentication failed. generic failure
Security strength factor: 0
. LOGOUT
* BYE LOGOUT received
. OK Completed
Connection closed.
[EMAIL PROTECTED]:~
- --
Anthony Chavez http://www.anthonychavez.org/
mailto:[EMAIL PROTECTED] jabber:[EMAIL PROTECTED]
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.4 (Darwin)
iD8DBQFBT5H2bZTbIaRBRXERApnEAJ9/4Uf2eYOjw+1YiXzaz43L2ZX5NQCfRS68
