Re: Announcing cyrus-user-map, a local recipient map generator for Postfix
Thanks to you and Eric Luyten for the information on the small overhead. So until someone ports smmapd to Postfix I agree that the recipient address validation setting is sufficient. Regards, Le 10.03.2009 11:50, Stefan Schmidt a écrit : > > Postfix cashes to test result for a configurable time spam, with > separately definable ranges for positive and negative test results. > It is - after an initial learning period - basically a hash table lookup > without the hazzles of manually generating the lookup table. > -- Farzad FARID / Architecte Open Source - Associé Pragmatic Source / http://www.pragmatic-source.com Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Announcing cyrus-user-map, a local recipient map generator for Postfix
Hi, Le 10.03.2009 02:56, Reinaldo de Carvalho a écrit : So I wrote a little tool that generates and update this map automatically, with the help of cron. You can download the script and documentation from my website http://www.pragmatic-source.com/en/opensource/source-code/cyrus-user-map-generate-cyrus-user-map-postfix or from GitHub: http://github.com/Farzy/cyrus-user-map Its not necessary use ruby (or another language) merely to call shell commands, do it on pure (ba|c|k)sh. You're right! But I chose a somewhat cleaner and higher level language to have a more readable script. You may rewrite it in *sh if you're interested and I'll include it in my package or add a link to it. It's also a proactive choice for the reason that Ruby is gaining in popularity, is available on many server platforms (even Sun promotes Ruby) and I'm pushing the use of Ruby for agile sysadmin tasks :) This is off-topic but interested *french speaking* people can view my lightning talk in this subjet: http://tinyurl.com/avuebz Best regards, -- Farzad FARID / Architecte Open Source - Associé Pragmatic Source / http://www.pragmatic-source.com Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Announcing cyrus-user-map, a local recipient map generator for Postfix
Le 10.03.2009 09:59, Stefan Schmidt a écrit : Nice thing, but just out of curiosity, why don't you use postfix' recipient verification mechanism? In smtp_recipient_restrictions add reject_unverified_recipient at a reasonable position in these restrictions. Everything else automagically happens. Thanks for asking. In the "Postfix Address Verification Howto" (http://www.postfix.org/ADDRESS_VERIFICATION_README.html) it says that: "/A Postfix MTA verifies a sender or recipient address by probing the nearest MTA for that address, without actually delivering mail. The nearest MTA could be the Postfix MTA itself, or it could be a remote MTA (SMTP interruptus). Probe messages are like normal mail, except that they are never delivered, deferred or bounced; probe messages are always discarded./" This is a very heavyweight process, especially if we have the Cyrus database at hand :) So I think that looking up an entry in a hash map is much faster and doesn't involve any SMTP/LMTP connection. And if the Postfix relay and the Cyrus database are not on the same physical computer, and someone's interested, I can extend my little tool to use a network connection to fetch the user database. Best regards -- Farzad FARID / Architecte Open Source - Associé Pragmatic Source / http://www.pragmatic-source.com Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Announcing cyrus-user-map, a local recipient map generator for Postfix
Hi, Having just deployed a Cyrus Imap server with Postfix, with only virtual users and no Unix or LDAP based imap accounts, I needed a way to generate a local recipient map for Postfix. This map is used by Postfix to identify existing email accounts and correctly reject incoming mail received for unknown recipients (as in http://www.postfix.org/LOCAL_RECIPIENT_README.html). So I wrote a little tool that generates and update this map automatically, with the help of cron. You can download the script and documentation from my website http://www.pragmatic-source.com/en/opensource/source-code/cyrus-user-map-generate-cyrus-user-map-postfix or from GitHub: http://github.com/Farzy/cyrus-user-map Regards -- Farzad FARID / Architecte Open Source - Associé Pragmatic Source / http://www.pragmatic-source.com Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: How to copy shared folders from one cyrus to another?
Hi, on 09.02.2007 15:21 Tuomas Toropainen said the following: > Hello > > I have succesfully copied users' mail folders with imapsync. But How > can I copy shared folders? The problem seems to be that imapsync > always adds "INBOX." to the destination folder name, like this: > > From [shared.vitsit] Parse 1 > To [INBOX.shared.vitsit] Parse 1 > Verifying [shared.vitsit] -> [INBOX.shared.vitsit] > > I have tried lots of different options, like this: > > imapsync --host1 source.imap.server --ssl1 --authuser1 cyrus --user1 > testi2 --authmech1 PLAIN --passfile1 cyrus-salasana.txt --host2 > localhost --ssl2 --authuser2 cyrus --user2 jorma --authmech2 PLAIN > --passfile2 cyrus-salasana.txt --syncinternaldates --folder > 'shared.vitsit' --prefix1 'INBOX.' > > Removing "--prefix1 'INBOX.'" makes no difference. > The following sample works perfectly for me. I even used a more complicated configuration: * Shared prefix on server 1 is "commun." (means "common" in french) * Shared prefix on server 2 is "shared." I copied "commun.test" to "shared.test". "shared.test" did not exist on the destination and was created automatically. imapsync --host1 SERVER1 --ssl1 --authmech1 LOGIN --user1 USER1 --host2 SERVER2 --ssl2 --authmech2 LOGIN --user2 USER2 --prefix1 "commun." --prefix2 "shared." --folder "commun.test" So the trick is to provide both source and destination shared prefixes, and give the complete source folder path. My imapsync version is *1.182*. Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Associé Pragmatic Source - http://www.pragmatic-source.com/ Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Working unified murder configs?
t;
notify cmd="notifyd"
listen="/var/run/cyrus/socket/notify" proto="udp" prefork=1
# --- Murder frontends -
# mupdate database service - must prefork at least 1
# (mupdate slaves)
mupdate cmd="mupdate" listen=3905 prefork=1 proto="tcp4"
}
EVENTS {
checkpoint cmd="/usr/sbin/ctl_cyrusdb -c" period=30
delprunecmd="/usr/sbin/cyr_expire -E 3 -X 3" at=0201
tlsprunecmd="/usr/sbin/tls_prune" at=0201
squatter_a cmd="/usr/sbin/squatter" at=0317
}
File /etc/imapd.conf:
As I said before, this is an incomplete example.
configdirectory: /var/lib/cyrus
defaultpartition: default
partition-default: /var/spool/cyrus/mail
admins: cyrus
lmtp_admins: cyrus postman
postuser: cyrus
allowanonymouslogin: no
allowplaintext: yes
allowapop: no
sasl_minimum_layer: 0
loginrealms: mydomain.com
sasl_pwcheck_method: auxprop saslauthd
sasl_auxprop_plugin: sasldb
sasl_auto_transition: no
mupdate_server: murder1.mydomain.com
mupdate_config: unified
mupdate_port: 3905
mupdate_authname: imap1
mupdate_realm: mydomain.com
mupdate_password: _HERE_IMAP1_PASSWORD
mupdate_worker_maxspare: 5
mupdate_worker_minspare: 1
mupdate_worker_start: 2
proxy_password: _HERE_CYRUS_PASSWORD
proxy_authname: cyrus
proxyservers: cyrus
Bootstrapping the whole thing
Actually I don't remember exactly how I bootstrapped the system :P
These steps are approximative:
* Monitor the syslog files on all servers to detect errors or
warnings. In my environment everything is logged in /var/log/mail.log.
* Start the cyrus services on the "murder1" server. Check that the
following services are running:
o cyrmaster -d
o mupdate -m
* Go on "imap1" (resp. "imap2")
* Using "cyradm" create some local users:
o cyradm -u cyrus localhost
o cyradm> createmailbox user.johndoe
o cyradm> etc.
* Launch "sudo -u cyrus ctl_mboxlist -m" manually. Check the
messages and the log file to see if everything went fine.
* Repeat previous operations (user creation & ctl_mboxlist -m) on
server "imap2"
Now run "sudo -u cyrus ctl_mboxlist -d" on all 3 servers and compare the
results. At least "murder1" should have a complete list of all folders.
If everything worked fine:
* Start the whole cyrus services on both imap servers
o /etc/init.d/cyrus start
* Check the log files
* Enjoy!
Now "sudo -u cyrus ctl_mboxlist -d" should always return a complete list
of all folders, regardless of the server it is run on. I you open an
IMAP connection with "imap1" or "imap2" you should be able to access any
folder (depending on your authorizations of course).
If there's anything wrong or incomplete in my mail, please tell me and
I'll try to correct it.
Regards,
--
Farzad FARID <[EMAIL PROTECTED]>
Architecte Open Source / Associé
Pragmatic Source - http://www.pragmatic-source.com/
Tel : +33 9 53 19 21 90 / Mob : +33 6 03 70 65 46
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Patches used at FastMail.FM
Hi, on 08.01.2007 17:23 Ken Murchison said the following: > > Jeff and I have already discussed pushing out a 2.3 release soon -- as > soon as we iron out all of the wrinkles in our 2.3 deployment on campus. > > We found a few small buglets in the IMAP proxy code that we didn't > expect. Either nobody else has a 2.3 Murder running, or they didn't > notice the random (infrequent) core dumps. > I setup a unified muder with Cyrus IMAP 2.3.7 for a customer with 500 accounts and 1 mailboxes. It's still in testing, all the accounts have not been migrated yet, but I haven't notice any annoying or blocking coredump. I too wonder who runs Cyrus IMAP 2.3 Murder, because hardly anybody ever answers my questions here and I have to reply to myself ;) Regards, -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Can cyradm list only local folder in a murder config?
Hi, In my cyrus 2.3 murder setup I have 2 imap servers. If I connect with 'cyradm' on one of the imap servers and run "lm user.%" I get a list of all the mailboxes on all servers. I'd like to get the list of only local folders, is it possible? Actually I'm even using a python cyrus library, so the solution has to work in the python script too. Regards, -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Does the quota include deleted but not yet expunged mails in v2.3 with delayed expunge?
Hi, I'm running Cyrus Imapd 2.3.7 with the delayed expunge mode. Do the messages deleted by the user, but not yet expunged by the system, count in the user's quota? I'd say yes but I'd like a confirmation. Regards, -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Problem with mupdate/sasl and random entropy [Re: Setting multiples acls in cyrus/murder slows down to a crawl]
Hi, So, once again I'll anwser my own question ;) The cyradm session slows down suddenly because the numerous sasl authentications the imap server does when connecting to the mupdate server completely drain the /dev/random entropy generator... Because I don't want to recompile cyrus-sasl (and the hardware random number generator on my Dell servers doesn't seem to work), I made a symlink from /dev/urandom to /dev/random and it solved the problem. Yet I have 2 other questions: Why isn't the RNG used by cyrus-sasl configurable at runtime, but only at compile-time? OpenLDAP has a config option called "TLSRandFile" for this. Why does the imap server (on which I'm connected with cyradm) establish a new connection to mupdate for *every single call* to "setaclmailbox user. cyrus kxa"? Because I change the ACL of 400 mailboxes, imap must reauthenticate 400 times in a row. Regards Farzad FARID wrote: Hi, I write a script for cyradm to set the ACLs for all the users, like this: setaclmailbox user.perez cyrus kxa setaclmailbox user.pirat cyrus kxa setaclmailbox user.plouvier cyrus kxa setaclmailbox user.pruche cyrus kxa setaclmailbox user.seltani cyrus kxa setaclmailbox user.serre cyrus kxa setaclmailbox user.solers cyrus kxa ...[400 accounts]... I then feed the script to cyradm. But after 10 lines, the execution suddenly slows down to a crawl, and only one "setaclmailbox" every 10 or 20 seconds is executed. I'm running a unified cyrus murder 2.3.7, with 2 imap servers and 1 mupdate server. I ran the script only on one imap server, and all the accounts belong to this server. These are the logs I see on the mupdate server. The imap server keeps connecting/disconnecting from the mupdate server, and the mupdate server seems to spend a lot of time in either cmd_find or cmd_set. Oct 13 12:19:04 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:04 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.fboudali) Oct 13 12:19:04 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.fboudali) Oct 13 12:19:05 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:05 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:05 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:05 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:06 oban cyrus/mupdate[23028]: cmd_find(fd:14, user.fboudali) Oct 13 12:19:23 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:23 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.bpincede) Oct 13 12:19:23 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.bpincede) Oct 13 12:19:24 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:24 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:24 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:24 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:26 oban cyrus/mupdate[23028]: cmd_find(fd:14, user.bpincede) Oct 13 12:19:29 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:29 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.albonnefoy) Oct 13 12:19:29 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.albonnefoy) Oct 13 12:19:29 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:29 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:29 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:29 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:29 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in When I try to trace the mupdate threads, all I see is that the threads seem to spend a lot a time waiting on locks (futex on the linux implementation of threads), and very little time reading or writing any file. Can anybody explain this to me and help me debug it? Or is there another way to set hundreds of ACLs in a row? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Setting multiples acls in cyrus/murder slows down to a crawl
Hi, I write a script for cyradm to set the ACLs for all the users, like this: setaclmailbox user.perez cyrus kxa setaclmailbox user.pirat cyrus kxa setaclmailbox user.plouvier cyrus kxa setaclmailbox user.pruche cyrus kxa setaclmailbox user.seltani cyrus kxa setaclmailbox user.serre cyrus kxa setaclmailbox user.solers cyrus kxa ...[400 accounts]... I then feed the script to cyradm. But after 10 lines, the execution suddenly slows down to a crawl, and only one "setaclmailbox" every 10 or 20 seconds is executed. I'm running a unified cyrus murder 2.3.7, with 2 imap servers and 1 mupdate server. I ran the script only on one imap server, and all the accounts belong to this server. These are the logs I see on the mupdate server. The imap server keeps connecting/disconnecting from the mupdate server, and the mupdate server seems to spend a lot of time in either cmd_find or cmd_set. Oct 13 12:19:04 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:04 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.fboudali) Oct 13 12:19:04 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.fboudali) Oct 13 12:19:05 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:05 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:05 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:05 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:06 oban cyrus/mupdate[23028]: cmd_find(fd:14, user.fboudali) Oct 13 12:19:23 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:23 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.bpincede) Oct 13 12:19:23 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.bpincede) Oct 13 12:19:24 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:24 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:24 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:24 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:26 oban cyrus/mupdate[23028]: cmd_find(fd:14, user.bpincede) Oct 13 12:19:29 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in Oct 13 12:19:29 oban cyrus/mupdate[23028]: cmd_set(fd:15, user.albonnefoy) Oct 13 12:19:29 oban cyrus/mupdate[23028]: cmd_find(fd:17, user.albonnefoy) Oct 13 12:19:29 oban cyrus/mupdate[23028]: accepted connection Oct 13 12:19:29 oban cyrus/mupdate[23028]: telling master 4 Oct 13 12:19:29 oban cyrus/master[22804]: service mupdate pid 23028 in READY state: serving one more multi-threaded connection Oct 13 12:19:29 oban cyrus/master[22804]: service mupdate now has 1 ready workers Oct 13 12:19:29 oban cyrus/mupdate[23028]: login: aberlour.srv.in.karavel.com [10.12.17.44] aberlour DIGEST-MD5 User logged in When I try to trace the mupdate threads, all I see is that the threads seem to spend a lot a time waiting on locks (futex on the linux implementation of threads), and very little time reading or writing any file. Can anybody explain this to me and help me debug it? Or is there another way to set hundreds of ACLs in a row? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: lmtp should give temporary failure for mailbox unknown
Hi, Daniel Eckl wrote: > Hmmm, well, a non existent mailbox is no temprary failure, it's very > permanent. okay, in this case it's a false error, but to make another > error to circumvent that is not a smooth solution. > > Perhaps you might want to set "soft_bounce = yes" in postfix' main.cf > until you have found the error. > See: http://www.postfix.org/postconf.5.html#soft_bounce > But in fact there are cases in a Cyrus Murder configuration when lmtpd returns a failure with a 5xx code because "mupdate" is simply not ready or temporarily unavailable. So simply setting soft bounces in Postfix is IMHO overkill and hides a "misfeature" in the Cyrus Imapd code. Therefore we should concentrate on finding the real cause(s) of the problem and try to solve them. I'll try to file a bug report by describing a reproducible test case for my setup. Ram, do you also use mupdate or any external source of information that could be the cause of the error lmtp gives you? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Is cyr_expire trying cross-server expiration in a Murder configuration?
Hi, Here is a patch for this problem, attached to my bug report #2884: https://bugzilla.andrew.cmu.edu/attachment.cgi?id=465&action=view. Regards Farzad FARID wrote: > Hi, > > I've set up a cyrus+murder configuratio in unified mode (back+front on > the same servers) with Cyrus 2.3.7. I have 2 imap servers (aberlour and > chivas), and one mupdate server. > > This morning I noticed lots of errors in the log file of "aberlour", > regarding cyr_expire: > > Sep 14 02:01:00 aberlour cyrus/master[24472]: about to exec > /usr/sbin/tls_prune > Sep 14 02:01:00 aberlour cyrus/master[24473]: about to exec > /usr/sbin/cyr_expire > Sep 14 02:01:00 aberlour cyrus/tls_prune[24472]: tls_prune: purged 0 out > of 0 entries > Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: IOERROR: opening > chivas!default/cyrus.header: No such file or directory > Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: unable to open/lock > mailbox commun.test > Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: IOERROR: opening > chivas!default/cyrus.header: No such file or directory > Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: unable to open/lock > mailbox user.aamili > [...] > > And vice-versa "chivas" has the same error messages mentionning > "aberlour!default/cyrus.header". It seems to happen every niht ever > since I installed murder. > > Why does this happen? It that normal or did I miss some important > configuration issue? > > Regards > > -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Squatter not working in murder/unified environment?
Hi, I finally investigated the problem myself and provided a patch. Here is the bug report: https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2881 The patch is small, 6 lines, and works in my test environment (32.000 mailboxes across 2 servers): https://bugzilla.andrew.cmu.edu/attachment.cgi?id=461&action=""> Regards Farzad FARID wrote: Hi, Environment : - Cyrus 2.3.7 - Unified Murder - 2 IMAP, 1 mupdate I have mailboxes on both imap servers (aberlour & chivas) and a shared mailbox "commun.test" on chivas. When I run "/usr/sbin/squatter -v" on aberlour, it stops on an error saying it can find a mailbox or partition located on chivas! On the command line I get: aberlour$ /usr/sbin/squatter -v error opening commun.test: System I/O error And in syslog: Sep 20 11:28:26 aberlour cyrus/squatter[5803]: indexing mailboxes Sep 20 11:28:26 aberlour cyrus/squatter[5803]: IOERROR: opening chivas!default/cyrus.header: No such file or directory Sep 20 11:28:26 aberlour cyrus/squatter[5803]: done indexing mailboxes If I run "/usr/sbin/squatter -v -r 'user.*'", it starts indexing the first mailboxes, located on aberlour, but as soon as it encounters a mailbox located on chivas it stops with an error: aberlour$ /usr/sbin/squatter -v -r 'user.*' error opening user.*: Mailbox does not exist Indexing mailbox user.aaraujo.DOSSIERS CLIENTS... Indexed 0 messages (0 bytes) into 112 index bytes in 0 seconds Indexing mailbox user.aaraujo.DOSSIERS CLIENTS.02- CLE DES CHAMPS... Indexed 1 messages (5505 bytes) into 17132 index bytes in 0 seconds [...] Indexing mailbox user.abaeza.tunisie... Indexed 0 messages (0 bytes) into 112 index bytes in 0 seconds Indexing mailbox user.abaeza.yield... Indexed 1520 messages (9372693 bytes) into 4953647 index bytes in 2 seconds error opening user.abbar.&AMk-KIP: System I/O error Total over all mailboxes: Indexed 8100 messages (51105784 bytes) into 26944157 index bytes in 11 seconds How can I run squatter correctly? Is this a bug or a configuration error on my side? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: admin authentication
Marten Lehmann wrote: Hello, we will use LDAP through saslauthd to authenticate our users. Is there a way to authenticate admin-users a different way at the same time? Best would be to hardcode a md5-password within the imapd.conf or to use /etc/passwd for that. But I don't want to pass everything through PAM just to authenticate the admin user. Hi, You can use saslauthd/LDAP for your users and a local sasldb2 file for your admins. You don't need PAM at all. I use both LDAP and sasldb with these config lines: sasl_pwcheck_method: auxprop saslauthd sasl_auxprop_plugin: sasldb The following commands, sasldblistusers2 & saslpasswd2, can be used to administer the /etc/sasldb2 file. With this configuration you will have the added benefit of digest-md5 or cram-md5 authentication for your admins (but not for LDAP users AFAIK, unless using cleartext password in the LDAP directory, which I wouldn't do). Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Loss of mail with Murder+LMTP (imapd 2.3.7): incoming mail rejected just after server restart
Hi, I corrected on problem on the system that was not Cyrus Imapd related, a bug the linux kernel 2.6.17 slowed down ext3 about 15 times when checking the mailboxes.db file on startup... Nevertheless! The mupdate slave in my unified murder configuration still take about 17 secondes before being operationnel. As imapd, lmtpd and mupdate are all started simultaneously, there still is a 17 second time frame on boot up during which all incoming mail is lost because lmtpd cannot reach mupdate... Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: connection from galia.srv.in.karavel.com [10.12.17.31] Sep 14 11:50:28 imap1 cyrus/master[26548]: about to exec /usr/lib/cyrus/bin/lmtpd Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: login: galia.srv.in.karavel.com [10.12.17.31] postman DIGEST-MD5 User logged in Sep 14 11:50:28 imap1 cyrus/lmtp[26548]: executed Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: kick_mupdate: can't connect to target: Connection refused Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: verify_user(user.user2) failed: Mailbox does not exist Sep 14 11:53:09 imap1 cyrus/master[26353]: process 26411 exited, status 0 Does anybody have an explanation or solution for this? Regards Farzad FARID wrote: Hi, In my unified murder configuration (cyrus imapd 2.3.7) with 2 imap servers and 1 mupdate server, I have a unwanted behaviour when LMTP tries to deliver mail for a mailbox that is not available. Lets says I have a user, "user2", whoses mailboxes are on "imap2". Scenario 1 : == - mupdate UP - imap1 UP - imap2 UP When mupdate, imap1 and imap2 are running, everything goes well. Even when I deliver mail via LMTP to "imap1", who doesn't own "users2"'s mailboxes. "imap1" then connects to "imap2" and correctly delivers the mail. ==> OK Scenario 2 : === - mupdate UP - imap1 DOWN, then UP for less than 10 minutes - imap2 DOWN Now I shut down both "imap1""imap2", and restart ONLY imap1. The mupdate server is always running and has never been shut down. Again, I send an email for "user2" via LMTP to "imap1", the mailbox being hosted on "imap2". Instead of returning a soft error (4XX) as I would have expected, imap1 returns a hard error (5XX) to the SMTP/LMTP server, and the mail is lost! ==> Incoming Mail lost Here are log transcripts: - On the LMTP server (Postfix 2.0.x): Sep 14 11:50:29 galia postfix/lmtp[24424]: 80E8473F35: to=<[EMAIL PROTECTED]>, relay=imap1[10.12.17.44], delay=1154, status=bounced (host imap1[10.12.17.44] said: 550-Mailbox unknown. Either there is no mailbox associated with this 550-name or you do not have authorization to see it. 550 5.1.1 User unknown (in reply to RCPT TO command)) - On "imap1": Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: connection from galia.srv.in.karavel.com [10.12.17.31] Sep 14 11:50:28 imap1 cyrus/master[26548]: about to exec /usr/lib/cyrus/bin/lmtpd Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: login: galia.srv.in.karavel.com [10.12.17.31] postman DIGEST-MD5 User logged in Sep 14 11:50:28 imap1 cyrus/lmtp[26548]: executed Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: kick_mupdate: can't connect to target: Connection refused Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: verify_user(user.user2) failed: Mailbox does not exist Sep 14 11:53:09 imap1 cyrus/master[26353]: process 26411 exited, status 0 Scenario 3 : == - mupdate UP - imap1 UP for more than 10 minutes - imap2 DOWN This time, if I wait *a few minutes* before resending a mail to "user2" via LMTP on "imap1". I do get a soft error, as expected. ==> Incoming Mail deferred. Here are the new log transcripts: - On the LMTP server: Sep 14 11:59:54 galia postfix/lmtp[25497]: CDB3573D48: to=<[EMAIL PROTECTED]>, relay=imap1[10.12.17.44], delay=0, status=deferred (host imap1[10.12.17.44] said: 451 4.4.3 Remote server unavailable (in reply to end of DATA command)) - On the "imap1" server: Sep 14 11:59:53 imap1 cyrus/master[26560]: about to exec /usr/lib/cyrus/bin/lmtpd Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: connection from galia.srv.in.karavel.com [10.12.17.31] Sep 14 11:59:53 imap1 cyrus/lmtp[26560]: executed Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: login: galia.srv.in.karavel.com [10.12.17.31] postman DIGEST-MD5 User logged in Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: connect(imap2) failed: Connection refused Question: == Can anybody tell me what went wrong and why "imap1" wasn't aware of the existence of mailboxes on "imap2" just after startup? I have another question, related to imapd startup : - I have 13000 mailboxes on "imap1" and 14000 mailboxes on "imap2". - When I start "imap1", it kicks "ctl_mboxlist -m" before spawning other processes. This mailbox synchroni
Squatter not working in murder/unified environment?
Hi, Environment : - Cyrus 2.3.7 - Unified Murder - 2 IMAP, 1 mupdate I have mailboxes on both imap servers (aberlour & chivas) and a shared mailbox "commun.test" on chivas. When I run "/usr/sbin/squatter -v" on aberlour, it stops on an error saying it can find a mailbox or partition located on chivas! On the command line I get: aberlour$ /usr/sbin/squatter -v error opening commun.test: System I/O error And in syslog: Sep 20 11:28:26 aberlour cyrus/squatter[5803]: indexing mailboxes Sep 20 11:28:26 aberlour cyrus/squatter[5803]: IOERROR: opening chivas!default/cyrus.header: No such file or directory Sep 20 11:28:26 aberlour cyrus/squatter[5803]: done indexing mailboxes If I run "/usr/sbin/squatter -v -r 'user.*'", it starts indexing the first mailboxes, located on aberlour, but as soon as it encounters a mailbox located on chivas it stops with an error: aberlour$ /usr/sbin/squatter -v -r 'user.*' error opening user.*: Mailbox does not exist Indexing mailbox user.aaraujo.DOSSIERS CLIENTS... Indexed 0 messages (0 bytes) into 112 index bytes in 0 seconds Indexing mailbox user.aaraujo.DOSSIERS CLIENTS.02- CLE DES CHAMPS... Indexed 1 messages (5505 bytes) into 17132 index bytes in 0 seconds [...] Indexing mailbox user.abaeza.tunisie... Indexed 0 messages (0 bytes) into 112 index bytes in 0 seconds Indexing mailbox user.abaeza.yield... Indexed 1520 messages (9372693 bytes) into 4953647 index bytes in 2 seconds error opening user.abbar.&AMk-KIP: System I/O error Total over all mailboxes: Indexed 8100 messages (51105784 bytes) into 26944157 index bytes in 11 seconds How can I run squatter correctly? Is this a bug or a configuration error on my side? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Loss of mail with Murder+LMTP (imapd 2.3.7): incoming mail rejected just after server restart
Hi, In my unified murder configuration (cyrus imapd 2.3.7) with 2 imap servers and 1 mupdate server, I have a unwanted behaviour when LMTP tries to deliver mail for a mailbox that is not available. Lets says I have a user, "user2", whoses mailboxes are on "imap2". Scenario 1 : == - mupdate UP - imap1 UP - imap2 UP When mupdate, imap1 and imap2 are running, everything goes well. Even when I deliver mail via LMTP to "imap1", who doesn't own "users2"'s mailboxes. "imap1" then connects to "imap2" and correctly delivers the mail. ==> OK Scenario 2 : === - mupdate UP - imap1 DOWN, then UP for less than 10 minutes - imap2 DOWN Now I shut down both "imap1""imap2", and restart ONLY imap1. The mupdate server is always running and has never been shut down. Again, I send an email for "user2" via LMTP to "imap1", the mailbox being hosted on "imap2". Instead of returning a soft error (4XX) as I would have expected, imap1 returns a hard error (5XX) to the SMTP/LMTP server, and the mail is lost! ==> Incoming Mail lost Here are log transcripts: - On the LMTP server (Postfix 2.0.x): Sep 14 11:50:29 galia postfix/lmtp[24424]: 80E8473F35: to=<[EMAIL PROTECTED]>, relay=imap1[10.12.17.44], delay=1154, status=bounced (host imap1[10.12.17.44] said: 550-Mailbox unknown. Either there is no mailbox associated with this 550-name or you do not have authorization to see it. 550 5.1.1 User unknown (in reply to RCPT TO command)) - On "imap1": Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: connection from galia.srv.in.karavel.com [10.12.17.31] Sep 14 11:50:28 imap1 cyrus/master[26548]: about to exec /usr/lib/cyrus/bin/lmtpd Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: login: galia.srv.in.karavel.com [10.12.17.31] postman DIGEST-MD5 User logged in Sep 14 11:50:28 imap1 cyrus/lmtp[26548]: executed Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: kick_mupdate: can't connect to target: Connection refused Sep 14 11:50:28 imap1 cyrus/lmtp[26411]: verify_user(user.user2) failed: Mailbox does not exist Sep 14 11:53:09 imap1 cyrus/master[26353]: process 26411 exited, status 0 Scenario 3 : == - mupdate UP - imap1 UP for more than 10 minutes - imap2 DOWN This time, if I wait *a few minutes* before resending a mail to "user2" via LMTP on "imap1". I do get a soft error, as expected. ==> Incoming Mail deferred. Here are the new log transcripts: - On the LMTP server: Sep 14 11:59:54 galia postfix/lmtp[25497]: CDB3573D48: to=<[EMAIL PROTECTED]>, relay=imap1[10.12.17.44], delay=0, status=deferred (host imap1[10.12.17.44] said: 451 4.4.3 Remote server unavailable (in reply to end of DATA command)) - On the "imap1" server: Sep 14 11:59:53 imap1 cyrus/master[26560]: about to exec /usr/lib/cyrus/bin/lmtpd Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: connection from galia.srv.in.karavel.com [10.12.17.31] Sep 14 11:59:53 imap1 cyrus/lmtp[26560]: executed Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: login: galia.srv.in.karavel.com [10.12.17.31] postman DIGEST-MD5 User logged in Sep 14 11:59:53 imap1 cyrus/lmtp[26530]: connect(imap2) failed: Connection refused Question: == Can anybody tell me what went wrong and why "imap1" wasn't aware of the existence of mailboxes on "imap2" just after startup? I have another question, related to imapd startup : - I have 13000 mailboxes on "imap1" and 14000 mailboxes on "imap2". - When I start "imap1", it kicks "ctl_mboxlist -m" before spawning other processes. This mailbox synchronisation process is very slow and takes at least 4 minutes. Why is this "ctl_mbox_list -m" so slow? Is this related to the fact that even when "imap1" is up and running, it still isn't fully operational before at least 10 minutes? Here is some info on my configuration. I can provide more info if needed. - All 3 servers running Debian stable+testing + home compiled Cyrus Imapd 2.3.7 - They all have bi-XEON with at least 3 Gb of RAM - Server load is very low, as this is a test environnment - 13000 mailboxes on imap1, 14000 on imap2 - 1200 total users, stored in LDAP - Database formats: ANNOTATION skiplist DUPLICATE berkeley-nosync MBOX skiplist PTS berkeley QUOTA quotalegacy SEEN skiplist SUBS flat TLS berkeley-nosync Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Is cyr_expire trying cross-server expiration in a Murder configuration?
Hi, I've set up a cyrus+murder configuratio in unified mode (back+front on the same servers) with Cyrus 2.3.7. I have 2 imap servers (aberlour and chivas), and one mupdate server. This morning I noticed lots of errors in the log file of "aberlour", regarding cyr_expire: Sep 14 02:01:00 aberlour cyrus/master[24472]: about to exec /usr/sbin/tls_prune Sep 14 02:01:00 aberlour cyrus/master[24473]: about to exec /usr/sbin/cyr_expire Sep 14 02:01:00 aberlour cyrus/tls_prune[24472]: tls_prune: purged 0 out of 0 entries Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: IOERROR: opening chivas!default/cyrus.header: No such file or directory Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: unable to open/lock mailbox commun.test Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: IOERROR: opening chivas!default/cyrus.header: No such file or directory Sep 14 02:01:00 aberlour cyrus/cyr_expire[24473]: unable to open/lock mailbox user.aamili [...] And vice-versa "chivas" has the same error messages mentionning "aberlour!default/cyrus.header". It seems to happen every niht ever since I installed murder. Why does this happen? It that normal or did I miss some important configuration issue? Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Unofficial Cyrus imapd 2.3.7 packages for Debian
Hi, I have created some unofficial Debian packages for Cyrus Imapd 2.3.7. People how are interested and want to test them can download them here : http://www.pragmatic-source.com/news/latest/cyrus-imapd-2.3-for-debian.html. These packages are built for Debian unstable are are not totally tested yet. Please don't hassle the official Debian developpers and ask your questions directly to me, although I cannot garantee that I'll be able to answer every question. Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Anyone with experience using imapsync
Hi, Rob Tanner wrote: > Hi, > > I'm trying to migrate mail from one IMAP server to another using the > perl program imapsync. Both the source and destination servers are > Cyrus IMAP4 v2.2.3 servers. I have added a second partition to the > destination server and made it the default by configuring imapd.conf > as follows: > [...] > > With this setup, I can use cyradm and by hand correctly adds users to > the new partition. When I use imapsync to copy users over. Instead > of folders such as user.xyz and user.xyx.sent and user.xyz.drafts, I > get userxyz and userxyzsent and userxyzdrafts -- all as separate and > entirely independent folders and not even a hierarchy. > > Here's a script I used just to test and move over one user: > > > #! /bin/bash > > ./imapsync \ >--host1 belgarath.linfield.edu --user1 cyrus --passfile1 > /home/rtanner/imapsync/cyrus.pwd \ >--host2 polgara.linfield.edu --user2 cyrus --passfile2 > /home/rtanner/imapsync/cyrus.pwd \ >--syncinternaldates \ >--subscribe \ >--include "^user\.aabryan.*$" > I had the same issue because the users existed in LDAP, but had no mailboxes yet on the new server (i.e. I had not executed "cm user.xyz" with cyradm). If you run imapsync with --debug and --debugimap, you should see that imapsync finds no prefix or separator for polgara. But in your case I think your error is that you connect as user "cyrus" and, as such, all maiboxes created on polgara will have wrong ACLs! The fact that imapsync does not find the prefix and the separator by itself is a side effect, AFAIK. First of all, you can add "--prefix2 INBOX. --sep2 ." to you command line, it works for me. Then, you should authenticate as "cyrus" but use each real user's login for authentication. This is a bit harder to configure, because the only mechanisms that support proxy-authentication are PLAIN and CRAM-MD5. imapsync only supports PLAIN but Cyrus forces you to use SSL with plain ;) Thereforce, after activating ssl in Cyrus, you should be able to use imapsync the following way: ./imapsync \ --host1 belgarath.linfield.edu --authuser1 cyrus --passfile1 /home/rtanner/imapsync/cyrus.pwd --user1 aabryan \ --host2 polgara.linfield.edu --authuser2 cyrus --passfile2 /home/rtanner/imapsync/cyrus.pwd --user2 aabryan \ --syncinternaldates \ --subscribe \ --prefix2 INBOX. --sep2 . --ssl1 --ssl2 \ --authmech1 PLAIN --authmech2 PLAIN Even better, put a list of all your users in a file, one by line, and run: for user in $(< /tmp/list_of_users.txt); do ./imapsync .. --user1 $user --user2 $user .. done If you run imapsync like thiis, you don't need to create the mailboxes on the destination server, imapsync will do it for you. Regards -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Sample Cyrus 2.3 config with murder and "unified" mupdate config?
Hi, I'm about to test Cyrus IMAPd 2.3 with murder and would like to know where I can find some documentation and examples of configuration with the new "unified" configuration. Moreover, now that the proxy functionnality has been merged into the normal daemons: What services do I have to configure on my imap servers? "imap" only, or also "proxyd"? "mupdate" slave? Do I still need a dedicated server to run "mupdate" master? Regards, -- Farzad FARID <[EMAIL PROTECTED]> Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
