thoughts on running an IMAP-over-SSL server exposed to the Internet?
I want to read my email on the iPhone. To do that, I have 2 options: 1. VPN 2. IMAP-over-SSL #1 is a bit convoluted, I already run a VPN server, with OpenVPN, but the iPhone doesn't have an OpenVPN client. Running *two* VPN networks seems excessive for a small personal server - not that the machine cannot handle it, but it just feels too complicated for the task at hand. #2 would be easy to implement, just poke a hole in the firewall for the imaps port. But then there's the issue of security, of course. I am running cyrus-imapd-2.3.7 on CentOS 5.x How comfortable y'all are with exposing Cyrus IMAPd's imaps port to the big wild Internet? Do you see the SELinux confinement as a must-have in this context, or are you okay with running it without any such MAC protections? -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP sync tool (rsync for IMAP)
Florin Andrei wrote: Anyway, slower is better than crashing. I'll try and contact the author to make him aware of this discussion thread. I did it and there's version 1.200 posted on Freshmeat claiming the memory issue has been fixed. I'll give it a try one of these days. http://freshmeat.net/projects/imapsync/ -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP sync tool (rsync for IMAP)
James Miller wrote: I have 2 users left on an OLD Cyrus installation I want to get off and was hoping to use imapsync. It seemed to work just fine except dates of the messages from the old server to the new were not retained. All of the messages have the date they were imported instead. I ran imapsync on the destination host, I wouldn't think that would make a difference - does it matter? /usr/bin/imapsync --syncinternaldates --host1 source.host --user1 bob \ --noauthmd5 --passfile1 /tmp/secret1 --host2 destination.host --user2 bob \ --noauthmd5 --passfile2 /tmp/secret2 I used it without --syncinternaldates and it worked fine, the dates were imported correctly. imapsync --host1 X.Y.Z.K --user1 joeblow --passfile1 pass1 \ --host2 A.B.C.D --user2 joeblow --passfile2 pass2 \ --authmech1 LOGIN --authmech2 LOGIN \ --prefix2 oldmail/ \ --subscribe The --prefix2 option was used because I did not actually migrate one account to another, but I just wanted to copy the old account under a folder on the new account. That doesn't seem to be your case. -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP sync tool (rsync for IMAP)
John Capo wrote: A 32MB message occupies a lot more than 32MB when its a Perl string. Use a temp file instead. my $message_file = /tmp/xfer-$$; unlink($message_file); $from-message_to_file($message_file, $f_msg); And further down. unless($new_id = $to-append_file($t_fold, $message_file, $flags_f, $d)) Instead of unless($new_id = $to-append_string($t_fold,$string, $flags_f, $d)) It's hard to tell without actually testing that change, but I assume it would slow down the script a little bit? Anyway, slower is better than crashing. I'll try and contact the author to make him aware of this discussion thread. -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP sync tool (rsync for IMAP)
Marten Lehmann wrote: Downside: If you have very large messages on the source IMAP servers, you better run the script on a machine with lots of RAM. On my system it died repeatedly with out of memory when it was hitting a 32MB message (on-disk size), and the system has 1GB RAM. strange, I didn't experience this while running a sync for 15.000 mailboxes some weeks ago with even larger messages partially. Yeah, and after a quick google search it turned out I'm not the only one to experience that. Ah well, except for just one very large message, everything else was transferred OK. -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: IMAP sync tool (rsync for IMAP)
Florin Andrei wrote: Sort of like rsync for IMAP, if that makes sense. So far, the only tool I've found is imapsync: http://freshmeat.net/projects/imapsync/ Seems to be working fine. It's recursive, it's smart (subsequent runs don't download the old messages again, only the new messages), the command-line switches are a rich set that's pretty much self-explanatory. It's easy to do non-destructive tests. It's flexible enough. Downside: If you have very large messages on the source IMAP servers, you better run the script on a machine with lots of RAM. On my system it died repeatedly with out of memory when it was hitting a 32MB message (on-disk size), and the system has 1GB RAM. It's probably best to run it on a 3rd system, one that's separate from both the source and the destination IMAP servers, otherwise unpleasant things might happen if the script starts to gobble up RAM on the IMAP server. Again, this 3rd system is best if it has lots of RAM if you allow large messages on your source IMAP server. -- Florin Andrei http://florin.myip.org/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
migrating the Sieve filters
I'm moving my Cyrus IMAP server to another machine. Instead of dumping/restoring the files, i decided to keep both servers running for a while, configure my IMAP client to connect to both, and use the IMAP client to move the messages from the old server to the new. Then i'll shut down the old server. Problem is, i don't know how to migrate the Sieve rules. Which files need to be moved over to migrate the filter rules? -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: New Cyrus Server - what would be ideal?
On Sun, 2004-07-11 at 13:07, Elizabeth Schwartz wrote: I'm planning to build a new, larger, server with Solaris 9, and cyrus, and migrate the existing accounts. Unanswered questions: sendmail or postfix? (or exim or qmail?) What combination of milters and filters? Silkymail or Squirrelmail? It depends on your goals. Definitely not Sendmail. It's less secure, slower, eats up more resources, the configuration is more obscure. Postfix looks better almost any way you look at it. With Postfix, i use amavisd-new to process email. amavisd-new calls SpamAssassin for antispam processing and ClamAV for antivir. It can integrate many other tools as well. I also installer Razor, which is used automatically by SpamAssassin, to enhance the antispam capabilities. http://www.ijs.si/software/amavisd/ As a webmail app, you can also look at IMP, as a part of the Horde suite. Horde is more complex, it offers calendaring tools, contact management, Sieve rules builder, bug tracker, etc. http://www.horde.org/ If your site has a lot of email traffic, you may want to separate the antispam/antivir processing (the system running amavisd-new) from the IMAP server (cyrus). -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
decrease log verbosity
Is there any way to decrease the syslog messages verbosity? E.g., for each message i get about 9 syslog messages about lmtpunix. I wonder if there's a config flag or something to discard all of them with the exception of either duplicate_check or duplicate_mark. The reason i want to keep one of these is that they contain the destination folder of the message. -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
best way to share directories
(newbie question) On a Cyrus IMAPd server with only a handful of users, what's the best way (what are the cyradm commands) to create a couple folders, not particularly owned by any one of them, to which all users have read/write access? Or maybe owned by an account, which is going to be a machine account? -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
script processing messages
What's the most simple method to connect to a Cyrus IMAP server, login as a user, go to a certain folder (the path is known), delete all messages in it, then logout - and do all that from a shell script? (a cron job) What i have to accomplish is to periodically grab all messages from a folder then empty the folder. I know i can simply copy the message files from the Cyrus spool, because that's exactly what i need (one message per file, and it must be the whole message, headers included), however i need a method to cleanly delete the messages afterwards. -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus crashed again badly
checkpoint: Invalid argument May 10 12:07:27 stantz imap[19943]: executed May 10 12:07:27 stantz imaps[19944]: executed May 10 12:07:27 stantz lmtpunix[19945]: executed May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: sync /var/lib/imap/db: cyrusdb error May 10 12:07:27 stantz lmtpunix[19945]: DBERROR db4: environment not yet opened May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR db4: DB_ENV-log_archive interface requires an environment configured for the logging subsystem May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: opening /var/lib/imap/deliver.db: Invalid argument May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: error listing log files: Invalid argument May 10 12:07:27 stantz lmtpunix[19945]: DBERROR: opening /var/lib/imap/deliver.db: cyrusdb error May 10 12:07:27 stantz ctl_cyrusdb[19942]: DBERROR: archive /var/lib/imap/db: cyrusdb error May 10 12:07:27 stantz lmtpunix[19945]: FATAL: lmtpd: unable to init duplicate delivery database May 10 12:07:28 stantz ctl_cyrusdb[19942]: archiving database file: /var/lib/imap/mailboxes.db May 10 12:07:28 stantz master[19933]: process 19945 exited, status 75 May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR db4: txn_checkpoint interface requires an environment configured for the transaction subsystem May 10 12:07:28 stantz master[19933]: service lmtpunix pid 19945 in READY state: terminated abnormally May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR: couldn't checkpoint: Invalid argument May 10 12:07:28 stantz master[19946]: about to exec /usr/lib/cyrus-imapd/imapd May 10 12:07:28 stantz master[19947]: about to exec /usr/lib/cyrus-imapd/lmtpd May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR: sync /var/lib/imap/db: cyrusdb error May 10 12:07:28 stantz imap[19946]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:28 stantz lmtpunix[19947]: DBERROR db4: fatal region error detected; run recovery May 10 12:07:28 stantz ctl_cyrusdb[19942]: DBERROR db4: DB_ENV-log_archive interface requires an environment configured for the logging subsystem May 10 12:07:28 stantz imap[19946]: DBERROR: dbenv-open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:28 stantz lmtpunix[19947]: DBERROR: dbenv-open '/var/lib/imap/db' failed: DB_RUNRECOVERY: Fatal error, run database recovery May 10 12:07:28 stantz master[19933]: exiting on SIGTERM/SIGINT -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus crashed again badly
On Mon, 2004-05-10 at 12:35, Florin Andrei wrote: This is the second time the Cyrus IMAP server crashes spectacularly. Nevermind, i managed to fix it, there were some corrupted files in /var/lib/imap/db, i removed them, and also deliver.db, everything got recreated, and now it's working. -- Florin Andrei http://florin.myip.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
mailboxes.db: cyrusdb error
Here's my little horror story, and how i got out of it.
I still have no explanation as for WHY it happened, other than maybe
some bug with cyrus-imapd? I'll let you guys draw conclusions.
Using:
cyrus-imapd-2.2.3
Linux Fedora Core 1
Last night i had to move my IMAP server to another location. I shut it
down normally (init 0), no errors being displayed during shutdown, all
was well.
This morning, when powering it up at the new location, Cyrus was not
working anymore:
Mar 12 10:29:02 stantz master[2276]: process started
Mar 12 10:29:02 stantz master[2277]: about to exec
/usr/lib/cyrus-imapd/ctl_cyrusdb
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: recovering cyrus databases
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: DBERROR: skiplist recovery:
0090 should be ADD or DELETE
Mar 12 10:29:06 stantz ctl_cyrusdb[2277]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:06 stantz master[2276]: process 2277 exited, status 75
Mar 12 10:29:06 stantz master[2324]: about to exec
/usr/lib/cyrus-imapd/idled
Mar 12 10:29:06 stantz idled[2324]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:06 stantz idled[2324]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:06 stantz idled[2324]: can't read mailboxes file
Mar 12 10:29:06 stantz idled[2324]: exiting
Mar 12 10:29:06 stantz master[2276]: process 2324 exited, status 75
Mar 12 10:29:06 stantz master[2276]: ready for work
Mar 12 10:29:06 stantz master[2332]: about to exec
/usr/lib/cyrus-imapd/ctl_cyrusdb
Mar 12 10:29:06 stantz master[2333]: about to exec
/usr/lib/cyrus-imapd/imapd
Mar 12 10:29:06 stantz master[2334]: about to exec
/usr/lib/cyrus-imapd/imapd
Mar 12 10:29:06 stantz ctl_cyrusdb[2332]: checkpointing cyrus databases
Mar 12 10:29:06 stantz master[2335]: about to exec
/usr/lib/cyrus-imapd/lmtpd
Mar 12 10:29:06 stantz imap[2333]: executed
Mar 12 10:29:06 stantz imaps[2334]: executed
Mar 12 10:29:06 stantz lmtpunix[2335]: executed
Mar 12 10:29:06 stantz ctl_cyrusdb[2332]: archiving database file:
/var/lib/imap/annotations.db
Mar 12 10:29:07 stantz imap[2333]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:07 stantz imap[2333]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:07 stantz imap[2333]: Fatal error: can't read mailboxes
file
Mar 12 10:29:07 stantz imaps[2334]: DBERROR: skiplist recovery: 0090
should be ADD or DELETE
Mar 12 10:29:07 stantz imaps[2334]: DBERROR: opening
/var/lib/imap/mailboxes.db: cyrusdb error
Mar 12 10:29:07 stantz imaps[2334]: Fatal error: can't read mailboxes
file
Mar 12 10:29:07 stantz master[2276]: process 2333 exited, status 75
Mar 12 10:29:07 stantz master[2276]: service imap pid 2333 in READY
state: terminated abnormally
Mar 12 10:29:07 stantz master[2276]: process 2334 exited, status 75
Mar 12 10:29:07 stantz master[2276]: service imaps pid 2334 in READY
state: terminated abnormally
To my utter dismay, none of the typical recovery procedures worked. I
tried these things:
/usr/lib/cyrus-imapd/ctl_mboxlist -d -f /var/lib/imap/mailboxes.db
(said it cannot read mailboxes.db)
/usr/lib/cyrus-imapd/reconstruct -m
(said it's not implemented! man, that was painful)
Fortunately, i googled up the mailing lists for messages containing
similar errors, and i found a script made by Edward Rudd, which i
modified slightly:
#!/bin/sh
TAB=`echo -e \\\t`
#cd /var/spool/imap/user
cd /var/spool/imap/f/user
find . -type d | grep ./ |
sed -e s/\.\/// |
sed -e s/\//\./g |
sed -e
s/\([a-z]*\)\(.*\)/user\.\1\2${TAB}default${TAB}\1${TAB}lrswipcda${TAB}cyrus${TAB}lrswipcda${TAB}/
cd -
This essentially rebuilt the database in plan text format. I only have
the f/ directory in /var/spool/imap (this is a single-user IMAP server),
and also a stage./ directory which i have no idea what it represents.
Hopefully i did the right thing.
Then, as the cyrus user, i reimported it:
/usr/lib/cyrus-imapd/ctl_mboxlist -u -f /var/lib/imap/mailboxes.db
mailboxes.txt
And now it works. I verified, i seem to be able to access all my
folders, i can create and delete folders from my IMAP client
(Evolution), Postfix is able to deliver messages via LMTP, Sieve
filtering seems to do its job...
That was scary. Hopefully someone has an explanation for the problem.
--
Florin Andrei
http://florin.myip.org/
---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
can't create directories
I try to create some mail folders in Cyrus, but i can't get the job done. I created a new account and, so far, there are only two folders visible from my IMAP client (Evolution): account \_Inbox \_Trash From Evolution, i try to create a folder like this: account \_Inbox \_test \_Trash But i get an error and the folder does not get created. However, i can create it just fine as an Inbox subfolder: account \_Inbox | \_test \_Trash Then i went to cyradm and run this command: cm user.account.test However, that creates the test folder as a subfolder of Inbox (see last diagram), not as a first-level folder. How can i create more first-level folders than just Inbox and Trash? -- Florin Andrei http://florin.myip.org/ --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
idled issue
I compiled cyrus with idled, and idled is enabled in cyrus.conf: idled cmd=idled However, at every significant operation that i do from the IMAP client, i get this error in syslog: Feb 25 17:48:00 stantz imap[456]: error sending to idled: 2 What could be the cause? Do i have to configure anything else for idled besides that line in cyrus.conf? -- Florin Andrei http://florin.myip.org/ --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: can't create directories
On Wed, 2004-02-25 at 18:37, Craig Ringer wrote: On Thu, 2004-02-26 at 09:38, Florin Andrei wrote: I try to create some mail folders in Cyrus, but i can't get the job done. All of what you described is normal - under Cyrus IMAPd, all user folders are subdirectories of the INBOX, both in the mail store and logically to the mail client. I don't know if it's possible to change this - do you need to? Most IMAP clients work fine with this, and all our users have no trouble with the idea. I see. I was asking because, previously, the structure of the mail folders didn't put everything under Inbox, and i wanted to preserve that. And actually, another IMAP server that i've been playing with (Dovecot) cannot store messages in a folder that already has subfolders, so i was kinda worried about not being able to use Inbox in Cyrus if i create subfolders in it (looks like i was wrong?). How about other important folders, such as Sent, Draft, etc? Do these need to be created as Inbox subfolders as well? I'm probably paying too much attention to the way Evolution creates the local folders structure, with Inbox, Sent, Draft, Trash, etc. at top level, and allowing for other folders to be created at the same level. -- Florin Andrei http://florin.myip.org/ --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
