slow populating a mailbox
I've been using the UW-IMAP mailutil to copy messages from my regular Unix mailboxes (for mutt) to IMAP. The messages are all going to a single folder for a single user. My impression is that this is rather slow, about 20 minutes for 100MG worth of messages. There is virtually no CPU use while this happens. The source traditional mailbox and destination IMAP server are on the same machine, using the same hard disk (no, it's not a real server). First, is this performance to be expected, or might there be something here I can improve? I have quite a bit of mail I'd like to migrate, so if there's an easy way to speed this up I'd like to do so. Second, where should I look to diagnose or solve this problem? I've seen a rather large array of answers on this list, and am unsure how to start. I'm also not sure how to strace imapd (I've seen the suggestion to use it and a previous request for details, but no response). This system is not in real use, so I have some flexibility. This is Debian GNU/Linux system with a SATA disk, using EVMS volume management and Reiser FS. 2.6 kernel. Pentium 4 CPU. I'm using the Debian packaging of Cyrus 2.2, and have generally not done a lot of tuning/tweaking at any level (kernel, file system, Cyrus). Thanks. Ross Boylan Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
listing specs for public folders using cyradm
Hi, I'm looking for a way to post info using the cyradmin utility for only the public folders. For example to show annotations of all user accounts: info user/* What I want to do now is to query for all none users ( i.e. all public folders), for example: info !user/* Is there a way to do this? I need this for both the 'lam' as the 'info' request. Kind regards, Pieter -- --- Able: 1996-2006: already 10 safe years in YOUR company! aXs GUARD has completed security and anti-virus checks on this e-mail (http://www.axsguard.com) --- Able NV: ond.nr 0457.938.087 RPR Mechelen Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: slow populating a mailbox
On Sun, 2006-09-24 at 23:17 -0700, Ross Boylan wrote: I've been using the UW-IMAP mailutil to copy messages from my regular Unix mailboxes (for mutt) to IMAP. The messages are all going to a single folder for a single user. My impression is that this is rather slow, about 20 minutes for 100MG worth of messages. There is virtually no CPU use while this happens. number of messages is much more important than the size of them. we've recently relocated a bunch of mailboxes using imapsync, we got a speed of about 300 KiB/s for our users. users which had mostly large files with attachments went faster, traditional Unix users went slower, perhaps a factor of two either way. The source traditional mailbox and destination IMAP server are on the same machine, using the same hard disk (no, it's not a real server). that's not improving matters, of course. Second, where should I look to diagnose or solve this problem? you could try oprofile to get systemwide performance numbers. This is Debian GNU/Linux system with a SATA disk, using EVMS volume management and Reiser FS. 2.6 kernel. Pentium 4 CPU. I'm using the Debian packaging of Cyrus 2.2, and have generally not done a lot of tuning/tweaking at any level (kernel, file system, Cyrus). I have no idea how to tweak ReiserFS (I don't want to lose my data again, so I stay away from it). with ext3 I'd suggest trying async mount, just to see the impact. if it helps, using a dedicated disk for the journal would be one way of improving the I/O wait. -- Kjetil T. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
admin authentication
Hello, we will use LDAP through saslauthd to authenticate our users. Is there a way to authenticate admin-users a different way at the same time? Best would be to hardcode a md5-password within the imapd.conf or to use /etc/passwd for that. But I don't want to pass everything through PAM just to authenticate the admin user. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Solaris 10 install of Cyrus
Hello, new to this list, but have been managing a Cyrus installation for a few years with a colleague, who has since left. I have been trying to install Cyrus Imap 2.2.12 on a Sun T2000, running Solaris 10, with little success. Well, actually it all works apart from the final part of the compilation which creates the perl module and the cyradm utility. I've tried all of what I could find as hints on the various list archives and still have the problem. Compiling with the bundled gcc, I used the following configure options: export CFLAGS=-fPIC export CC=gcc ./configure --with-dbdir=/usr/local/BerkeleyDB.4.4 --with-openssl=/usr/sfw/lib --with-sasl=/usr/lib/sasl2 --without-krbdes --without-krb --without-krb impl --without-snmp Followed by: make All goes well until the end: ### Making all in /usr/local/src/cyrus/cyrus-imapd-2.2.12/perl/imap Checking if your kit is complete... Looks good Note (probably harmless): No library found for -lssl Note (probably harmless): No library found for -lcrypto Writing Makefile for Cyrus::IMAP make[2]: Entering directory `/usr/local/src/cyrus/cyrus-imapd-2.2.12/perl/imap' cp IMAP/Admin.pm blib/lib/Cyrus/IMAP/Admin.pm cp IMAP.pm blib/lib/Cyrus/IMAP.pm cp IMAP/Shell.pm blib/lib/Cyrus/IMAP/Shell.pm cp IMAP/IMSP.pm blib/lib/Cyrus/IMAP/IMSP.pm /usr/bin/perl /usr/perl5/5.8.4/lib/ExtUtils/xsubpp -typemap /usr/perl5/5.8.4/lib/ExtUtils/typemap -typemap typemap IMAP.xs IMAP.xsc mv IMAP.xsc IMAP.c cc -c -I../../lib -I/usr/lib/sasl2/include -I/usr/sfw/lib/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -xarch=v8 -D_TS_ERRNO -xO3 -xspace -xildoff -DVERSION=\1.00\ -DXS_VERSION=\1.00\ -KPIC -I/usr/perl5/5.8.4/lib/sun4-solaris-64int/CORE -DPERL_POLLUTE IMAP.c cc: unrecognized option `-KPIC' cc: language ildoff not recognized cc: IMAP.c: linker input file unused because linking not done Running Mkbootstrap for Cyrus::IMAP () chmod 644 IMAP.bs rm -f blib/arch/auto/Cyrus/IMAP/IMAP.so LD_RUN_PATH=/usr/local/BerkeleyDB.4.4/lib:/usr/lib:/lib cc -G IMAP.o -o blib/arch/auto/Cyrus/IMAP/IMAP.so ../../lib/libcyrus.a ../../lib/libcyrus_min.a -L/usr/local/BerkeleyDB.4.4/lib -R/usr/local/BerkeleyDB.4.4/lib -ldb-4.4 -lsasl2 -lrt cc: IMAP.o: No such file or directory make[2]: *** [blib/arch/auto/Cyrus/IMAP/IMAP.so] Error 1 make[2]: Leaving directory `/usr/local/src/cyrus/cyrus-imapd-2.2.12/perl/imap' make[1]: *** [all] Error 1 make[1]: Leaving directory `/usr/local/src/cyrus/cyrus-imapd-2.2.12/perl' make: *** [all] Error 1 Then manually changing the Makefile in perl/imap to make the compiler options -fPIC and removing the optimize options allows the make to complete and make install to work, but with some errors: cc -c -I../../lib -I/usr/lib/sasl2/include -I/usr/sfw/lib/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\1.00\ -DXS_VERSION=\1.00\ -fPIC -I/usr/perl5/5.8.4/lib/sun4-solaris-64int/CORE -DPERL_POLLUTE IMAP.c IMAP.c: In function `XS_Cyrus__IMAP_DESTROY': IMAP.c:400: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_setflags': IMAP.c:439: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_clearflags': IMAP.c:465: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_flags': IMAP.c:492: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_servername': IMAP.c:521: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_processoneevent': IMAP.c:546: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP__authenticate': IMAP.c:580: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_addcallback': IMAP.c:644: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP__send': IMAP.c:771: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_getselectinfo': IMAP.c:887: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_fromURL': IMAP.c:929: warning: cast to pointer from integer of different size IMAP.c: In function `XS_Cyrus__IMAP_toURL': IMAP.c:981: warning: cast to pointer from integer of different size Running Mkbootstrap for Cyrus::IMAP () chmod 644 IMAP.bs rm -f blib/arch/auto/Cyrus/IMAP/IMAP.so LD_RUN_PATH=/usr/local/BerkeleyDB.4.4/lib:/usr/lib:/lib cc -G IMAP.o -o blib/arch/auto/Cyrus/IMAP/IMAP.so ../../lib/libcyrus.a ../../lib/libcyrus_min.a -L/usr/local/BerkeleyDB.4.4/lib -R/usr/local/BerkeleyDB.4.4/lib -ldb-4.4 -lsasl2 -lrt chmod 755 blib/arch/auto/Cyrus/IMAP/IMAP.so cp IMAP.bs blib/arch/auto/Cyrus/IMAP/IMAP.bs chmod 644 blib/arch/auto/Cyrus/IMAP/IMAP.bs cat cyradm.sh cyradm chmod a+x cyradm cp cyradm blib/script/cyradm /usr/bin/perl -MExtUtils::MY -e MY-fixin(shift) blib/script/cyradm Manifying blib/man1/cyradm.1 Manifying
Re: admin authentication
Marten Lehmann wrote: Hello, we will use LDAP through saslauthd to authenticate our users. Is there a way to authenticate admin-users a different way at the same time? Best would be to hardcode a md5-password within the imapd.conf or to use /etc/passwd for that. But I don't want to pass everything through PAM just to authenticate the admin user. Hi, You can use saslauthd/LDAP for your users and a local sasldb2 file for your admins. You don't need PAM at all. I use both LDAP and sasldb with these config lines: sasl_pwcheck_method: auxprop saslauthd sasl_auxprop_plugin: sasldb The following commands, sasldblistusers2 saslpasswd2, can be used to administer the /etc/sasldb2 file. With this configuration you will have the added benefit of digest-md5 or cram-md5 authentication for your admins (but not for LDAP users AFAIK, unless using cleartext password in the LDAP directory, which I wouldn't do). Regards -- Farzad FARID [EMAIL PROTECTED] Architecte Open Source / Pragmatic Source http://www.pragmatic-source.com/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Solaris 10 install of Cyrus
I'm tearing my hair out and can't see what's wrong. I've even tried using the Blastwave pkg format solaris package - this does install but seems to be lacking in cyradm. cyradm is in a separate package named cyrus_imapd_utils. Note however that it requires the blastwave perl to be installed. Yann Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: admin authentication
Hello, You can use saslauthd/LDAP for your users and a local sasldb2 file for your admins. You don't need PAM at all. unfortunately neither sasldb nor auxprop is compiled in at redhat enterprise linux 4 and I wouldn't like to compile everything from scratch: # saslauthd -v saslauthd 2.1.19 authentication mechanisms: getpwent kerberos5 pam rimap shadow ldap * sasl_pwcheck_method: auxprop saslauthd * sasl_auxprop_plugin: sasldb Could I just write sasl_pwcheck_method: getpwent saslauthd instead so /etc/passwd will be used whereas a linux-user with the same name as the admin-user of cyrus would have to be created? Regards Marten Lehmann Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Strange behave of Cyrus server
Hi all, I have configured Cyrus imapd server on my Linux machine. I have to enhance the server code, for which I will have to debug it. The problem is: whenever I start the services, sometimes, I dont get the Server Ready message, if sometimes I get it, then I am not able to login. I get the following error: . NO Login failed: generic failure One thing more, When I try to debug the imapd process, then sometimes it is attached to /usr/sbin/imapd as: Attaching to program: /usr/sbin/imapd, process 4971 And sometimes it gets attached directly as: Attaching to process 4957 I cant put the breakpoint on the required functions like cmdloop, cmd_login etc. in both the cases. I can only do so if the gdb will attach to /usr/src/redhat/BUILD/cyrus-imapd-2.2.3/imap/imapd, in my case. If any of you is having answers to any of my problem, please please help me. Thanks, Neelu. DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Pvt. Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Pvt. Ltd. does not accept any liability for virus infected mails. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: different certs for pop3 and imap
Hi! # File containing the certificate used for XXX. If not specified, the global # certificate is used. A value of disabled will disable SSL/TLS for XXX. You can use : imap_tls_cert_file: /etc/ssl/certs/cyrus-imap.pem imap_tls_key_file: /etc/ssl/private/cyrus-imap.key pop3_tls_cert_file: /etc/ssl/certs/cyrus-pop3.pem pop3_tls_key_file: /etc/ssl/private/cyrus-pop3.key lmtp_tls_cert_file: /etc/ssl/certs/cyrus-lmtp.pem lmtp_tls_key_file: /etc/ssl/private/cyrus-lmtp.key sieve_tls_cert_file: /etc/ssl/certs/cyrus-sieve.pem sieve_tls_key_file: /etc/ssl/private/cyrus-sieve.key Hope it help Eddy - Message d'origine - De : Marten Lehmann [EMAIL PROTECTED] À : info-cyrus@lists.andrew.cmu.edu Envoyé : 25 septembre 2006 11:12 Objet : different certs for pop3 and imap Hello, as we are transparently migrating to cyrus we need pop3 and imap services listen to different ip-address and the ssl-portions of pop3 and imap need to come along with different keys and certs. How can I configure this? I can see only one tls_cert_file and tls_key_file in imapd.conf. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
LMTP problem
Hi Trying to send mail local on my machine i get this error. [EMAIL PROTECTED] exim]# exim -v xxx From: zappe To: xx Subject: Test Test LOG: MAIN = xxx U=root P=local S=367 [EMAIL PROTECTED] exim]# delivering 1GRsrR-j4-7C LOG: MAIN == xxx R=localuser T=local_delivery defer (-1): LMTP connection closed after initial connection What can be wrong then? /Daniel Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: slow populating a mailbox
On Sun, 2006-09-24 at 23:17 -0700, Ross Boylan wrote: First, is this performance to be expected, or might there be something here I can improve? I have quite a bit of mail I'd like to migrate, so if there's an easy way to speed this up I'd like to do so. Second, where should I look to diagnose or solve this problem? Depending on the number of messages, it could be the constant fsync() that slows it down. Try my fakesync library with LD_PRELOAD and perform a test migration; I'd like to know if it makes a difference, because I haven't had chance to test it much. Obviously, you really only want to run this during testing or migration, because it would make normal operation considerably less reliable. http://haus.nakedape.cc/svn/public/trunk/small-projects/fakesync/ Wil -- Wil Cooley [EMAIL PROTECTED] http://nakedape.cc signature.asc Description: This is a digitally signed message part Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: admin authentication
On Mon, 2006-09-25 at 16:17 +0200, Marten Lehmann wrote: Hello, You can use saslauthd/LDAP for your users and a local sasldb2 file for your admins. You don't need PAM at all. unfortunately neither sasldb nor auxprop is compiled in at redhat enterprise linux 4 and I wouldn't like to compile everything from scratch: # saslauthd -v saslauthd 2.1.19 authentication mechanisms: getpwent kerberos5 pam rimap shadow ldap auxprop is part of the Cyrus IMAP itself, not saslauthd; it won't be listed there. * sasl_pwcheck_method: auxprop saslauthd * sasl_auxprop_plugin: sasldb Could I just write sasl_pwcheck_method: getpwent saslauthd instead so /etc/passwd will be used whereas a linux-user with the same name as the admin-user of cyrus would have to be created? Try it? Wil -- Wil Cooley [EMAIL PROTECTED] http://nakedape.cc signature.asc Description: This is a digitally signed message part Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Recursively create mailboxes where cyrus.* files missing
I've just finished migrating a number of accounts from a UW-IMAP server. I'd like to recursively create mailboxes in the user namespace where there are no cyrus.* files in the folder: /var/spool/imap/user/bob/parent/child/grandchild corresponds to the user namespace: user.bob.parent.child.grandchild There are no cyrus.* files in the parent or child folders, which will cause problems in my environment. This can be solved manually with cyradm: localhost cm user.bob.parent localhost cm user.bob.parent.child but I'd prefer to handle this in an automated way, since it involves a lot of mailboxes. Before I roll my own perl script, I thought I'd check here in case someone can suggest an existing tool or share a solution that worked for them. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
problem deleting messages with cyrus-imapd 2.3.3-6
after deleting a whole bunch of messages from my INBOX (and expunging/purging trash), when i inspect the /var/spool/imap/b/user/bryan directory i notice that the messages are still there. they are also still in the trash folder as well. they are no longer displayed in the client (squirrelmail or evolution), however. this also happens when i copy a message from one folder to another -- the message no longer shows up in the client but it doesn;t get removed from the spool directory on the server. is this a known problem with this release (2.3.3.6)? bz Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: problem deleting messages with cyrus-imapd 2.3.3-6
This is the delayed expunge feature, added in 2.3. man imapd.conf, page down to expunge_mode. It's a speed enhancement and has the side effect of allowing sysadmin undelete/unexpunge without recourse to backups. You should have something like this in cyrus.conf for garbage collection. man cyr_expire for details. delprune cmd=cyr_expire -E 2 -X 2 at=0400 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
