Re: Moving the Cyrus Mailstore
On Fri, 20 Oct 2006, Wickham, Larry wrote: Hi, Our server is suffering from impending doom, while servicing 11,500 users. Our mail store is in the default location under /var/spool/imap. The partition that / (/var is under here in our setup) is mounted on is 25 GB and I would like to move it to the partition where /home is mounted which has 1+ TB available. We have considered several solutions. The chosen solution would have minimal downtime and minimal chance of failure. 1) Repartitioning: Down the server and boot from a rescue platform. Backup and Resize /home. Create a new partition in the free space formerly occupied by /home. Copy /var to the new partition. Rename the existing /var. Edit fstab and boot. 2) Swaping mount points: Down the server and boot from a rescue platform. Move the contents of /home to a folder on the smaller drive. Copy /var to the larger drive. Modify fstab. 3) Migrating between mail stores Create a new mail store under /home and migrate the mail boxes? Not sure how or if we can do this. Please let me know your opinion on these solutions and any new suggestions are welcome and appreciated. Here's what I would do: 1. Decide on a location in /home to store the cyrus stuff (/home/cyrus?). 2. Use rsync to get a rough copy of /var/spool/imap sync'd to /home/cyrus. Do this well ahead of the migration and periodically run rsync again to keep the list of changes small. 3. Stop cyrus. 4. Do a final rsync. 5. Create a symlink from /var/spool/imap to /home/cyrus. 6. Start cyrus Simple, and it keeps downtime to a minimum. Rsync is awesome. Andy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Moving the Cyrus Mailstore
> 3) Migrating between mail stores > Create a new mail store under /home and migrate the mail boxes? Not sure how > or if we can do this. cp -a. Or mv. You choose. :) You shouldn't have to "down" the server and do any rescue-booting, just turn off the cyrus and delivery (MTA?) services, move the data, adjust cyrus' configuration to point to the new mail store, and start everything back up. John -- John Madden Sr. UNIX Systems Engineer Ivy Tech Community College of Indiana [EMAIL PROTECTED] Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Moving the Cyrus Mailstore
Hi, Our server is suffering from impending doom, while servicing 11,500 users. Our mail store is in the default location under /var/spool/imap. The partition that / (/var is under here in our setup) is mounted on is 25 GB and I would like to move it to the partition where /home is mounted which has 1+ TB available. We have considered several solutions. The chosen solution would have minimal downtime and minimal chance of failure. 1) Repartitioning: Down the server and boot from a rescue platform. Backup and Resize /home. Create a new partition in the free space formerly occupied by /home. Copy /var to the new partition. Rename the existing /var. Edit fstab and boot. 2) Swaping mount points: Down the server and boot from a rescue platform. Move the contents of /home to a folder on the smaller drive. Copy /var to the larger drive. Modify fstab. 3) Migrating between mail stores Create a new mail store under /home and migrate the mail boxes? Not sure how or if we can do this. Please let me know your opinion on these solutions and any new suggestions are welcome and appreciated. Larry Wickham [EMAIL PROTECTED] Systems Operations Specialist Eastern New Mexico University 1500 S. Ave K Portales, NM 88130 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
On Oct 20, 2006, at 04:09, Michael Menge wrote: Hi, i had much better performance with mailutil from UW-Imapd. It uses the IMAP-protocol like imapscyn but is not a scipt but a binary program and uses the imap APPEND command and does noe checks to see wich E-Mails are on the new server. Michael Menge I second the usefulness of mailutil. What also makes it neat is that it can read (natively) any c-client supported mail spool -- be it imap, local filesystem mail folders, etc. So, if you've got backend mail store that has a c-client driver, you can pull the data directly from the filesystem and bypass IMAP on the reading end altogether. -rob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Sieve
I have a user that would like copies off all email to go to a subfolder
of his mailbox and to keep a copy in the main inbox. However it seems
as if duplicate delivery suppression is killing that plan. Short of
making a new mailbox, how can I accomplish this.
The script I am using is below.
--
Thanks
Andre
# Sieve Filter
# Generated by Ingo (http://www.horde.org/ingo/) (October 20, 2006,
9:52 am)
require "fileinto";
if address :all :comparator "i;ascii-casemap" :is "To" "*" {
keep;
fileinto "Archived-Mail";
stop;
}
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: delayed expunge and subfolders
In the cyrus.cache file there are always 10 cache fields per message. But
what could cause this code in mailbox.c to fail in the middle of skipping
over those fields:
/* Compute size of this record */
cacheitembegin = cacheitem = mailbox->cache_base + cache_offset;
for (cache_ent = 0; cache_ent < NUM_CACHE_FIELDS; cache_ent++) {
cacheitem = CACHE_ITEM_NEXT(cacheitem);
}
I see this failure when I try to use cyr_expire with -X 1 or with any
number greater than zero.
(gdb) frame 0
#0 process_records (mailbox=0xffbff080, newindex=0xe30a8,
index_base=0xfecd , exists=29,
deleted=0x2eb77c0, numdeleted=0xffbfbedc, quotadeleted=0xffbfbef0,
numansweredflag=0xffbfbeec, numdeletedflag=0xffbfbee8,
numflaggedflag=0xffbfbee4, newcache=0xe3098,
new_cache_total_size=0xffbfbee0, expunge_fd=-1, last_offset=0,
decideproc=0x16e18 , deciderock=0xffbffd60,
expunge_flags=2) at mailbox.c:1930
(gdb) p cache_ent
$1 = 8
OS: Solaris 9/Sparc
CC: Gcc 3.4.5
Cyrus: 2.3.7
--
Leena Heino University of Tampere / Computer Centre
( liinu at uta.fi ) ( http://www.uta.fi/laitokset/tkk )
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyrus ipurge
Hi all, I have mailboxes in the format of [EMAIL PROTECTED], and I want to start deleting their spam on a regular basis. I'm testing this out using ipurge, so i try: ipurge -d 7 -f '[EMAIL PROTECTED]', and after running that i'm just returned to the command prompt. no errors and no log output. i'm at a bit of a loss as to why it's not working - does anyone have any ideas? paul Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus IMAP 2.2.12 + Berkeley DB 4.5.20: Compile error in cyrusdb_berkeley.o
Hi, When compiling the latest cyrus imap server 2.2.12 with berkeley db 4.5.20 on Solaris 9 with gcc I get this error: ==CUT== [EMAIL PROTECTED]:/tmp/cyrus-imapd-2.2.12> gmake ### Making all in /tmp/cyrus-imapd-2.2.12/man gmake[1]: Entering directory `/tmp/cyrus-imapd-2.2.12/man' gmake[1]: Nothing to be done for `all'. gmake[1]: Leaving directory `/tmp/cyrus-imapd-2.2.12/man' ### Making all in /tmp/cyrus-imapd-2.2.12/et gmake[1]: Entering directory `/tmp/cyrus-imapd-2.2.12/et' gmake[1]: Nothing to be done for `all'. gmake[1]: Leaving directory `/tmp/cyrus-imapd-2.2.12/et' ### Making all in /tmp/cyrus-imapd-2.2.12/lib gmake[1]: Entering directory `/tmp/cyrus-imapd-2.2.12/lib' gcc -c -I.. -I/opt/adncyrus/include -I/opt/adncyrus/include -I/opt/adncyrus/include -I../et -I/opt/adncyrus/include -DHAVE_CONFIG_H -g -O2 \ cyrusdb_berkeley.c cyrusdb_berkeley.c: In function `init': cyrusdb_berkeley.c:152: warning: passing arg 2 of pointer to function from incompatible pointer type cyrusdb_berkeley.c:162: error: structure has no member named `set_lk_max' gmake[1]: *** [cyrusdb_berkeley.o] Error 1 gmake[1]: Leaving directory `/tmp/cyrus-imapd-2.2.12/lib' gmake: *** [all] Error 1 ==CUT== Looks like an API fluctuation of Berkely DB. Thanks in advance. Regards, Bernd *System Information* > uname -a SunOS adnpool02 5.9 Generic_112233-12 sun4u sparc SUNW,Sun-Fire-V240 > gcc -v Reading specs from /share/app/gnu/gcc/3.4.1/lib/gcc/sparc-sun-solaris2.8/3.4.1/specs Configured with: ../configure --prefix=/share/app/gnu/gcc/3.4.1 Thread model: posix gcc version 3.4.1 > ld -v GNU ld version 2.14 20030612 *Compile Instructions* PATH="/share/app/gnu/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/ccs/bin" CC=gcc CFLAGS=-O3 -fPIC -Wall CXX=g++ CXXFLAGS=-O3 -fPIC -Wall CPPFLAGS="-I/opt/adncyrus/include" LDFLAGS="-R/opt/adncyrus/lib -L/opt/adncyrus/lib -R/opt/adncyrus/lib/sparcv9 -L/opt/adncyrus/lib/sparcv9" LD_OPTIONS=$LDFLAGS unset LD_LIBRARY_PATH export PATH CC CFLAGS CXX CXXFLAGS LDFLAGS CPPFLAGS LD_OPTIONS ### OpenSSL gtar -C /tmp -xvpzf openssl-0.9.8d.tar.gz cd /tmp/openssl-0.9.8d ./config --prefix=/opt/adncyrus \ --openssldir=/opt/adncyrus/openssl shared \ -R/opt/adncyrus/lib -L/opt/adncyrus/lib \ -R/opt/adncyrus/lib/sparcv9 -L/opt/adncyrus/lib/sparcv9 gmake gmake install cd - ### OpenLDAP gtar -C /tmp -xvpzf openldap-2.3.27.tgz cd /tmp/openldap-2.3.27 ./configure --prefix=/opt/adncyrus \ --sysconfdir=/etc/opt/adncyrus \ --disable-slapd --disable-slurpd \ --localstatedir=/var/opt/adncyrus \ --with-tls gmake depend gmake gmake install cd - ### Berkeley DB gtar -C /tmp -xvpzf db-4.5.20.tar.gz cd /tmp/db-4.5.20/build_unix ../dist/configure --prefix=/opt/adncyrus gmake gmake install cd - ### Cyrus SASL gtar -C /tmp -xvpzf cyrus-sasl-2.1.22.tar.gz cd /tmp/cyrus-sasl-2.1.22 ./configure --prefix=/opt/adncyrus \ --with-dbpath=/var/opt/adncyrus/sasldb2 \ --sysconfdir=/etc/opt/adncyrus \ --with-dblib=berkeley \ --with-bdb-libdir=/opt/adncyrus/lib \ --with-bdb-incdir=/opt/adncyrus/include \ --with-openssl=/opt/adncyrus \ --with-ldap=/opt/adncyrus \ --with-plugindir=/opt/adncyrus/lib/sasl2 \ --with-saslauthd=/var/opt/adncyrus/socket \ --with-des=/opt/adncyrus \ --enable-shared \ --enable-static \ --disable-gssapi \ --disable-kerb5 gmake gmake install cd - ### Cyrus IMAP gtar -C /tmp -xvzf cyrus-imapd-2.2.12.tar.gz cd /tmp/cyrus-imapd-2.2.12 ./configure --prefix=/opt/adncyrus \ --with-auth=unix \ --with-cyrus-prefix=/opt/adncyrus \ --with-dbdir=/opt/adncyrus \ --with-openssl=/opt/adncyrus \ --with-sasl=/opt/adncyrus \ --sysconfdir=/etc/opt/adncyrus \ --with-statedir=/var/opt/adncyrus/socket \ --disable-krb4 \ --disable-cram \ --disable-gssapi \ --without-krb gmake Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: CREATE INBOX/attention: NO Invalid mailbox name
Hello, ok, then I have to rename them manually. Rename what manually? Seriously, are you considering the problem correctly? I mean that I have to rename the INBOX/* folders before the migration. Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: idled vs poll
Scott Adkins wrote: cannot connect to saslauthd server: Connection refused Failed to connect to socket /var/cyrus/imap/socket/lmtp for local_cyrus_deliver transport: Connection refused error sending to idled: 0 The first thing I would do here is to use LMTP as a TCP socket instead of a UNIX domain socket. We use Tru64 in our environment, and when I first deployed Cyrus under that architecture, the first thing I noticed what that as load increased, the more "connection refused" messages I saw with LMTP. Pointing my MTA to a TCP LMTP socket completely eliminated that problem for us. However, you should still keep the UNIX socket, since the "deliver" program still uses that instead of the TCP socket (the last time I checked). Additionally, when setting up the TCP socket, it would be good to set it to listen only on localhost or a private address (to prevent Internet users from connecting to your LMTP server and bypassing your MTA and spam/virus filtering controls). It is either that, or you configure authentication for the LMTP server (which, admittedly, I have never done). For SASL, I don't know if there can be any changes there. We use UNIX sockets for it as well, and I haven't investigated to see if there is a TCP socket option. That might help if there is one. On our system, when I see load increase, I definitely see SASL authentication take longer as well. We have worked most of our load problems out (Tru64 related), so that has improved considerably. We use the "poll" method, not "idled". Good luck. Thanks for those suggestions. We managed to mitigate the SASL problem by running it as a real-time process, which at least allowed people to log in; it's also been suggested that, since we're on Solaris, rebuilding it to use RPC doors instead might help. And I suppose we might be able to work around the other symptoms as well - but the stress placed on the system is so disproportionate to the advantage gained from idled that the obvious thing to do (which we've done) is to switch back to the poll method; we don't see any socket errors at all with idled disabled, and the system load is trivial. regards, Adam. -- Adam Stephens Network Specialist - Email & DNS [EMAIL PROTECTED] Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: very slow syncing, any ideas?
Hi, i had much better performance with mailutil from UW-Imapd. It uses the IMAP-protocol like imapscyn but is not a scipt but a binary program and uses the imap APPEND command and does noe checks to see wich E-Mails are on the new server. Michael Menge Quoting Marten Lehmann <[EMAIL PROTECTED]>: Hello, I'm about to migrate several thousand mailboxes from Maildir to Cyrus using the tool imapsync. It does its job very well and when I tested the migration on a small development server it was very fast. But now on the production system the synchronisation is very slow with a maximum of one message per second (and we have gigabytes of messages in the storage, partically > 10,000 messages per mailbox!). The general load of the system isn't very high, maybe a load average of 30. I disabled the duplicate message suppression. The mailboxes.db is about 8 megabytes big with approx. 13,000 mailboxes and 4 default folders each (Drafts, Junk, Sent, Trash). I have the following entries in my configuration which should provide a better hierarchie and balance of directories than if they were all in one main directory: altnamespace: true hashimapspool: true unixhierarchysep: true virtdomains: userid I also tried to move the old Maildirs to a different server, so that getting messages from the old mailbox and putting it to the new mailbox through IMAP doesn't come up with reads and writes on the same server. But the performance benefit was minimal. But in the end, syncing is still really slow. It would take weeks to sync all mailboxes that way. How else could we move them to the new storage if doing it through IMAP is too slow? On the other hand we would like to keep all flags so I guess syncing it with IMAP is the only choice? What could be the reason to be that slow? Is it the big mailboxes.db? Regards Marten Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html M.Menge Tel.: (49) 7071/29-70316 Universitaet Tuebingen Fax.: (49) 7071/29-5912 Zentrum fuer Datenverarbeitung mail: [EMAIL PROTECTED] Waechterstrasse 76 72074 Tuebingen smime.p7s Description: S/MIME krytographische Unterschrift Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
