High Availability
Hi, We used cyrus for many years and switch to a proprietary system. We are juste looking back to cyrus. I would like to know the status of cyrus and HA: This documentation seems to consider that replication is edge.. http://cyrusimap.org/docs/cyrus-imapd/2.4.9/install-replication.php and it has been written in 2007 *Note that Cyrus replication is still relatively young in the grand scheme of things, and if you choose to deploy you are doing so at your own risk. * Is there somewhere a documentation, an howto for HA (proxies, murder and replication) Thanks Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univ-amu.fr/showuser.php?uid=lalot Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: ext3 / XFS [Was: Re: Does anyone allow unlimited or extremely large quotas?]
We use ext4 for more than one year now. Efficient and stable. A good choise 12 spool of 250GB over 10 FC disks using metalun. Dom 2010/11/16 Robert Mueller r...@fastmail.fm This is depends on what filesystem you are useing, I have mailboxes with hundreds of thousands of messages in them on XFS and have no problems, but on ext3 I start seeing slowdowns with a bit over ten thousand messages. Was dir_index enabled on that ext3 filesystem? Prior to dir-index ext3 was very slow for large folders. dir_index is not enabled by default in ext3. FYI our experience at Fastmail 2 years back was that reiserfs still much better than ext3 (even with dir_index) at handling large numbers of files in folders. We tried switching one server to ext3, but after a week or two it was being crushed by load and we switched back to reiserfs. However we've recently found that ext4 is at least as good as reiserfs at handling large directories, so we've started switching everything to ext4 and so far the migration is going well. So don't use ext3, but ext4 is ok. Oh, and we recently setup a spare machine with btrfs and tried replicating a few partitions to it. That wasn't good. Started off promising, but by the time it was 1/3 full, the machine was utterly crawling. Clearly not ready for production yet. Rob Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Recomendations for a Migration of a Cyrus mailStore with 70K users.
We just finished migrating using 12 spools ext4 RAID5 for much less users
(6000) with quotas up to 3Gb.
It tooks a few days on line just rename mailboxes to another spool.
my script looks like this. Just take care no to abort it in the middle of a
rename. It would be better to add a handler for SIGUP/SIGTERM to exit
properly.
New spool are sp0, sp1 etc.. I check if it's an old spool then rename
mailboxes.
Hope this helps
Dom
#!/usr/bin/perl
use Cyrus::IMAP::Admin;
my $DEBUG=0;
my $quota=3072000; # 3 000 Mo selon horde
my %Hspools=(
'a' = 'sp0',
'b' = 'sp1',
'c' = 'sp2',
'd' = 'sp3',
'e' = 'sp4',
'f' = 'sp5',
'g' = 'sp6',
'h' = 'sp0',
'i' = 'sp11',
'j' = 'sp3',
'k' = 'sp5',
'l' = 'sp7',
'm' = 'sp8',
'n' = 'sp5',
'o' = 'sp4',
'p' = 'sp9',
'q' = 'sp11',
'r' = 'sp10',
's' = 'sp4',
't' = 'sp10',
'u' = 'sp11',
'v' = 'sp9',
'w' = 'sp11',
'x' = 'sp11',
'y' = 'sp11',
'z' = 'sp11',
);
### Connect IMAP
my $client = Cyrus::IMAP::Admin-new('127.0.0.1');
$client-authenticate( -user = 'cyrus', -password = 'xxx',-mechanism
= LOGIN) or die $client-error;
@res=`/usr/cyrus/bin/ctl_mboxlist -d /var/lib/imap/mailboxes.db`;
open (LOG,'/root/movebal.txt');
$i=0;
foreach (@res){
if (/^user\.(\S+)\s+\d+\s+(\S+)\s+/){
$user=$1;
$spool=$2;
next if $user=~/\./;
next if $spool=~/^sp\d+/;
if (IsUserOnline($user)){
print $user en ligne\n;
next;
}
$i++;
$lettre=substr($user,0,1);
$client-setacl(user/$user, 'cyrus' = 'all');
$client-setacl(user/$user/*, 'cyrus' = 'all');
$client-rename(user/$user,user/$user,$Hspools{$lettre});
$now=localtime;
printf LOG $now $user $spool $lettre $Hspools{$lettre}\n;
if (defined($client-error)){
printf LOG Pb pour déplacer la bal $uid $Hspools{$lettre}:
,$client-error,\n;
}
$client-setquota(user/$user,'STORAGE' = $quota);
}
}
close LOG;
sub IsUserOnline{
my ($user)=...@_;
@ret=`grep $user\tuser.$user /var/lib/imap/proc/*`;
foreach (@ret){
return 1 if /$user/;
}
}
2010/6/16 Nestor A Diaz nes...@tiendalinux.com
Hello Cyrus People.
I have been always a proud user of the cyrus email system, currently my
larger cyrus installation is about 70K users, which have been working
fine, however i need to make some adjustems to improve the resposiviness
of the mailstore.
At the begining i use only a default mailstore, on a ext3 formatted
filesystem over a raid 10 / LVM, it works fine, however when it reaches
the 32768 directories, (a limitation of ext3) i have to create another
partition, and then another one, so i have currently 3 partitions in total.
Having defined some criteria in order to group every user into a
partition i will like some recomendations, taking in mind the current
cyrus server tools which i currently don't know what are their current
capabilities, my plan is to forget using cyrus partitions, and instead
using LVM partitions and the propper links to the physical partitions
from the logical one.
So i currently have:
partition-default : 30K users
partition-alt1: 20K users
partition-alt2 : 20K users.
and then i will like to have just one partition:
partition-default ext4 formatted with links to physical partitions based
on the first letter [a-z] of the user.
I think LVM is a great advantage, and in some way aliviate the need for
cyrus partitions, i it will lead me deal with the grow of a partition
and data administration so easy without too much downtime.
So, it means that accordingly to this stragegy which seems to be the
most simple for me, what would be the best technical path to accomplish
a good migration ? allowusermoves: yes option and renamemailbox is
enoguth to do this task ? what if i want to do this online ? can i ? how
? is there any script that will help me do that ?
Any suggestions or migration histories are welcome !
What about murder / perdition ? they make any sense in this configuration ?
Keep this good work !
Slds.
--
Nestor A. Diaz
Ingeniero de Sistemas
Tel. +57 1-485-3020 x 211
Cel. +57 316-227-3593
Tel. SIP: sip:2...@tiendalinux.com sip%3a...@tiendalinux.com
Email/MSN: nes...@tiendalinux.com
http://www.tiendalinux.com/
Bogota, Colombia
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
--
Dominique LALOT
Ingénieur Systèmes et Réseaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus via NFS
Hello, I asked somebody from another french university. They use a NetApp, put evrything on it and it works perfectly. I stayed on SAN anyway, but I would have liked the snapshot and the facilities to restore mails... Dom 2010/5/12 Agustín Eijo ague...@gmail.com Hello, I have a Mail server with 6000 mailbox running in debian 5.0.4 with cyrus-imapd-2.2 (2.2.13-14+lenny3) I want move both the configdirectory and partition-default to a storage NetApp 2020 via NFS. In my linux I mount in fstab whith this nfs options: ( hard,intr,proto=tcp,_netdev ) Has anyone done this? Any recommendation or commentary, before move this to NFS ? Thanks, and sorry for my English I'm from Argentine. Agustin. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: moving mailboxes to another spool without stopping service
Thanks for you answer. I tested through cyradm an was able to do the trick. renm user.test user.test spoolX moves user.test to another spool. I sent a mail and consulted the mbox using imp/horde while doing the move and saw nothing bad. Good job! Thanks Dom 2010/4/6 Sebastian Hagedorn haged...@uni-koeln.de Hi, --On 6. April 2010 15:54:20 +0200 LALOT Dominique dom.la...@gmail.com wrote: We have reached rapidly the size of one spool and we want to migrate some mailboxes to another spool. What is the best way to do that? It would be nice that ltmp reports a temporary failure when moving in order to keep the incoming mail in the smtp spool. it does. And also a way to prevent the user from working on the moving data on the imap/pop side Is there a way to do that? Just RENAME the folder. We run a cronjob at night that does this. It greps in /var/lib/imap/proc to make sure there is no active process for the user to be moved, but AFAIK that's not absolutely necessary. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. .:.Regionales Rechenzentrum (RRZK).:. .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
moving mailboxes to another spool without stopping service
Hello, We have reached rapidly the size of one spool and we want to migrate some mailboxes to another spool. What is the best way to do that? It would be nice that ltmp reports a temporary failure when moving in order to keep the incoming mail in the smtp spool. And also a way to prevent the user from working on the moving data on the imap/pop side Is there a way to do that? Thanks in advance Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Autocreate patch for 2.3.15?
same for us. I vote for inclusion 2009/11/26 Michal Hlavinka mhlav...@redhat.com On Thursday 12 November 2009 08:28:45 Michael Menge wrote: Quoting Simon Matter simon.mat...@invoca.ch: Has anyone used the 2.3.14 autocreate patch on 2.3.15? There does not seem to be a patch for 2.3.15 on the site and I have tried using the 2.3.14 patch and it does not apply cleanly. It fails at imap/Makefile.in Find attached my version of the patch adopted for 2.3.15. Note that this is not an official patch from UOA but it is widely used because it's the patch used in my cyrus-imapd rpms. IIRC UOA is not using 2.3.x which is why they are not so fast upgrading the patches. Is there a reason that this patch is not in the official cyrus release? I also vote for inclusion, we use it for ages and it would make thinks better (at least from package maintainer's point of view). Regards, Michal Hlavinka Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
strange quota behaviour
Hello, Sometimes, I have some bad quotas. Here is a trace: [cy...@imap-perso ~]$ cyradm -user cyrus localhost Password: localhost.localdomain lq user/xx STORAGE 197/100 (0.0197%) [r...@imap-perso ~]# du -hs /var/spool/imap2/m/user/xx 890M/var/spool/imap2/m/user/xx reconstruct -rf user/xx [cy...@imap-perso ~]$ quota -f user/xx user.xx: usage was 201755, now 820057155 Quota % Used Used Root 100 80 800837 user/xx (note that: its better but 800M and 890 makes a difference. That user has lots of folders and subfloders) my version: cyrus-imapd-2.3.14 It happens quite a lot. Too much for me. I used some years ago to force a quota -f on each users. It worked on a 2.2.12 version, but when we migrate to a 2.3.x version, suddenly, the quotas were broken with over 1000%. We went back to 2.2.12 and got rid of quota -f (too dangerous) But now, that problem is coming back again and I still don't understand why. Fixing a quota should be easy to do. quota -f is very fast. Too fast I think.. Can I generalize a reconstruct -rf followed by a quota -f without fears? Do you experience such problems? Thanks Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: strange quota behaviour
2009/9/1 Michael Menge michael.me...@zdv.uni-tuebingen.de Quoting LALOT Dominique dom.la...@gmail.com: Hello, Sometimes, I have some bad quotas. Here is a trace: [cy...@imap-perso ~]$ cyradm -user cyrus localhost Password: localhost.localdomain lq user/xx STORAGE 197/100 (0.0197%) [r...@imap-perso ~]# du -hs /var/spool/imap2/m/user/xx 890M/var/spool/imap2/m/user/xx reconstruct -rf user/xx [cy...@imap-perso ~]$ quota -f user/xx user.xx: usage was 201755, now 820057155 Quota % Used Used Root 100 80 800837 user/xx Do you use delayed expunge, squatter? The size of cyrus.* files and eMails in delayed expunge mode are not included in the quota but in du. Reconstructing the mailbox without -k will delete the eMails in delayed expunge mode. No I have no expunge mode, my install is fairly basic.. Dom (note that: its better but 800M and 890 makes a difference. That user has lots of folders and subfloders) my version: cyrus-imapd-2.3.14 It happens quite a lot. Too much for me. I used some years ago to force a quota -f on each users. It worked on a 2.2.12 version, but when we migrate to a 2.3.x version, suddenly, the quotas were broken with over 1000%. We went back to 2.2.12 and got rid of quota -f (too dangerous) But now, that problem is coming back again and I still don't understand why. Fixing a quota should be easy to do. quota -f is very fast. Too fast I think.. Can I generalize a reconstruct -rf followed by a quota -f without fears? Do you experience such problems? Thanks Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot M.MengeTel.: (49) 7071/29-70316 Universität Tübingen Fax.: (49) 7071/29-5912 Zentrum für Datenverarbeitung mail: michael.me...@zdv.uni-tuebingen.de Wächterstraße 76 72074 Tübingen Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Api for seen state, perl preferred
Sorry to be late to answer, and thanks for the thread. I tried that command and gets lots of info: getting id in a folder, I got that [cy...@smtp ~]$ ./test.pl lastread:Tue May 19 14:14:04 2009 52 lastchange:Tue May 19 14:05:51 2009 1:52 I put an unread mail in that folder, it appears to be at uid 53 [cy...@smtp ~]$ ./test.pl lastread:Tue May 19 14:14:04 2009 52 lastchange:Tue May 19 14:05:51 2009 1:52 Nothing changed Then clicking on it: [cy...@smtp ~]$ ./test.pl lastread:Mon May 25 14:51:06 2009 53 lastchange:Mon May 25 14:51:06 2009 1:53 Yes that information has been updated. I begin to understand but my subfolder was easy, few messages and just one unread. When it's mixed it's more difficult: For my INBOX: [cy...@smtp ~]$ ./test.pl lastread:Mon May 25 14:49:37 2009 30714 lastchange:Mon May 25 14:45:48 2009 1:30421,30423:30443,30445,30448:30449,30451,30453:30455,30457:30467,30472,30474,30477:30478,30480,30485:30490,30492,30494,30496:30498,30502,30504,30506,30508:30509,30511:30514,30516:30518,30520,30522:30523,30526:30528,30530:30537,30548:30558,30577,30579:30581,30606:30607,30610,30612,30615:30619,30621,30623:30643,30650,30652,30656,30659,30663:30666,30669:30670,30672:30675,30677:30678,30681,30683,30685,30687:30689,30691,30696:30697,30699:30701,30703:30705,30710,30713 [cy...@smtp ~]$ ll /var/spool/imap2/l/user/lalot/3071* -rw--- 1 cyrus cyrus 1642 mai 25 16:31 /var/spool/imap2/l/user/lalot/30710. -rw--- 1 cyrus cyrus 31880 mai 25 16:35 /var/spool/imap2/l/user/lalot/30712. -rw--- 1 cyrus cyrus 2058 mai 25 16:41 /var/spool/imap2/l/user/lalot/30713. -rw--- 1 cyrus cyrus 8268 mai 25 16:47 /var/spool/imap2/l/user/lalot/30714. -rw--- 1 cyrus cyrus 2622 mai 25 16:50 /var/spool/imap2/l/user/lalot/30715. -rw--- 1 cyrus cyrus 7982 mai 25 16:56 /var/spool/imap2/l/user/lalot/30716. [cy...@smtp ~]$ ./test.pl lastread:Mon May 25 16:49:37 2009 30714 lastchange:Mon May 25 16:45:48 2009 1:30421,30423:30443,30445,30448:30449,30451,30453:30455,30457:30467,30472,30474,30477:30478,30480,30485:30490,30492,30494,30496:30498,30502,30504,30506,30508:30509,30511:30514,30516:30518,30520,30522:30523,30526:30528,30530:30537,30548:30558,30577,30579:30581,30606:30607,30610,30612,30615:30619,30621,30623:30643,30650,30652,30656,30659,30663:30666,30669:30670,30672:30675,30677:30678,30681,30683,30685,30687:30689,30691,30696:30697,30699:30701,30703:30705,30710,30713 The last 3 emails haven't been red. and we stopped at 30713 I clicked on last one: [cy...@smtp ~]$ ./test.pl lastread:Mon May 25 17:03:06 2009 30716 lastchange:Mon May 25 17:03:07 2009 1:30421,30423:30443,30445,30448:30449,30451,30453:30455,30457:30467,30472,30474,30477:30478,30480,30485:30490,30492,30494,30496:30498,30502,30504,30506,30508:30509,30511:30514,30516:30518,30520,30522:30523,30526:30528,30530:30537,30548:30558,30577,30579:30581,30606:30607,30610,30612,30615:30619,30621,30623:30643,30650,30652,30656,30659,30663:30666,30669:30670,30672:30675,30677:30678,30681,30683,30685,30687:30689,30691,30696:30697,30699:30701,30703:30705,30710:30711,30713,30716 So, I have all the ranges of what has been red. I can put the ranges in a structure, then do a search before xx and compare if the mails are in the range. I believe it can do the job with that. Thanks Dom 2009/5/20 Bron Gondwana br...@fastmail.fm On Wed, May 20, 2009 at 07:34:52AM -0400, Adam Tauno WIlliams wrote: Maybe I'm looking in the wrong place; but on my test server (cyrus-imapd-2.3.11) my cyrus.header looks like - estate1:/var/spool/imap/user/awilliam # cat cyrus.header Cyrus mailbox header The best thing about this system was that it had lots of goals. --Jim Morris on Andrew 46a0a0b041114dce Junk $Label1 $Label2 $Label3 $Label4 $Label5 $MDNSent NotJunk receipt-handled $has_cal awilliam lrswipcda Is 46a0a0b041114dce the id? Yeah, just means you have no quota root. That's the ID. [br...@imap3 hm]$ /usr/cyrus/bin/cyr_dbtool -C /etc/imapd-slot308-master.conf /mnt/meta8/slot308/store23/conf/user/b/brong.seen skiplist get 6af857f64475158a 1 1242810803 386716 1242800567 1:386615 Looks like I've seen all my email! Also notice 4 other fields at the start: version lastread lastuid lastchange seen_items lastchange is an epoch time stamp? Yes. I'm not clear on what lastuid is; looks like it is a message id? What operation sets/updates the lastuid value? It's a UID. Here's a more interesting couple of folders to give you an idea of what ranges look like: 798b2df94146a5fe1 1095746485 1526 1095740252 1:559,561:772,774:920,922:1109,:1115,1117:1138,1140:1170,1172:1195,1197:1521 798b2df9415762611 1125837870 34307 1123831223 1:28289,29118,30871,31887,31935,32058,32209:32210,32214:32219,3,32228 So - in summary. It's not easy. Some plumbing required... (lastuid gives you \Recent as per the IMAP flag - anything newer than that is recent!) So any
Api for seen state, perl preferred
Hello, I'm following a previous thread. I would like to be able to open a seen skiplist database in order to verify if a particular user has red its mail. If possible, a way to do that in PERL would be perfect. Thanks in advance Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: reading seen state as cyrus priviledged user
Well, I'm thinking about another way to do that. Running as localhost of cyrus-imap and as cyrus user, I'm able to open seen skiplist database. Is there a Perl module to interact with? Thanks Dom 2009/5/11 Bron Gondwana br...@fastmail.fm On Sun, May 10, 2009 at 08:44:27PM +0200, LALOT Dominique wrote: Hello, I would like to run scripts for deleting unread old mails. But using the imap API, I can only read the good seen status if I logged as the user. Checking as cyrus tells me, that nothing has been read. Is there an su option for imap? How can I do that? Yeah, that would be nice actually. I'm pretty sure there's nothing like it at the moment. There's probably something nasty you can do via SASL, but being able to see a particular user's seen state is a different matter entirely. I'm thinking something like this: A01 FETCH 1:* (USERFLAGS.brong) * FLAGS 1 \Seen.brong * FLAGS 2 \Recent A01 OK FETCH Completed might just about do the trick... obviously it would test if you were either an admin or that user before allowing it... Bron ( also, IMAP language hacks. Bah. Along with our DIGEST.SHA1 hack we're hardly talking IMAP anymore... ) -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: delete specific mail in all folders
That could be something like that script (test and adapt). I'm just a
beginer with Mail::IMAPTalk and I don't use correctly search method
The trick will be to find the from and subject and/or the date..
Dom
#!/usr/bin/perl
use Mail::IMAPTalk;
$IMAP = Mail::IMAPTalk-new(
Server = 'localhost',
Username = 'cyrus',
Password = 'xxx',
Uid = 1 )
|| die Failed to connect/login to IMAP server;
#http://linuxfr.org/forums/27/21722.html
$IMAP-set_root_folder('inbox', '/', 1, 'user');
@folder=$IMAP-list(user.%,'*');
foreach $bals (@folder){
$tot=0;
$msgnb=0;
foreach $bal (@$bals){
if ($bal=~/user\//){
#$IMAP-examine($_);
#$count=$IMAP-message_count($_);
$IMAP-select($bal) || warn cant access $bal: $@;
$ResSearch = $IMAP-search('1:*', 'FROM', 'xxx'); # find the good
search to do
$Res = $IMAP-fetch($Res, '(rfc822.size envelope)') || die cant
access $bal: $@;
foreach $id (keys %$Res){
$size=$Res-{$id}-{'rfc822.size'};
$subject=$Res-{$id}-{'envelope'}-{'Subject'};
print id=$id size=$size sub=$subject\n;
}
$IMAP-store($ResSearch, '+flags', '(\\deleted)');
$lasterr=$IMAP-get_last_error();
if (defined $lasterr){
next if ($lasterr=~/No fetch data/);
warn $bal: $@;
next;
}
}
}
}
$IMAP-logout();
2009/5/11 Gerald Nowitzky w...@igne.de
Hello!
I have been confronted with a request today: We are running cyrus as imap
server. We have currently about 3 million files and 150GB in our mail dir.
One user has accidently sent something confidential to all users via a
list.
The request was, of course, to delete the mail from all mailboxes. Is there
any reasonable approach to do something like this?
Thanks!
(Gerald)
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
--
Dominique LALOT
Ingénieur Systèmes et Réseaux
http://annuaire.univmed.fr/showuser.php?uid=lalot
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: delete specific mail in all folders
2009/5/11 Paul van der Vlis p...@vandervlis.nl Gerald Nowitzky schreef: Hello! I have been confronted with a request today: We are running cyrus as imap server. We have currently about 3 million files and 150GB in our mail dir. One user has accidently sent something confidential to all users via a list. The request was, of course, to delete the mail from all mailboxes. Is there any reasonable approach to do something like this? Maybe you could use 'find' with some characteristics of the message, like the size and the date. the size may differ as the destination change, same for date, as it's a bulk delivery which could last several minutes. You could replace the found files with the same message, but where all what is confidential was removed. Nice idea, but you will keep the subject in the index anyway. So, if the subject is Fuck the boss, you will not get rid of that.. With regards, Paul van der Vlis. -- http://www.vandervlis.nl/ Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Sieve international charachters
That would be great. It's a long awaited feature for us. Thanks Dom 2009/5/9 Bron Gondwana br...@fastmail.fm On Fri, May 08, 2009 at 03:32:47PM +0200, Gabriele Bulfon wrote: Hello, does sieve filtering accepts international characters (e.g. ?,? and so on)? Until now I could never input an automatic response containing such characters. But users are a bit disappointed to use apostrophe instead of their normal chars. Is there any way to encode such chars inside sieve scripting? Thx for any help. YES! Except I haven't pushed it into upstream Cyrus yet... http://github.com/brong/cyrus-sieve/tree/57b26fc75fe1748e54c8f6d51c16e8d8f8ee10bf It needs a bunch of changes from my cyrus-imapd tree as well to provide the UTF8 decoding functionality. I'm hoping to get it all polished up and ready for the next Cyrus release. It's a pretty intrusive set of changes, but it's been running happily for over a month here at FastMail now. Bron. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: reading seen state as cyrus priviledged user
2009/5/11 Dan White dwh...@olp.net LALOT Dominique wrote: Hello, I would like to run scripts for deleting unread old mails. But using the imap API, I can only read the good seen status if I logged as the user. Checking as cyrus tells me, that nothing has been read. Is there an su option for imap? How can I do that? You can authenticate as an admin, and authz as the user you wish to see seen state for. E.g.: imtest -m DIGEST-MD5 -a cyrus -u dwh...@olp.net localhost Interesting, I'll look at what is generated using debug mode of cyrus Thanks Dom Depending on your environment, the '/vendor/cmu/cyrus/imapd/sharedseen' annotation may be useful. See the 'cyradm' man page for details. -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
reading seen state as cyrus priviledged user
Hello, I would like to run scripts for deleting unread old mails. But using the imap API, I can only read the good seen status if I logged as the user. Checking as cyrus tells me, that nothing has been read. Is there an su option for imap? How can I do that? Thanks Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: limit tcp sessions opened by an IMAP client
Outlook 2007 has won the price.. It's a pity there is no options like, for a given address, no more than 5 simultaneous connexions Dom 2009/4/14 Joseph Brennan bren...@columbia.edu LALOT Dominique dom.la...@gmail.com wrote: . I've seen once entourage on macosx ignoring 5xx code from our smtp server, and trying to upload a 50Mo file every minute. Outlook will try every second, under some conditions! Funny, I was thinking Outlook Express for this imap problem. I've seen it start a new imap login to see whether there's new mail in the inbox it already has open (this is horrible in U Wash imap, where the new session kills the older one). If these were evenly timed, like every 5 minutes, I would have said Outlook Express. But these are at irregular intervals, so I think it's something else. Joseph Brennan Lead Email Systems Engineer Columbia University Information Technology Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
limit tcp sessions opened by an IMAP client
Hello, I've looked at google before asking, but I didn't find something. Some imap clients are using many tcp connexions. I would like to know if there is a way to limit them? Thanks Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: limit tcp sessions opened by an IMAP client
Look at this one: [r...@smtp ~]# host 82.240.88.126 126.88.240.82.in-addr.arpa domain name pointer val13-2-82-240-88-126.fbx.proxad.net. [r...@smtp ~]# netstat -atpn | grep 82.240.88.126 tcp0 0 139.124.132.126:993 82.240.88.126:60250 ESTABLISHED 9209/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60229 ESTABLISHED 8824/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60235 ESTABLISHED 8016/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60234 ESTABLISHED 8570/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60265 ESTABLISHED 10316/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60180 ESTABLISHED 3795/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60190 ESTABLISHED 5258/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60167 ESTABLISHED 5882/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60213 ESTABLISHED 6758/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60219 ESTABLISHED 8421/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60196 ESTABLISHED 7486/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60206 ESTABLISHED 7520/imapd tcp0 0 139.124.132.126:993 82.240.88.126:63218 ESTABLISHED 6288/imapd tcp0 0 139.124.132.126:993 82.240.88.126:60158 ESTABLISHED 5504/imapd I don't know how many processes we can have with a decent speed. For the moment, it turns to be around 1000 processes, but I don't know the max whe can stand. So the idea of mayak can be a solution. Filter with iptables 193.218.15.25 13 82.240.88.126 16 80.13.69.148 12 for the top, I got lines like this: Apr 14 16:10:25 smtp imaps[13462]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:11:43 smtp imaps[13530]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:11:43 smtp imaps[31581]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:12:41 smtp imaps[13644]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:12:42 smtp imaps[13481]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:15:08 smtp imaps[14234]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:15:08 smtp imaps[29088]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:17:14 smtp imaps[14080]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Apr 14 16:17:15 smtp imaps[14212]: login: val13-2-82-240-88-126.fbx.proxad.net [82.240.88.126] x plaintext+TLS User logged in Checking mail a little bit too much. 2009/4/14 Joseph Brennan bren...@columbia.edu LALOT Dominique dom.la...@gmail.com wrote: Hello, I've looked at google before asking, but I didn't find something. Some imap clients are using many tcp connexions. I would like to know if there is a way to limit them? This could make the client fail and increase your helpdesk calls. Do you mean more than five? Whatever you do should check both host and user, so that you don't cut off multiple users on a timeshare host or a firewall gateway. Joseph Brennan Lead Email Systems Engineer Columbia University Information Technology Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: limit tcp sessions opened by an IMAP client
2009/4/14 Joseph Brennan bren...@columbia.edu Strange to see so many logins spread over a short time. They seem to be in pairs, which is the way some clients start up. I wonder if the client thinks the connection has dropped, and so it starts new sessions. I realize the server's netstat shows them as still connected. It might be interesting to log sessions and see what's going on. Or to strace live processes. And of course ask the user what it looks like from his/her end. That's what I will do. But I'm trying to find configuration options to get the server stronger faced to bad behaved clients. I've seen once entourage on macosx ignoring 5xx code from our smtp server, and trying to upload a 50Mo file every minute. I don't know what will be this one. Dom Joseph Brennan Lead Email Systems Engineer Columbia University Information Technology Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser.php?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Doing Cyrus *right* for a large installation
2009/1/15 Rudy Gevaert rudy.geva...@ugent.be Andrew Morgan wrote: For those of you running large installations, how do you back them up? We have 7 backends: 400G 320G 81G 80% /mail/mail1 400G 273G 128G 69% /mail/mail2 450G 390G 61G 87% /mail/mail3 400G 322G 79G 81% /mail/mail4 400G 270G 131G 68% /mail/mail5 450G 397G 54G 89% /mail/mail6 50G 4.8G 46G 10% /mail/mail7 For mail1-6 incremental backup time takes between 2h and 7h, we suspect a storage (misconfiguration) issue. You can have a look at file fragmentation. We noticed severe degradation on ext3fs. More than 60% of the spool was fragmented. Try a filefrag on your data. To correct it, you have to copy to a fresh new spool Dom Fulls take a lot longer, but I can't give you any time estimates now. For the moment we don't backup the replica's. Nor use them. But we could do it. We are 'planning' a storage migration too the coming months. Rudy -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Rudy Gevaert rudy.geva...@ugent.be tel:+32 9 264 4734 Directie ICT, afd. Infrastructuur ICT Department, Infrastructure office Groep SystemenSystems group Universiteit Gent Ghent University Krijgslaan 281, gebouw S9, 9000 Gent, Belgie www.UGent.be -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: choosing a file system
Please, we are following a long thread I introduced a while ago which was speaking about file system. If you want to ask question about something you red in that thread, but not properly speaking about file system, please, be kind enough to start another thread and stop replying in that thread. Dom 2009/1/19 Jorey Bump l...@joreybump.com: Andrew McNamara wrote, at 01/19/2009 01:29 AM: Yeah, except Postfix encodes the inode of the queue files in its queue IDs, so it gets very confused if you do this. Same with restoring queues from backups. You should be able to get away with this if, when moving the queue to another machine, you move the queued mail from hold, incoming, active and deferred directories into the maildrop directory on the target instance. This (somewhat old, but still correct, I think) message from Wietse might shed more light on it: Date: Thu, 12 Sep 2002 20:33:08 -0400 (EDT) From: wie...@porcupine.org (Wietse Venema) Subject: Re: postfix migration I want to migrate postfix to another machine. What are also the steps so that I won't lose mails on the process? This is the safe procedure. 1) On the old machine, stop Postfix. 2) On the old machine, run as super-user: postsuper -r ALL This moves all queue files to the maildrop queue. 3) On the old machine, back up /var/spool/postfix/maildrop 4) On the new machine, make sure Postfix works. 5) On the new machine, stop Postfix. 6) On the new machine, restore /var/spool/postfix/maildrop 7) On the new machine, start Postfix. There are ways to skip the postsuper -r ALL step, and copy the incoming + active + deferred + bounce + defer + flush + hold directories to the new machine, but that would be safe only with an empty queue on the new machine. This has become somewhat off-topic for this list, but you might be able to simply sync the entire Postfix queue to the backup machine, and run postsuper -s before starting Postfix on the backup. From the postsuper man page: -s Structure check and structure repair. This should be done once before Postfix startup. Rename files whose name does not match the message file inode number. This operation is necessary after restoring a mail queue from a different machine, or from backup media. The important thing to keep in mind is that Postfix embeds the inode number in the filename simply to keep the name unique while the message resides on the filesystem. Obviously, this approach breaks when the files are copied to another filesystem. Renaming them appropriately on the new destination ensures no files will be overwritten as the queue is processed or new messages enter the queue. Of course, the scheme I proposed earlier requires that once the backup Postfix is brought up, it must be impossible for the primary to begin resyncing files to the same location on the backup if it becomes active again (or refuses to die a graceful death). Certainly tricky, but it sounds like the use case is to preserve the queue in case of a total failure, just to make sure the mail goes out (even it means it goes out twice). Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: choosing a file system
2009/1/5 Patrick Boutilier bouti...@ednet.ns.ca David Lang wrote: On Sat, 3 Jan 2009, Rob Mueller wrote: But the new Solid-State-Disks seem very promising. They are claimed to give 30x the throughput of a 15k rpm disk. If IO improves by 30 times that should make all these optimizations unnecessary. As my boss used to tell me ... Good hardware always compensates for not-so-good software. What we've found is that the meta-data (eg mailbox.db, seen db's, quota files, cyrus.* files) use WAY more IO than the email data, but only use 1/20th the space. By separating the meta data onto RAID1 10k/15k RPM drives, and the email data onto RAID5/6 7.2k RPM drives, you can get a good balance of space/speed. how do you move the cyrus* files onto other drives? metapartition_files and metapartition-default imapd.conf options in cyrus-imapd 2.3.x So, then, may be we can easily store pure email data on an NFS appliance, keeping metadata on traditionnal filesystem, which can be synced using low level tools Dom David Lang Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: choosing a file system
Thanks for everybody. That was an interesting thread. Nobody seems to use a NetApp appliance, may be due to NFS architecture problems. I believe I'll look to ext4 that seemed to be available in last kernel, and also to Solaris, but we are not enough to support another OS. Dom And Happy New Year ! 2008/12/31 Bron Gondwana br...@fastmail.fm On Tue, Dec 30, 2008 at 02:43:14PM -0700, Shawn Nock wrote: Bron and the fastmail guys could tell you more about reiserfs... we've used RHSuSE/reiserfs/EMC for quite a while and we are very happy. Yeah, sure could :) You can probably find plenty of stuff from me in the archives about our setup - the basic things are: * separate metadata on RAID1 10kRPM (or 15kRPM in the new boxes) drives. * data files on RAID5 big slow drives - data IO isn't a limiting factor * 300Gb slots with 15Gb associated meta drives, like this: /dev/sdb6 14016208 8080360 5935848 58% /mnt/meta6 /dev/sdb7 14016208 8064848 5951360 58% /mnt/meta7 /dev/sdb8 14016208 8498812 5517396 61% /mnt/meta8 /dev/sdd2292959500 248086796 44872704 85% /mnt/data6 /dev/sdd3292959500 242722420 50237080 83% /mnt/data7 /dev/sdd4292959500 248840432 44119068 85% /mnt/data8 as you can see, that balances out pretty nicely. We also store per-user bayes databases on the associated meta drives. We balance our disk usage by moving users between stores when usage reaches 88% on any partition. We get emailed if it goes above 92% and paged if it goes above 95%. Replication. We have multiple slots on each server, and since they are all the same size, we have replication pairs spread pretty randomly around the hosts, so the failure of any one drive unit (SCSI attached SATA) or imap server doesn't significantly overload any one other machine. By using Cyrus replication rather than, say, DRBD, a filesystem corruption should only affect a single partition, which won't take so long to fsck. Moving users is easy - we run a sync_server on the Cyrus master, and just create a custom config directory with symlinks into the tree on the real server and a rewritten piece of mailboxes.db so we can rename them during the move if needed. It's all automatic. We also have a CheckReplication perl module that can be used to compare two ends to make sure everything is the same. It does full per-message flags checks, random sha1 integrity checks, etc. Does require a custom patch to expose the GUID (as DIGEST.SHA1) via IMAP. I lost an entire drive unit on the 26th. It stopped responding. 8 x 1TB drives in it. I tried rebooting everything, then switched the affected stores over to their replicas. Total downtime for those users of about 15 minutes because I tried the reboot first just in case (there's a chance that some messages were delivered and not yet replicated, so it's better not to bring up the replica uncleanly until you're sure there's no other choice) In the end I decided that it wasn't recoverable quickly enough to be viable, so chose new replica pairs for the slots that had been on that drive unit (we keep some empty space on our machines for just this eventuality) and started up another handy little script sync_all_users which runs sync_client -u for every user, then starts the rolling sync_client again at the end. It took about 16 hours to bring everything back to fully replicated again. Bron. -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
choosing a file system
Hello, We are using cyrus-imap for a long time. Our architecture is a SAN from EMC and thanks to our DELL support we are obliged to install redhat. The only option we have is to use ext3fs on rather old kernels. We have 4000 accounts for staff and 2 for students The system is rather fast and reliable. BUT.. Once, there was a bad shutdown corrupting ext3fs and we spent 6 hours on an fsck. Next we discovered that our backup system was going slower and slower. We just pointed out that it was due to fragmentation, and guess what, there's no online defrag tool for ext3. I'm looking for other solutions: ext4fs (does somebody use such filesystem?), xfs zfs (but we should switch to solaris or freebsd and throw away our costly SAN) use a NetApp Appliance (are you using such a device?, NFS seems to be tricky with cyrus..) Thanks for your advice Dom -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: choosing a file system
John, No, that was due to framentation. A fresh copy (one night to copy, then 2 hours to backup, 6 times faster then) solved that problem. There's a filefrag utility, and for some mailboxes, it was over 60%. I have 3 500Mo spools at the moment. And one is left for the copy.. You copy first your data, then you destroy randomly small files and you fill the holes randomly.. Ext4 is said to do delayed allocation, in order to have a decent idea of the file size when writing to disk Dom 2008/12/30 John Madden jmad...@ivytech.edu Once, there was a bad shutdown corrupting ext3fs and we spent 6 hours on an fsck. Next we discovered that our backup system was going slower and slower. We just pointed out that it was due to fragmentation, and guess what, there's no online defrag tool for ext3. Sure it isn't due to the number of files on those filesystems? File-level backups will slow down linearly as the filesystems grow, of course. I solve this by adding more spools (up to 8 at the moment with about 350k mailboxes) so they can be backed up in parallel. All on ext3. John -- John Madden Sr. UNIX Systems Engineer Ivy Tech Community College of Indiana jmad...@ivytech.edu Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html -- Dominique LALOT Ingénieur Systèmes et Réseaux http://annuaire.univmed.fr/showuser?uid=lalot Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus imap badlogin response delay
Thanks to a friend, it just a patch on imap.d where there is a hardcoded sleep(3). Il left a usleep (10) It works better now. Php/imap client is trying three times before leaving.. LALOT Dominique a écrit : Hello, When there is a bad login, the answer is very long. I want to get if faster even if it's worst for security purpose. I've tried to find an answer about that question, buit nothing evident. I'm using cyrus imap 2.2.12. sasl_pwcheck_method: saslauthd sasl_mech_list: PLAIN allowplaintext: yes usr/sbin/saslauthd -m /var/run/saslauthd -a pam -c -t 3600 -s 3000 -n 15 /etc/pam.d/imap auth sufficient /lib/security/pam_cas.so -simap://smtp-etumel.univ-mrs.fr -f/etc/pam_cas.conf authsufficientpam_ldap.so authrequired pam_unix.so nullok nodelay accountsufficient pam_ldap.so account required pam_unix.so nodelay time testsaslauthd -u fred -p pipo 0: NO authentication failed real0m0.056s user0m0.000s sys 0m0.000s It's quich enough. But under cyradm - 5 secondes I saw nothing valuable in imapd.conf. What can I do? Thanks Dom -- Dominique LALOT Ingénieur Système Réseau CISCAM Pole Réseau Université de la Méditerranée http://annuaire.univ-mrs.fr/showuser.php?uid=lalot --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyrus imap badlogin response delay
Hello, When there is a bad login, the answer is very long. I want to get if faster even if it's worst for security purpose. I've tried to find an answer about that question, buit nothing evident. I'm using cyrus imap 2.2.12. sasl_pwcheck_method: saslauthd sasl_mech_list: PLAIN allowplaintext: yes usr/sbin/saslauthd -m /var/run/saslauthd -a pam -c -t 3600 -s 3000 -n 15 /etc/pam.d/imap auth sufficient /lib/security/pam_cas.so -simap://smtp-etumel.univ-mrs.fr -f/etc/pam_cas.conf authsufficientpam_ldap.so authrequired pam_unix.so nullok nodelay accountsufficient pam_ldap.so account required pam_unix.so nodelay time testsaslauthd -u fred -p pipo 0: NO authentication failed real0m0.056s user0m0.000s sys 0m0.000s It's quich enough. But under cyradm - 5 secondes I saw nothing valuable in imapd.conf. What can I do? Thanks Dom -- Dominique LALOT Ingénieur Système Réseau CISCAM Pole Réseau Université de la Méditerranée http://annuaire.univ-mrs.fr/showuser.php?uid=lalot --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
