Re: Multiple SSL certificates
Kjetil Torgrim Homme wrote: On Mon, 2006-02-20 at 10:48 +0600, mitrohin a.s. wrote: On Fri, Feb 17, 2006 at 11:56:00AM -0500, Rosenbaum, Larry M. wrote: How do I configure Cyrus to use multiple SSL certificates/private keys? I want users to be able to reach the server as either a.example.com or a.b.example.com. I have certificates and keys for both names. generate new ssl certificate with subjectAltName = DNS:host1, DNS:host2. when I researched this a couple of years ago, I was unable to find a CA who would sign such a certificate. not that there are many CA to choose from, anyway: there's Thawte/VeriSign and CyberTrust (née beTrusted née Baltimore). if you have an alternative, I'd be happy to know. Thawte have wildcard certificates. *.example.com you can look one of them here - https://secure.datingplace.com/ the CN is *.datingplace.com regards -- ---- - João Assad - ParPerfeito Comunicação LTDA - http://www.parperfeito.com.br/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: High-Availability IMAP server
brad wrote: On Tue, 2005-09-27 at 08:51 +0800, Ow Mun Heng wrote: On Mon, 2005-09-26 at 10:03 -0700, Aaron Glenn wrote: On 9/26/05, David <[EMAIL PROTECTED]> wrote: Is there any way to achieve this goal using cyrus? Which is the best approach to this scenario? Run daily imapsync via cron and a Load Balancer forward the requests to the active one? Any help would be appreciated. There is replication code in the 2.3 branch; though from what I can tell it hasn't been touched in a few months and makes me wonder if it's being actively developed still. Nevertheless, in my exhaustive search for any and all information on IMAP replication, I came across a few list posts detailing the 2.3 replication code in production, without many issues, for over a year. I would be eternally grateful if someone on the list more knowledgeable detailed their experiences with replication. I would be very interested in this solution as well. I would also be interested in replication advice. Thanks, I too am very interested in this replication solution. Where can I get the src and documentation ? Regards, João Assad Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: frequent mupdate master mailboxes.db corruption, anyone else?
Sergio Devojno Bruder wrote: João Assad wrote: Sergio Devojno Bruder wrote: AHA: Sep 21 09:08:49 mupdate mupdate[17026]: IOERROR: mapping /var/lib/imap/mailboxes.db file: Cannot allocate memory Sep 21 09:08:49 mupdate mupdate[17026]: failed to mmap /var/lib/imap/mailboxes.db file I remember Joao Assad had the same problem, no? -- Sergio Devojno Bruder Sorry , I missed the original post. Is the original poster using Fedora or RHEL ? CentOS 3.0 (migrating today to CentOS 4.1 x86-64 bits), ie, RHEL. -- Sergio Devojno Bruder I belive I found a mmap bug on Fedora Core 2 and RHEL. I never got a confirmation that it is indeed a bug, since at the time support for fedora core 2 ended and the fedora devs decided to move the bug to devel. Anyway, I wrote a patch that changes the way cyrus use mmap. Instead of doing a munmap and a new mmap when needed, my patch changes it so it calls mremap instead. It has been working here ever since, never had the corruption again. You can find the patch and somewhat detailed information about the problem in the end of the bugzilla report here - https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=152548 . a copy of the patch is also posted on cyrus bugzilla - http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2640 I hope that helps you. and please review my code because as it's stated in the bugzilla report Im not a very good C programmer ;-) Best regards, João Assad Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: frequent mupdate master mailboxes.db corruption, anyone else?
Sergio Devojno Bruder wrote: Henrique de Moraes Holschuh wrote: On Thu, 15 Sep 2005, Sergio Devojno Bruder wrote: I'm fighting with frequent corruptions of my mupdate master server (high volume, currently 3.9M mailboxes) with Cyrus 2.2.10. First things first: Triple check your system RAM. AHA: Sep 21 09:08:49 mupdate mupdate[17026]: IOERROR: mapping /var/lib/imap/mailboxes.db file: Cannot allocate memory Sep 21 09:08:49 mupdate mupdate[17026]: failed to mmap /var/lib/imap/mailboxes.db file Sep 21 09:08:49 mupdate master[5866]: process 17026 exited, status 75 Sep 21 09:08:49 mupdate master[5866]: service mupdate pid 17026 in READY state: terminated abnormally I remember Joao Assad had the same problem, no? -- Sergio Devojno Bruder Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html Sorry , I missed the original post. Is the original poster using Fedora or RHEL ? Regards -- ---- - João Assad - ParPerfeito Comunicação LTDA - http://www.parperfeito.com.br/ Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
too many open files problem.
Hello guys, cyrus is complaining about too many open files when I try to rename(move between servers) or delete a large number of mailboxes. I have a script that connects to the cyrus master server and issue renames or deletes. My script can be configured to close and reopen the connection after every X renames/deletes but that doesnt seem to help... even if I use 1 different connection for every rename/delete I will get the too many open files error after some time. When moving mailboxes, I get the errors on the backend server Im moving the mailboxes from. cyrus/imap[30082]: IOERROR: creating quota file /imap/conf/quota/H/user.658729.NEW: Too many open files cyrus/imap[30082]: DBERROR: error storing user.658729: cyrusdb error cyrus/imap[30082]: IOERROR: opening quota file /imap/conf/quota/H/user.658729: Too many open files cyrus/imap[30082]: DBERROR: error fetching user.658729: cyrusdb error cyrus/imap[30082]: IOERROR: open on /imap/mboxes/H/user/658729/1.: Too many open files cyrus is started with ulimit -S -n 100 Regards, João Assad --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mupdate exits with signal 11
Derrick J Brashear wrote: auth_unix's purpose is for groups, not for something pam will do for you. technically if you're not using unix groups for access control you don't need any of the group functionality. Really, though, I should figure out what the getgr*_r interface difference which is hitting you is and fix it. How can I get rid of it ? I would love to get rid of it in my production servers so I can finally be sure that I fixed the mmap problem. I cant be sure that I fixed it because mupdate keep dying on me. I can keep testing the getgr*_r stuff in my test enviroment meanwhile. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Derrick J Brashear wrote: is there a munmap for every mmap when I start cyrus, both mupdate master and slave comes up, so I get cyrus/mupdate[2678]: MMAP at map_refresh: mmap(0, 408084480, PROT_READ, MAP_SHARED, 8, 0) = 1098067968 cyrus/mupdate[2679]: MMAP at map_refresh: mmap(0, 408084480, PROT_READ, MAP_SHARED, 8, 0) = 1098067968 The thing that got my attention is , cyrus have been running for a few hours now and every now and then I get this: cyrus/mupdate[2679]: MUNMAP at map_refresh: munmap(1098067968, 408084480) = 0 cyrus/mupdate[2679]: MMAP at map_refresh: mmap(0, 408100864, PROT_READ, MAP_SHARED, 8, 0) = 1718439936 mupdate PID 2679 called something that requested a map_refresh .. which decided to do a munmap and then mmap. That happens quite often, and I dont see anything wrong with that. On the other hand, mupdate PID 2678, after doing that initial map_refresh / mmap never munmaped or mmaped again. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
yet another gdb backtrace of the mmap problem http://www.gazzag.com/gdb.output2.gz regards --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
ctl_mboxlist -u doesnt work on master server
On a cyrus mupdate master server, dump the mailbox list with ctl_mboxlist -d stop cyrus and delete mailboxes.db and db/* , start cyrus to create a new mailboxes.db reimport all the data with ctl_mboxlist -u < dump. all the data will be there, you can even dump it again and compare the dumps but when you try to login as any user and SELECT INBOX, cyrus will complain it cannot find the mailbox partition. also mbpath USER wont return anything.. no error , nada. I've checked the dump format and its correct. eg user.testbackendhost!defaulttestlrswipcda cyrus version is 2.2.12 also repeating the same process on one of the backends works perfectly. It seems to be master related only. Regards --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
got a full gdb backtrace of the mmap crash. Its 400k so you can download it from: http://www.gazzag.com/gdb.output.gz regards --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
mupdate thread died with SIGSEGV
This doesnt seem to be related to the mmap problem The only errors I got on cyrus log when it happened were a bunch of this: Apr 10 16:35:03 cyrus-fe1 cyrus/lmtp[30983]: mupdate-client: connect(10.1.5.101): Connection timed out This time I have strace and gdb logs. Im sending attached the last 1000 lines of the mupdate process strace, the last 1000 of the mupdate thread strace and the gdb trace I managed to extract from the core dump. Note that strace was configured to not log futex calls. gdb.output.gz Description: Binary data strace.proc.output.gz Description: Binary data strace.thread.output.gz Description: Binary data
Re: cyrus-murder problems with database corruption in the frontend/master
Out of curiosity.. everytime cyrus needs to mmap/munmap something. it uses its mmap encapsulation, which is map_refresh and map_free. So every once in a while I get this on strace. 14:27:02.148097 munmap(0x54968000, 373915648) = 0 14:27:02.148765 mmap2(NULL, 373932032, PROT_READ, MAP_SHARED, 8, 0) = 0x54964000 which is a map_free followed by a map_refresh. Now this... 14:27:31.041808 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x54963000 14:27:31.042150 munmap(0x54963000, 4096) = 0 Happens way much more. Its not cyrus doing it directly because its not going through it's mmap encapsulation, so it can only be some lib used by cyrus. What is it ? --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Henrique de Moraes Holschuh wrote: Just thought of something. Please set the vm.overcommit_memory syscall to 2 (it is available in /proc/sys, I think. But the right way is to use /etc/sysctl.conf and sysctl). Make *really* sure you have enough swap when you do that. You will *really* need it. Some look on /proc/meminfo (especially on CommitLimit and Committed_AS) might shed some light if that is the problem. Yeah I thought that might help too, I did that yesterday after the last corruption I also set the overcommit_ratio to 100 So far everything is running smooth. but I also recuced the mupdate_workers configs, which usually makes cyrus last longer. Im getting anxious here, I already have 643M worth of straces since I started it. So far no ENOMEMs . My current meminfo MemTotal: 2074268 kB MemFree:277240 kB Buffers: 93244 kB Cached:1437800 kB SwapCached: 3216 kB Active:1247724 kB Inactive: 402988 kB HighTotal: 1178756 kB HighFree: 153984 kB LowTotal: 895512 kB LowFree:123256 kB SwapTotal: 2096472 kB SwapFree: 2093068 kB Dirty:1448 kB Writeback: 0 kB Mapped: 136136 kB Slab:46308 kB CommitLimit: 4170740 kB Committed_AS: 1494736 kB PageTables: 90924 kB VmallocTotal: 114680 kB VmallocUsed: 2072 kB VmallocChunk: 112208 kB HugePages_Total: 0 HugePages_Free: 0 Hugepagesize: 2048 kB --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Derrick J Brashear wrote: So, prior to this presumably you've mmap2()'d some memory, have there been any munmaps for these mmap2s 00:25:58.583761 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68e8000 00:25:58.585461 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68e3000 00:25:58.586673 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68df000 00:25:58.588172 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68db000 00:25:58.589402 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68d7000 00:25:58.592491 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 00:25:58.593194 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 00:25:58.594149 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 00:25:58.594661 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 00:25:58.595493 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 00:25:58.596004 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb68cc000 I have the corresponding munmaps. 00:25:58.584008 munmap(0xb68e8000, 4096) = 0 00:25:58.585683 munmap(0xb68e3000, 4096) = 0 00:25:58.586896 munmap(0xb68df000, 4096) = 0 00:25:58.588394 munmap(0xb68db000, 4096) = 0 00:25:58.589621 munmap(0xb68d7000, 4096) = 0 00:25:58.592712 munmap(0xb68cc000, 4096) = 0 00:25:58.593409 munmap(0xb68cc000, 4096) = 0 00:25:58.594363 munmap(0xb68cc000, 4096) = 0 00:25:58.594898 munmap(0xb68cc000, 4096) = 0 00:25:58.595706 munmap(0xb68cc000, 4096) = 0 00:25:58.596225 munmap(0xb68cc000, 4096) = 0 Im generating full traces now separated by threads, with each thread logging in a different file. So I belive we will be able to nail down exactly what the hell is going on. I've also changed mupdate_workers options to the following values: mupdate_workers_start: 50 mupdate_workers_minspare: 50 mupdate_workers_maxspare: 50 mupdate_workers_max: 50 to make sure I dont end up with 10s strace files to look into. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Derrick J Brashear wrote:
I've noticed that once I get the mmap error, the system will still
run without spitting db errors for anywhere from a few mins to a few
hours. Also, I never get more than one mmap error before the db
becomes unnusable.
that's not shocking. i'd still like to know why mmap fails, but,
failing that, and since you're already getting corruption, can you in
mupdate.c replace fatal with a function which does:
void fatal(const char *s, int code)
{
assert(0);
exit(code); /* shut up GCC */
}
and then give us a backtrace from the core which you will then get?
After doing that, the mupdate process now exits with signal 11 as expected.
OTOH the core isnt getting dumped to disk for some reason...
ulimit -c unlimited didnt work.
cyrus-fe1 cyrus/mupdate[32003]: IOERROR: mapping
/var/lib/imap/mailboxes.db file: Cannot allocate memory
cyrus-fe1 cyrus/master[32000]: process 32003 exited, signaled to death
by 11
cyrus-fe1 cyrus/master[32000]: service mupdate pid 32003 in READY state:
terminated abnormally
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
cyrus/mupdate[12614]: IOERROR: mapping /var/lib/imap/mailboxes.db file: Cannot allocate memory cyrus/mupdate[12614]: failed to mmap /var/lib/imap/mailboxes.db file cyrus/master[12580]: service mupdate pid 12614 in READY state: terminated abnormally Henrique de Moraes Holschuh wrote: Check ulimits. If fixing the ulimit stuff stops the problems, we know that the issue is caused by something that mmaps or mallocs and does not check the result properly. Derrick J Brashear wrote: Resource limited memory, or are you really running out of memory? Letting processes continue running in the face of an mmap failure needs to be re-examined I guess. The system have plenty of RAM available and ulimit -a reports the max virtual memory as unlimited core file size(blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files(-n) 20 pipe size (512 bytes, -p) 8 stack size(kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes(-u) 32755 virtual memory(kbytes, -v) unlimited --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
João Assad wrote: I needed to restart cyrus master and got this error minutes after doing it. cyrus/mupdate[12614]: IOERROR: mapping /var/lib/imap/mailboxes.db file: Cannot allocate memory cyrus/mupdate[12614]: failed to mmap /var/lib/imap/mailboxes.db file cyrus/master[12580]: service mupdate pid 12614 in READY state: terminated abnormally And now, like 20 mins later my mailbox database is corrupted again. cyrus-fe1 cyrus/mupdate[14517]: IOERROR: mapping /var/lib/imap/mailboxes.db file: Cannot allocate memory cyrus/mupdate[14517]: failed to mmap /var/lib/imap/mailboxes.db file cyrus/master[13313]: service mupdate pid 14517 in READY state: terminated abnormally and a few hundred log lines later, the usual cyrus/mupdate[14975]: DBERROR: skiplist recovery /var/lib/imap/mailboxes.db: 14BA9FFC should be ADD or DELETE cyrus/mupdate[14975]: DBERROR: error updating database user.733844: cyrusdb error I just noticed I had the same IOERROR before the last corruption, I didnt notice it before because it seems to be always some 100s lines of log before the DBERROR the new corrupted mailbox db will be available shortly at http://www.gazzag.com/mailboxes2.db.gz Regards João Assad --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Derrick J Brashear wrote: On Thu, 31 Mar 2005, João Assad wrote: My db just got corrupted again 2 hours ago. seems like moving 200k mailboxes between backends really speed it up. I have my corrupted mailboxes.db and I can send it to anyone interested in taking a look. how about a url for it? there you go - http://www.gazzag.com/mailboxes.db.gz --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus-murder problems with database corruption in the frontend/master
Henrique de Moraes Holschuh wrote: On Thu, 31 Mar 2005, Sergio Devojno Bruder wrote: We've done 2 murder instalations, cyrus 2.2.3 and cyrus 2.2.10, and already have seen the same type of corruption. (should ADD or DELETE), but not with the same frequency. Are you guys running the same distribution or the same kernel? It looks like searching for common points between the two installations would give some hints on where the problem lies. My db just got corrupted again 2 hours ago. seems like moving 200k mailboxes between backends really speed it up. I have my corrupted mailboxes.db and I can send it to anyone interested in taking a look. fedora core 2 kernel-smp-2.6.10-1.14_FC2 Untill Tuesday: cyrus-imapd-2.2.10-3.fc2 cyrus-imapd-murder-2.2.10-3.fc2 After Tuesday I made a custom 2.2.12 source installation --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
