Re: SMS notification in a cyrus-imapd environment
On Wed, 27 Jul 2005 09:52:22 +0200 (CEST) "Simon Matter" <[EMAIL PROTECTED]> wrote: > I'm interested to hear how people implemented SMS text notification in a > cyrus-imapd environment. I'm keeping it as simple as possible with freely available tools. Webmail user has a checkbox if he wants to forward his mail to sms. That generates a sieve rule to redirect his mail to [EMAIL PROTECTED] MTA (postfix) notices the domain and routes the mail via dedicated transport to a mail2sms script, that strips it down and takes care of the strane sms chartable and then forwards the msg to our local mobile operater via their ssl channel. All this has some added cruft (which mails to forward and when, billing, user <-> phonenum mapping, ...) but works ok. -- Jure Pečar http://jure.pecar.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyrus quota beavior
On Fri, 01 Jul 2005 09:37:40 -0400 Ken Murchison <[EMAIL PROTECTED]> wrote: > Christian Anton wrote: > > > The problem: It is not anymore possible to delete this mail because most > > of my users use IMAP or squirrelmail and when they "delete" a message > > it ist moved to "Trash"-Folder. Moving a message is not possible > > because the mailbox is OVER quota. > > > > Is it possible to make cyrus not only reject mails when the quota is > > already reached but rejecting them if they would not fit in the > > mailbox? > > Not without a patch, but it shouldn't be too difficult. This would be a > good feature candidate that you should add to bugzilla. I'd suggest patching your webmail instead to directly mark mail as deleted and then expunge it in case it cannot copy it to trash, maybe with a warning to the user about that. I know that at least JAWMail is working that way. -- Jure Pečar http://jure.pecar.org/ --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Preserving seen state when converting from mbox
On Thu, 25 Sep 2003 15:04:48 -0400 (EDT) Igor Brezac <[EMAIL PROTECTED]> wrote: > I do not think you have enough time to transfer several gigs (if not more) > of email in just a few hours over IMAP protocol. Just for the reference, it took us about two weeks to transfer about 300gb of mailspool from one live system to another (with perdition in front), on a reasonably fast hw, with custom php scripts doing imap2imap copy (and related user imgration) in parallel. -- Jure Pecar
Re: Sql based Spamassassin with sieve
On Mon, 15 Sep 2003 09:24:41 -0600 Darron Froese <[EMAIL PROTECTED]> wrote: > On Monday, September 15, 2003, at 01:53 AM, Joakim Ryden wrote: > > >>> How about this? > >>> http://au2.spamassassin.org/full/2.5x/dist/sql/README > >>> > >>> works for me. > >>> -- > >>> darron froese > >> I started from that page too. My actual problem is How Do I tell > >cyrus> ( or sieve ) to run spamassassin with the user name to whom the > >mail> is addressed I didn't follow this thread from the beginning, but the solution is very simple to me. In the contrib directory of spamassassin (i think) is a script spamcheck.py, which is a simple pyton script that takes mail as an input, checks it with the spamd and delivers it via lmtp. In postfix 2.x you can set peruser transport maps, if you dont want to set this script up as a default mailbox transport. It has been working for me perfectly for about a year (yes with peruser sql prefs), now i've moved on to amavisd-new ... -- Jure Pecar
Re: database recovery...
On Wed, 10 Sep 2003 10:48:34 -0400 (EDT) Igor Brezac <[EMAIL PROTECTED]> wrote: > My installation of cyrus 2.2-CVS on Solaris 9 shows the same memory > footprint per process. 20M is taken by mmap(), the rest is shared > physical memory and about 1M of 'private' physical memory per process > (imapd, pop3d, lmtpd, etc). 2.2.1 looks OK on linux ... process sizes are around 1.5mb. -- Jure Pecar
Re: Cyrus Server Migration
On Thu, 4 Sep 2003 10:01:33 -0500 "Andrew Brink" <[EMAIL PROTECTED]> wrote: > 1. ext2 or ext3, we are also considering a cluster solution using a > shared File system (GFS) Any experiences with these? reiserfs or xfs. shared fs ... it's probably less painfull to just have shared storage that is mounted only from one box at a time. > 2. db3 or db4? The previous server we have has some of the famous > locker problems and other db4 problems (we believe). Rob, maybe you can > bless the "official" db version to use. I am fine using db4, provided > that people are successfully using it in production. db3; i heard that latest versions have solved issues with db4. -- Jure Pecar
Re: cyrus accounts and monitoring activity/usage
On Mon, 25 Aug 2003 11:15:52 -0700 (PDT) joe ritter <[EMAIL PROTECTED]> wrote: > I am interested in looking at stats such as users that > have never logged in, users that have accessed their > account in the last 30 days, 60 days and 90 days etc. We're using auxprop with mysql auth and it was trivial to add a couple more lines into sasl to do another sql query upon auth attempt to update last_seen field with now(). Then it is trivial to rank users among those that visit daily, weekly, monthly or only once in a while ... I still have to take some time and develop this further to run separate queries on successful login and on failed login ... -- Jure Pecar
Re: robustness ...
On Fri, 1 Aug 2003 09:59:30 -0400 (EDT) Rob Siemborski <[EMAIL PROTECTED]> wrote: > Of course, once you're running on a corrupted file system, all bets are > off. Any number of things could be wrong: the binaries could have been > damaged, files may have been reassembled incorrectly, or even be missing > entirely. Corrupted binaries are easy to detect and replace; corrupted data is not. But lets limit the problem scope to the cases, when cyrus starts processing certain file and hits some unexpected stuff. One such case i reported about two months ago, looping on the broken skiplist files. > Cyrus does go to great lengths to defend itself against crashes during > transactional operations (so that data isn't partially committed if the > system crashes), but defending against general filesystem corruption is > an entirely different animal. > > Given the amount of memory mapping involved in Cyrus, asking it to > successfully operate given a corrupt filesystem is sort of like telling > any program to operate in the face of unreliable main memory. Sure, if > you're being very very careful you may be able to get some semblance of > correct behavior, but you'll also take huge performance hit in the common > (un-corrupted) case, and you still may not be able to survive in the > corruption case. I agree. However, lets just try to get the known problems sorted out, before even thinking of a general fit-all-crashes-and-corruptions solution. > Cyrus does provide tools to help recover from filesystem crashes. These > include the database recovery utilities, the chk_cyrus utility (which we > wrote after a severe filesystem crash of our own!), reconstruct, and so > on. chk_cyrus is a nice addition, indeed. I cooked up a bunch of shell scripts to do something similliar, parsing the raw mailboxes dump output and looking for directories in the fs. > Should we strive to do better? Probably, but when faced with a decision > of whether to track down a problem in Cyrus during normal operation, or > track down a problem in Cyrus in the face of filesystem corruption, I'm > going to have to pick the former almost every time, since it has wider > applicability and there already exist tools to return the Cyrus data store > back to a consistent state. Fully agree. Well, the reason i wrote this mail in the first place could easily be labeled as a problem during normal operation :) > In any case, if you are so worried about resilience in your software, why > are you using alpha quality software on your production system? Mostly for these two reasons: a) features b) testing Virtdomains are a much needed feature, so i got a cvs snapshot running on my personal sever as soon as Ken commited them into cvs. It worked well enough, so i went on and got 2.2a in production. It worked excellent untill hardware bit me, and only then some problems came up, which Ken fixed promptly (thanks again, Ken). You see, i'm still young enough to afford a sleepless week or two every now and then. And if i can help to spare someone else such expirience, i'd gladly do so. Since i'm not a programmer myself, just a poor sysadmin, i do the way i can: to whine about all the unusuall problems i notice :) -- Jure Pecar
robustness ...
I was running a cvs snapshot of 2.2 from august 2002 for my personal mail till now, when i upgraded to 2.2.1. A folder for info-cyrus developed some strange behaviour a couple of months ago: some clients would see a bounch of mail that was no longer there (sylpheed), some clients would see it ok (sylpheed-claws), some clients would not display it at al (jawmail webmail). It somehow became corrupted in some subtle way. (reconstruct was not functional in that snapshot) So on to upgrade ... pretty straightforward, much less fun that i expected. First lets see what's up with the info-cyrus folder: looks ok, i try to delete the last mail that was delivered into it, imapd crashes: Program received signal SIGSEGV, Segmentation fault. mailbox_expunge (mailbox=0x8119be0, iscurrentdir=1, decideproc=0, deciderock=0x0) at mailbox.c:1884 1884cacheitem = CACHE_ITEM_NEXT(cacheitem); (gdb) bt #0 mailbox_expunge (mailbox=0x8119be0, iscurrentdir=1, decideproc=0, #deciderock=0x0) at mailbox.c:1884 1 0x08054944 in cmd_expunge #(tag=0x8133598 "53", sequence=0x0) at imapd.c:3354 2 0x0804ed87 in #cmdloop () at imapd.c:907 3 0x0804dfdb in service_main (argc=1, #argv=0x8128ed0, envp=0xbb5c) at imapd.c:569 4 0x0804cece in main #(argc=1, argv=0xbb54, envp=0xbb5c) at service.c:514 5 0x401da627 #in __libc_start_main (main=0x804c6a4 , argc=1, ubp_av=0xbb54, #init=0x804b354 <_init>, fini=0x80920a0 <_fini>, rtld_fini=0x4000dcd4 <_dl_fini>, stack_end=0xbb4c) at ../sysdeps/generic/libc-start.c:129 Luckily, reconstruct -r fixed the problems. Why am i writing all this? I still think that imapd should not crash or do other unreasonable things (like looping with 100% cpu consumption) when confronted with corrupted files. I had many interesting expiriences on our production system, where fs badly crashed. IMHO Cyrus still needs some work in the robustness area. If only i'd know more about programming ... -- Jure Pecar
Re: Cyrus 2.2.1-BETA Released
On Wed, 30 Jul 2003 17:02:08 -0400 (EDT) Rob Siemborski <[EMAIL PROTECTED]> wrote: > You want to get the copy from CVS though, not the included one, otherwise > you'll lose all of your current sript activations (though not the scripts > themselves). > > -Rob Thanks for the tip ... ... but either i'm stupid (must be the hour ..) or your cvs is seriously messed up ... i just pulled the cyrus HEAD branch to find out it has no masssievec, not even sievec! They are not visible through cvsweb, i tried HEAD, MAIN, cyrus-release-2-2-1. Also install-upgrade docs looked kinda old. Then i pulled 2.2.1 tarball and it was all there ... how come? Anyway it's gonna be a nice long night (ehm, morning) before i get this mailbox running on 2.2.1 ... :) -- Jure Pecar
Re: Cyrus 2.2.1-BETA Released
On Thu, 17 Jul 2003 11:20:20 -0400 (EDT) Rob Siemborski <[EMAIL PROTECTED]> wrote: > Additionally, the > sieve bytecode now is stored in network byte order and is therefore > portable across architectures. What does this mean to us that run 2.2a in production? :) Is this backward compatible / will be converted on the fly or would we have to recommit every sieve script? -- Jure Pecar
Re: ctl_cyrusdb looping
On Tue, 17 Jun 2003 10:54:06 -0400 (EDT) Rob Siemborski <[EMAIL PROTECTED]> wrote: > Most likely it was processing your duplicate delivery database, which > can be quite large and take some time to process (you can generally tell > what is going on by truss/strace on the process). > > Options you have are to just delete it or to wait. Killing the process > in the middle of recovery is probably not ideal. > > If you're using Berkeley DB for your duplicate delivery database, you > many want to look at increasing your checkpoint frequency. This could > help reduce the amount of log that needs to be played back during > recovery. > > -Rob I'm seeing the same on my 2.2.0a here ... deliverdb is now at 826mb, i have checkpoint event set with period=10 and i find one or two ctl_deliver processes running, eating all the cputime available. strace shows it's chewing the db files as it should ... Maybe i should experiment with -E 2 or even 1 ... What are the consequences of removing deliver.db? As i understand, nothing critical. Killing ctl_deliver usualy results in a lmtp hang and cyrus restart is needed to recover. I think it would be smart for ctl_deliver to check if some other ctl_deliver process is already running ... -- Jure Pecar
Re: Corruption?
On 12 Jun 2003 11:17:15 -0700 Patrick Nelson <[EMAIL PROTECTED]> wrote: > Had my Cyrus server lockup... not cyrus related... but when I rebooted > the server my users could not get their mailbox contents. > > The error thats listed from Evolution when trying to view the, say INBOX > is: > > Error while 'Opening folder imap://@/INBOX': No > such folder INBOX > > Looking at the /var/log/messages I see that the user authenticated and > then I see an error like: > > master[]: process exited, signaled > to death by 7 > > How can I fix this? > > If you had a harddisk crash, check your binaries. Dump your mailboxes.db into plaintext format and see what you got there. Fix, undump back. (It's handy to have some other clean cyrus setup to compare the plaintext format; be sure not to forget the \t at the end of each line) Check other database files if they contain some weird data (strings should reveal if any fs blocks were strangely relinked by fsck as you should spot parts of mails & other junk in files where they should not be) When you get the imap login done and imapd dies at the point of . select inbox, the usual culprit is the corrupted seen file. Delete it for each user that has problems. Good luck, -- Jure Pecar
Re: more fun ...
On Mon, 2 Jun 2003 22:49:29 +0200 Jure Pecar <[EMAIL PROTECTED]> wrote: > It looks like i'll have to dump mailboxes.db to a flatfile, fix all the > acl errors and reimport it tonight ... after rebuild & fix up, now it's no longer [EMAIL PROTECTED], it's [EMAIL PROTECTED] that's showing up ... # telnet mailstore 143 Trying ... Connected to mailstore. Escape character is '^]'. * OK mailstore.domain.com Cyrus IMAP4 v2.2.0-ALPHA server ready . login [EMAIL PROTECTED] . OK User logged in . create user/[EMAIL PROTECTED] . OK Completed . create [EMAIL PROTECTED] . OK Completed . getacl user/[EMAIL PROTECTED] * ACL user/[EMAIL PROTECTED] [EMAIL PROTECTED] lrswipcda [EMAIL PROTECTED] lrswipcda . OK Completed . getacl [EMAIL PROTECTED] * ACL [EMAIL PROTECTED] anyone lrs any explanation here? to me it looks like somethign is messed up somewhere in the unixhiersep/vdomains area ... but still, i'm pretty sure this worked ok before ... -- Jure Pecar
more fun ...
While still recovering from the fs corruption, i've hit some more weird
things in 2.2a.
Consider this:
mailstore.domain.com> cm user/[EMAIL PROTECTED]
mailstore.domain.com> lam user/[EMAIL PROTECTED]
[EMAIL PROTECTED] lrswipcda
[EMAIL PROTECTED] lrswipcda
like ... WTF? where do these ACLs come from? I wouldn't object much to cyrus
admin user set like this, but the second line ...
Anyway, i've stepped through mailbox creation with gdb and this is where
things start to look wrong first:
mboxlist_mycreatemailboxcheck (name=0xbffeb110 "domain.com!user.notyethere",
new_mbtype=0, partition=0x0, isadmin=1, userid=0x8136848 "[EMAIL PROTECTED]",
auth_state=0x8137720, newacl=0xbffe9e54, newpartition=0xbffe9e58, RMW=1,
localonly=0, force_user_create=0, tid=0xbffe9e5c) at mboxlist.c:350
350 switch (r) {
375 parentlen = 0;
376 while ((parentlen==0) && (p = strrchr(parent, '.')) && !strchr(p,
'!')) {
398 }
377 *p = '\0';
379 r = mboxlist_mylookup(parent, NULL, NULL, &parentpartition,
mboxlist_mylookup (name=0xbffe9bf0 "domain.com!user", typep=0x0, pathp=0x0,
partp=0xbffe9bdc, aclp=0xbffe9be0,
tid=0xbffe9e5c, wrlock=0) at mboxlist.c:162
I'm sure this was functioning perfectly well before the crash.
The only major thing i did that i imagine could affect this in some way was
a rebuild of a mailboxes.db from a filesystem using a variant of the shell
script found on cyrus-utils.sf.net ...
It looks like i'll have to dump mailboxes.db to a flatfile, fix all the acl
errors and reimport it tonight ...
--
Jure Pecar
Re: wishlist for 2.2 final
On Thu, 29 May 2003 09:07:42 -0400 (EDT) Rob Siemborski <[EMAIL PROTECTED]> wrote: > Uh, this has been possible for quite some time (like, since the 2.0 > series). > > e.x. sieveshell -u otheruser -a adminuser server uhm, in timsieved too? then i must be doing something wrong. will take another look. > > anything else on the CMU's list before 2.2? and, any ETA for 2.2? :) > > ANNOTATEMORE support. Possibly BINARY support. When its ready. nice. -- Jure Pecar
Re: wishlist for 2.2 final
On Wed, 28 May 2003 23:25:50 -0400 (EDT) Igor Brezac <[EMAIL PROTECTED]> wrote: > Do you mean db4 (4.1.25)? If so, yes. I am testing the patch as we > speak, but I believe it will solve the lockers issue and recovery problems > after cyrus restart. This should also fix issues with berkeley > mailboxes.db some folks have reported. Although, people should use > skiplist for mailboxes.db. The patches will be in the cvs soon. excellent. i might have another one for the 'reliability' section: * if skiplist seen state file is corrupted (content mixed up with some other file), imapd tends to go in a loop consuming 100% cpu and doing nothing (at least strace shows nothing). reconstruct didnt fix this (should it?). -- Jure Pecar
wishlist for 2.2 final
OK, after running 2.2a in production for two months and just finished recovering from an ugly disk crash & corruption, here are some of my whishes for 2.2 final: * figure out db3 lockers. i see some work here by igor brezac; igor, does your patch solve the issue entirely? * option (something like -v) for ctl_* and reconstruct tools to be way more verbose; when admin feels that some tool is misbehaving, an option like this could save hours of figuring out what's going on. * timsieved to allow admin user to set users' scripts. i think this was mentioned some time ago already. anything else on the CMU's list before 2.2? and, any ETA for 2.2? :) after i get our system's hardware in order i'll take a look at making reconstruct.c more verbose ... -- Jure Pecar
Re: sieve in 2.2a
On Sun, 6 Apr 2003 03:51:12 +0200 Jure Pecar <[EMAIL PROTECTED]> wrote: > Any hints how to debug this further? Ok i found out what is going on here, it's the fulldirhash that's messed somewhere in the lmtp. See bug #2005 in bugzilla for details. PS. I'm willing to shorten someone's amazon wishlist (or equivalnet) if this gets fixed up by yesterday ;) -- Jure Pecar
sieve in 2.2a
Hi all, What's the sieve status in the 2.2a? Is it supposed to work? I have a simple script: redirect "[EMAIL PROTECTED]"; keep; that looks fine if run by the sieve test app. However, i never see the mail actually hitting the maillog; looks like sieve does not call sendmail at all. I have sendmail set in imapd.conf that points to the sendmail binary. With debug logs on, i see sieve: executed, sieve accepted connection, sieve login and sieve: entered bc_action_emit with filelen: 16 and thats about it. Any hints how to debug this further? -- Jure Pecar
Re: postfix doesn't know cyrus user
On Fri, 4 Apr 2003 16:32:39 +0200 "Maik Knittel" <[EMAIL PROTECTED]> wrote: > hello, > > after we've got cyrus running, we have another problem. > We can't send mail with postfix (smtp). It seems that > Postfix doesn't know the cyrus user. you have to set up the local_recipient_table map that lists all users that you want to accept mail for. how you do that greatly depends on your setup. -- Jure Pecar
timesieved && cyrus admin user
Hi all, As cyrus admin user i can login to imap and manage mailboxes, ACLs & stuff. Can i do the same for sieve and timsieved? During a migration from our old setup to new i would also like to migrate our legacy forwarding & stuff to sieve, but have yet to figure out how. I'm using fulldirhash so finding out where exaclty the user's sieve dir is on disk and planting his script there is not really an option. -- Jure Pecar
Re: interesting limitation
On Mon, 31 Mar 2003 10:42:39 -0500 (EST) Dave O <[EMAIL PROTECTED]> wrote: > > 2 level hashing would work, but I don't know if Cyrus supports that. It > would most likely be trivial to implement. > > eg spool/s/sm/user/smith Yes, i was thinking about that too ... In fact i would prefer it over fulldir hash code, because this way i always know where on disk the user's mailbox is. -- Jure Pecar
interesting limitation
Hi all, Recently i was testing a 2.2 branch on linux with Veritas vxfs. I wanted to create 20 users in the form of userN, where n is 1..20. I soon found out that vxfs won't let me create more than 32k subdirs in one dir. This is clearly a limitation of the filesystem. How does other filesystems handle this? The solution here is full dir hash. But, the next limit is at 26*32k users. Is anyone actually nearing this number of users on a single box? Probably not, but who knows what the future may bring ... -- Jure Pecar
Re: cyrus 2.2 from yesterday cvs
On Sun, 23 Feb 2003 15:55:14 -0500 (EST) Igor Brezac <[EMAIL PROTECTED]> wrote: > Do you get a syslog entry for each failed authentication from > saslauthd: AUTHFAIL: [EMAIL PROTECTED] service=pop3...? If this is > the case, things are working fine. no, i get AUTHFAIL: user=test service=imap realm= [internal error ] when using [EMAIL PROTECTED] as username. -- Jure Pecar
Re: cyrus 2.2 from yesterday cvs
I hit another problem: I have a mysql plugin for saslauthd, which i put together (and posted here about half a year ago). Among the first lines of auth_mysql() i print out the values i get for login and realm. If i test it with testsaslauthd, i get [EMAIL PROTECTED] as the login and empty realm, which i then split internally and insert into mysql query i read from conf file. It works perfectly. However, using imtest or telnet to pop3/imap ports, i see that i get test as login and empty realm. So something between imapd and saslauthd eats away the @domain part. This only happens if enable virtdomains in imapd.conf. If i turn it off, i get @domain part in the login as expected. Ken? -- Jure Pecar
cyrus 2.2 from yesterday cvs
Hi all, I started playing with the recent cvs code. Basically i want to set up a multidomain server with one default domain and mysql auxprop auth. So i have this in imapd.conf (among other things): admins: cyrus allowplaintext: yes defaultdomain: example.com hashimapspool: 1 imapidresponse: 1 loginrealms: example.com unixhirearcysep: 1 virtdomains: 1 sasl_pwcheck_method: auxprop sasl_mysql_statement: select password_plain from users where username='%u' and realm='%r' sasl_mysql_verbose: yes Here are my current findings: cyradm still does not compile to a working state outofthebox. Looks like env variables like BDB_LIB and SASL_LIB are not passed down to perl/imap. When exporting them by hand and going through standard 'perl Makefile.PL; make; make install' it works. There's a comment at the top of sasl*/plugins/mysql.c which states that ' %r is the realm which could be the kerbros realm, the FQDN of the computer the sasl app is on or what ever is after the @ on a username'. Looking at the code it looks like 'whatever is after @ on a username' take precedence over server FQDN (line 276 of plugins/mysql.c in sasl-2.1.12). Now, if i try to log in with imtest, it looks like it does not care what is after @ at all. Regardless if i set -a [EMAIL PROTECTED] or -u [EMAIL PROTECTED] or both, i get such log entries: mysql plugin doing query select password_plain from users where username='cyrus' and realm='myhost.example.com'. Same goes for IMAP logins. I have a default domain set in imapd.conf. I created its ondisk hirearchy with mkimap -d example.com and expect its mailboxes to appear underneath. However, mailboxes under default domain appear at the top level, as if virtdomains is off. (is this intentional? i havent run mkimap without -d yet) If i want to set quota for them i get System I/O error (the relevant config/quota directories are not present). If i create another domain, blah.net, and create some mailboxes under it, they appear properly and i can set their quota. Now it's entirely possible that i'm doing something wrong. In this case, i'd like to know what :) My environment is rehdat 7.3 with all updates. -- Jure Pecar
Re: imapd's hang when maxchild count is reached
On Wed, 05 Feb 2003 15:21:26 -0500 Scott Adkins <[EMAIL PROTECTED]> wrote: > I solved this problem a long time ago by passing an environment variable > from the master process to the child process when the child process is > spawned indicating that the server is full. I used CYRUS_MAXCHILD, and > the child process already checks for the CYRUS_VERBOSE variable when it > starts in order to properly set the debugging level. If that variable > was set, then the child would output an error message indicating that the > server was full and to try again later. It would then close the client > connection and then exit. > > A couple things to note: > > 1) I prefer the client to be notified when the server is full and not > simply get connection refused messages or have the email client just > sit there, appearing to hang, while the server waits for a connection > to become free. > > 2) My method worked, but it didn't take advantage of the process reuse > feature. Basically, the master process only gets one chance to pass > an environment variable off to the child process. So, once that > variable is set in the child, the child will always believe that the > max has been reached. That is why I had the child process go away, > as it is basically useless after handling that one connection. > > 3) If there was a good way for the master to notify the child process > on each connection pass (either when passing the connection to an > already available child, or when passing the connectioin to a newly > spawn child) what the current status of maxchild is, then it would > be quite efficient to send the server full messages, close the > connection and wait for the master process to hand it another. I > don't know enough about how the master and child process communicate > to know how to make this work. > > Scott Now that you're discussing the mechanisms to refuse new connections politely because of some condition, i would like to suggest another condition to check: system loadavg. Some programs (sendmail for example) know how to reject connections if loadavg is >= some admin defined value. -- Jure Pecar
Re: Latest on Cyrus and GFS?
On Friday 17 January 2003 21:01, Jared Watkins wrote: > I've not been able to find the answer to the GFS question in the archives.. > so here goes. > > If one were using a dual attach scsi cabinet.. or fibre channel.. would it > possible to use cyrus on GFS from two or more servers all with r/w access? well, as cyrus is primarily i/o hungry (as any other mail app) there's no real benefit of having multiple machines doing i/o on the same storage ... the most such setup can give you is more weird problems. i'm doing one of these 'shared' storage setups these days too. the solution i chose is kimberlite, well, RH advanced server actually, where i have only one box using the storage at a time. the other box can still be usefull, for example chewing mails with spamassassin. in case there is some problem with either of the boxen, the healty one can take over the service. -- Jure Pecar
Re: cyrus 2.2 status
On Thu, 12 Dec 2002 20:31:41 -0500 Ken Murchison <[EMAIL PROTECTED]> wrote: > I addition to what Rob already mentioned, there needs to be more work > done on documenting the virtdomain support and tying some loose ends. Yes, virtdomains are actually the #1 thing i'm interested in cyrus 2.2 ... I'm sure there are more people interested, so i think it would be nice to provide either a stable, known working cvs branch of 2.2 or a patch with a backport of virtdomains stuff to 2.1. I'm willing to help here, just give me some directions. -- Jure Pecar
cyrus 2.2 status
Hi all, what is the current status of the cyrus 2.2 cvs branch? judging by the cvs commits lately, there are just various little cleanups here and there ... is there anything big left on the TODO list for 2.2? my little wish would be the sql cyrusdb interface, discussed here a week or two ago, even if marked exeprimental or something. -- Jure Pecar
cyrus on vxfs
Hi all, I'm sure some of you use vxfs on solaris for cyrus mail spool ... What are your expiriences? Tunning tips? Maybe any vxfs on linux expiriences out there? Thanks, -- Jure Pecar
Re: PostgreSQL backend: a waste of time?
On Mon, 25 Nov 2002 10:04:03 -0600 [EMAIL PROTECTED] wrote: > Seems kinda ironic in a way---doesn't MySQL use BerkeleyDB? I guess > it's all in the indexing/caching yes, it is one of the choices for table type. others are MyISAM and InnoDB, the latter supports transactions and is as fast as oracle if not a bit faster. -- Jure Pecar
Re: Best way to backup cyrus system (lots of inodes!)
On Thu, 21 Nov 2002 13:04:22 -0500 Steve Barber <[EMAIL PROTECTED]> wrote: > Is anyone successfully implementing any sort of incremental backup > solution in a large production environment? Other than filesystem snapshots (if your underlying volume manager knows about it) i don't see some very usefull/efficient backuping tools for large mail spools. For example, i still rely only on raid mirrors :) which leaves me cold in case of rm -rf / (or some such) but safe in case a disk deceides to die. Puting all the mail spools in the RDBMS just for the ease of backuping ... i heard from the Oracle people that their DB is already 30% slower than plain fs and cyrus is already very i/o demanding ... i don't want to introduce another bottleneck. Altough, implementing all the little databases cyrus has (mailboxes, duplicate deliveries, ...) in some RDBMS might be usefull for large setups. Which still leaves the backuping problem intact ... -- Jure Pecar
max users per cyrus instance
With all the talk about the distributed filesystems, I have a related question: How many users do you have per cyrus instance? My expirience from our webmail setup is 150k users on a single box with 10 10k rpm scsi disks. CPU load is hovering around 5% (4 500mhz p3 xeons), loadavg around 10 most of the day. Clearly i/o wait is the cause here. I'm planning to expand onto some nice fibrechannel disk arrays with 15 disks and want to know if one cyrus instance would hold up to 600k users. Does anyone have such numbers in production? What combination of cyrus mailstore hashing and filesystems offered by linux today would be most interesting in such scenario? -- Jure Pecar msg08775/pgp0.pgp Description: PGP signature
Re: Unexplained heavy loads lately on Cyrus servers
On Wed, 02 Oct 2002 16:34:31 -0400 twk <[EMAIL PROTECTED]> wrote: > In the last 2 weeks or so, both sets of servers have been running under a > much larger load than usual, which does not seem to correlate with any > know events (significantly more accounts, network problems, etc.). We have > added more memory to some of these systems, and sometimes an additional > drive just for swap. Hmm ... i can add a 'me too' ... i'm running cyrus 2.0.16 on a free webmail service and in the last few months the loadavg was jumping high enough for the sendmail to stop accepting connections. Now i managed to move the MTA to frondend MXes, but the load didnt drop as much as i expect it would ... it's still hovering around 5. Since cpuload is about the same, this must be related to some increased i/o activity. Only half a year ago this same box was humming nicely with approx the same nubmer of users ... -- Jure Pecar msg08495/pgp0.pgp Description: PGP signature
Re: Cyrus continues to stop working.. no fix available?
On Tue, 14 May 2002 00:23:07 -0500 Dustin Puryear <[EMAIL PROTECTED]> wrote: > This is extremely interesting. Michael, do you find this happens at > seemingly random times though? We can go a week or two with no > problems, and then bam, I get a 911. Of course, our volume is > considerably lower than yours. Another issue, and one that may > differentiate our problems from yours (but hopefully not as your at > least have a work-around), is that I can sometimes restart Cyrus, and > even after a restart, no new connections are serviced. (They connect, > but get no service.) I've found that when this happens Cyrus will > often appear to work for a VERY short while, and then revert back to > the point where connections occur but no service (pop3d) responds. > > Shouldn't a restart completely fix the problem? If so we may be > fighting something different. A reboot also doesn't always clear up > the problem. Again, Cyrus will come up, but then fail shortly > thereafter. I've seen the exact same behaviour you're describing here on Linux when there's a lack of available entropy. Pop3d is more susceptible to this problem, because it calls some sasl functions that read some bits from /dev/random. I didn't check why exactly are they necessary and why imapd does not use these functions. Linux gathers entropy from disk i/o events and keyboard keystrokes iirc, but i saw it drain even on an extremly busy mail server. Imapd goes on, but pop3d processes just pile up, each one accepting a connection but not starting any service. During the peak hours i've seen this effect numerous times, resulting in too many open files and thus blocking other processes. One of the solutions is to recompile the kernel with the netdev-random patches (http://www.tech9.net/rml/linux/), and if this does not help, find /dev/random references in the sasl library and replace them with /dev/urandom. Iirc it's only defined in one place. -- Jure Pecar msg07742/pgp0.pgp Description: PGP signature
Re: Cyrus IMAPd v2.1.3 and sasl_checkapop()
Quoting David Fuchs <[EMAIL PROTECTED]>: > configure: error: libsasl2 without working sasl_checkapop. Cannot > continue. For me it worked only after i put LDFLAGS and CPPFLAGS pointing to the sasl2 place in front of ./configure ... -- Jure Pecar
Re: Cyrus IMAPd v2.1.3 and sasl_checkapop()
Quoting David Fuchs <[EMAIL PROTECTED]>: > configure: error: libsasl2 without working sasl_checkapop. Cannot > continue. For me it worked only after i put LDFLAGS and CPPFLAGS pointing to the sasl2 place in front of ./configure ... -- Jure Pecar
Re: DB vs. Flat File
On Mon, 04 Feb 2002 20:15:22 -0500 Walter Wong <[EMAIL PROTECTED]> wrote: > With the release of Cyrus IMAP 2.1.2 there will be a new backend option: > skiplist. > > We have already moved our development system to use it and performance > seems very good. I expect we'll be moving our production system to it > within a few months. It should not suffer from the same problems with > insert/modify operations that the flat file has and it has better list > speed than DB. > Is there any particular reason mysql cannot be used for this purpose? As i understand things this backend only has a need for a lot of simple queries and this is where mysql works best. -- Jure Pecar
Re: sieve & sasl2
On Thu, 31 Jan 2002 14:04:22 -0500 Darren Nickerson <[EMAIL PROTECTED]> wrote: > I had the same problem, which turned out to be due to an incorrect path to > sasl2's plugins. I have sasl and sasl2 coexisting on the same box, so: > > /usr/lib/sasl2 > /usr/lib/sasl same here, expect that /usr/lib/sasl2 is a symlink pointig to the /opt/saslV2/lib/sasl2. > are two very different beasts. Sadly, when I had compiled sasl2, I had > forgotten to give configure the: > > --with-plugindir=/usr/lib/sasl2 > > flag. So it was looking in the default (incorrect version) /usr/lib/sasl and was >failing to load the plugins. > Oh, i see it now. I specified /opt/saslV2/usr/lib/sasl2 for plugin dir ... Thanks for the hint :) -- Jure Pecar
sieve & sasl2
Hi list, I have a cyrus 2.1.1 authenticating via saslauthd -a pam. When i telnet to sieve port, i get: $ telnet 0 2000 Trying 0.0.0.0... Connected to 0. Escape character is '^]'. "IMPLEMENTATION" "Cyrus timsieved v1.1.0" "SIEVE" "fileinto reject envelope vacation imapflags notify subaddress regex" OK According to documentation, there should be available authentication methods listed after "SIEVE", but there are none. I also don't see any auth mechanisms in imtest. $ ls -p /usr/lib/sasl2 cyrus.conf@libcrammd5.so.2.0.0libkerberos4.la libplain.so@ Cyrus.conf@libdigestmd5.la libkerberos4.so@ libplain.so.2@ libanonymous.la libdigestmd5.so@ libkerberos4.so.2@ libplain.so.2.0.0 libanonymous.so@ libdigestmd5.so.2@ libkerberos4.so.2.0.0 libsasldb.la libanonymous.so.2@ libdigestmd5.so.2.0.0 libotp.la libsasldb.so@ libanonymous.so.2.0.0 libgssapiv2.la libotp.so@ libsasldb.so.2@ libcrammd5.la libgssapiv2.so@libotp.so.2@ libsasldb.so.2.0.0 libcrammd5.so@ libgssapiv2.so.2@ libotp.so.2.0.0 libcrammd5.so.2@ libgssapiv2.so.2.0.0 libplain.la Both cyrus.conf files point to /etc/imapd.conf. What am i doing wrong? -- Jure Pecar
Re: user unknown?
On Tue, 15 Jan 2002 15:14:53 -0500 Lawrence Greenfield <[EMAIL PROTECTED]> wrote: > The admin users always use the standard namespace. > > So you want to > > cm user/test > > Larry Now thats much better :) I knew it was something simple. Lmtp is now aware of where to deliver. Thanks for hint :) -- Jure Pecar
Re: user unknown?
On Tue, 15 Jan 2002 15:09:33 -0500 Lawrence Greenfield <[EMAIL PROTECTED]> wrote: > the mailbox for the user named "test" is called "user.test". > > you want to > > cm user.test > but not with unixhierarchysep: yes altnamespace: yes in this case i get user^test directory on disk and lmtp still rejects rcpt to: or with user unknown. -- Jure Pecar
Re: user unknown?
On Tue, 15 Jan 2002 14:39:03 -0500
Lawrence Greenfield <[EMAIL PROTECTED]> wrote:
> You're authenticating as an administrative user?
Yes. I have admins: cyrus line in imapd.conf.
> So for my mailbox, cyradm should be like:
> mail1.andrew.cmu.edu> lm user.leg
> user.leg (\HasChildren)
> mail1.andrew.cmu.edu>
$ cyradm -u cyrus imap
IMAP Password:
imap>
imap> cm test
imap> sam test test all
imap> sam test cyrus all
imap> lm test
test (\HasNoChildren)
$ telnet imap 24
Trying 192.168.200.4...
Connected to imap.
Escape character is '^]'.
220 mail.animeslovenija.org LMTP Cyrus v2.1.1 ready
lhlo feather2.animeslovenija.org
250-mail.animeslovenija.org
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-SIZE
250-AUTH EXTERNAL
250 IGNOREQUOTA
mail from:<[EMAIL PROTECTED]>
250 2.1.0 ok
rcpt to:
550 5.1.1 User unknown
quit
221 2.0.0 bye
Connection closed by foreign host.
> Do the log files have anything interesting?
on debug level, nothing.
strace of lmtp process is equally noninformative:
read(0, "rcpt to:\r\n", 4096) = 16
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
write(1, "550 5.1.1 User unknown\r\n", 24) = 24
--
Jure Pecar
user unknown?
Hi list,
I'm putting together a new imap server. Sasl, cyrus, authentication works
ok, i can add new user through cyradmin and set its ACLs. Now telneting to
lmtp port and trying to send mail to that user results in 550 5.1.1 User
unknown error message. Why?
I think i must be doing something wrong since i managed to came to this
point with cyrus 2.0.16 and with 2.1.1. If i understand cyrus correctly,
user -> mailbox mapping is all done in cyrus and is not related to how
auth stuff works. So where could i go wrong?
Thanks for hints.
Output of ver command in cyradm:
name : Cyrus IMAPD
version: v2.1.1 2002/01/14 20:08:56
vendor : Project Cyrus
support-url: http://asg.web.cmu.edu/cyrus
os : Linux
os-version : 2.4.12-ac3
environment: Cyrus SASL 2.1.0
Sleepycat Software: Berkeley DB 3.1.17: (July 31, 2000)
OpenSSL 0.9.6 24 Sep 2000
CMU Sieve 2.0
TCP Wrappers
lock = flock
auth = unix
idle = poll
notify = unix
dirhash = full
mboxlist = db3
/etc/imapd.conf (with mostly defaults)
configdirectory: /db/cyrus/config
defaultpartition: default
partition-default: /data/cyrus/data
umask: 077
servername: mail.animeslovenija.org
allowanonymouslogin: no
allowplaintext: yes
quotawarn: 90
timeout: 30
imapidlepoll: 60
poptimeout: 10
admins: cyrus
autocreatequota: 1
singleinstancestore: yes
duplicatesuppression: yes
sieveusehomedir: false
sievedir: /data/cyrus/sieve
sendmail: /usr/sbin/sendmail
postmaster: postmaster
hashimapspool: true
sieve_maxscriptsize: 64
sieve_maxscripts: 10
sasl_maximum_layer: 256
sasl_minimum_layer: 0
sasl_pwcheck_method: saslauthd
lmtpsocket: /db/cyrus/config/socket/lmtp
idlesocket: /data/cyrus/idled
notifysocket: /data/cyrus/notify
unixhierarchysep: yes
altnamespace: yes
userprefix: Other Users
sharedprefix: Shared Folders
logtimestamps: yes
/etc/cyrus.conf:
# standard standalone server implementation
START {
# do not delete these entries!
# for cyrus 2.0.x
# mboxlist cmd="ctl_mboxlist -r"
# deliver cmd="ctl_deliver -r"
# for cyrus 2.1.x
recover cmd="ctl_cyrusdb -r"
# this is only necessary if using idled for IMAP IDLE
# idledcmd="idled"
}
# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imap cmd="imapd" listen="imap" prefork=0
# imapscmd="imapd -s" listen="imaps" prefork=0
pop3 cmd="pop3d" listen="pop3" prefork=0
# pop3scmd="pop3d -s" listen="pop3s" prefork=0
sieve cmd="timsieved" listen="sieve" prefork=0
# at least one LMTP is required for delivery
lmtp cmd="lmtpd -a" listen="192.168.200.4:lmtp" prefork=1
lmtpunix cmd="lmtpd -a" listen="/var/imap/socket/lmtp" prefork=0
}
EVENTS {
# this is required
checkpoint cmd="ctl_cyrusdb -c" period=30
# this is only necessary if using duplicate delivery suppression
delprune cmd="ctl_deliver -E 3" period=1440
}
saslauthd is running with -a pam, which calls pam_mysql and goes down to
mysql db. auth is working. btw, when will something like mysql plugin for
saslauthd be available? :)
--
Jure Pecar
/dev/random and sasl ...
Hi list,
We just had a sudden pike of activity on our mail server. The result is that
pop3d simply stopped working. Pop3d processes are just getting spawned, but
they do nothing. If i telnet to port 110 the connection gets established,
but i dont get the greeting from cyrus. Imapd works ok as ever.
I managed to strace a pop3d just after it started, and it stopped on:
...
open("/data/cyrus/config/proc/27611", O_RDWR|O_CREAT|O_TRUNC, 0666) = 13
fstat64(0xd, 0xb544)= 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0x40dec000
_llseek(13, 0, [0], SEEK_SET) = 0
write(13, "pop.user's.domain[his.ip"..., 40) = 40
_llseek(13, 0, [40], SEEK_CUR) = 0
ftruncate(13, 40) = 0
open("/dev/random", O_RDONLY) = 15
read(15,
where it sits still.
This is cyrus-2.0.17cvs from around dec 15., running on redhat 6.2 with
2.4.17 kernel.
Now afaik /dev/random is blocking if it's out of entropy, while /dev/urandom
is not. Is there a reason why using /dev/random at all? I see frequent
slowdowns in response time of pop3 and this could well be the reason.
I can't find any reference to /dev/random in cyrus ... so where can i look
for code that calls it? I'd like to find out why it is not /dev/urandom ...
Ah, it's probably in sasl. True, config.h ... Is there a reason not to use
/dev/urandom there? Any side effects if i change that and recompile?
--
Jure Pecar
cvs down?
Any reason for the cvs server to be down? $ cvs -d :pserver:[EMAIL PROTECTED]:/cvs login (Logging in to [EMAIL PROTECTED]) CVS password: cvs [login aborted]: connect to cvs.andrew.cmu.edu:2401 failed: Connection timed out -- Jure Pecar
Re: Webmail for Cyrus Imap ?
On Tue, 11 Dec 2001 22:18:34 -0500 "Alex Kamalov" <[EMAIL PROTECTED]> wrote: > Guys, > > I was wondering if anyone can tell me a preferred webmail interface for > cyrus-imap ? Which one should I use that could be relatively painless, > albeit with functionality ? > Try jawmail (jawmail.sf.net), up & running in <5min. -- Jure Pecar
cvs
All that talk about cyrus 2.1 and sasl v2 made me want to take a look at it ... webcvs lists just too many branches to choose from :) so which one do i pick? imapd-sasl-v2(-base) or cyrus-2_1 ? -- Jure Pecar
user.sub question
Hi list, We're running a ~100.000 users free webmail service, based on cyrus and ldap. Recently, some of our users started complaining that they can't see their subfolders anymore. I found the .sub files and also found this from the doc/server-design file: * User directory: The subdirectory "user" under the configuration directory contains files with per-user information. For each user "USER" with an INBOX, it may contain the file "USER.sub". The file contains a list of the user's subscribed mailboxes. Most of the .sub files I see are empty (zero lenght), altough _all_ of our users have sent-mail subfolder. I found some .sub files that only contained user.username line, and some that looked like what I would expect (user.username, user.username.sent-mail, user.username.somesubfolder). Of those that were complaining, I found that some didn't have .sub file and some have them, both empty and with a some subfolders. So I'm a bit confused here ... Is anyone willing to explain this stuff in more detail than there is in server-design doc file? Or at least suggest me which source files to dig through to find out how is all that supposed to work? Or, better yet, suggest me a way to fix those users' mailboxes? We're using cyrus 2.0.14. -- Jure Pecar
