Re: Performance question...

2004-02-25 Thread Simon Brady
On Wed, 25 Feb 2004, Henrique de Moraes Holschuh wrote:

 db 3.2 is safe. I don't know about 3.3.  4.0 and 4.1 are a NO GO for SMP.

Is this all Linux SMP installations, or just those using NPTL?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand

---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: lmtpd reporting faulty parse errors

2004-02-16 Thread Simon Brady
On Mon, 16 Feb 2004 [EMAIL PROTECTED] wrote:

 Feb 15 04:26:00 host.cwi.nl lmtpd[28106]: [ID 358100 local6.info] sieve parse 
 error for user: line 7: fileinto not required
  [...] 
 Did anyone experience this before? Does anybody have any idea why this
 is happening?

If the script includes a fileinto action it needs to begin with the line

  require fileinto;

This is because fileinto is an optional capability for Sieve
implementations (see RFC 3028 sections 2.10.5, 3.2 and 4.2). Admittedly 
the error message is ambiguous to someone who doesn't already know the 
special meaning being given to the verb require: perhaps something like

  require statement missing for fileinto capability

would be clearer?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand

---
Home Page: http://asg.web.cmu.edu/cyrus
Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: skiplist vs db format

2004-01-25 Thread Simon Brady
On Fri, 23 Jan 2004 [EMAIL PROTECTED] wrote:

 We are using Berk. db4 and the IO wait for the disk containing the db is
 consistantly over 50%.  The partition is 7GB, which also contains the
 user, quota, proc, and db directory. [...]

One easy change you can make is to mount /var/imap/proc on tmpfs. At our
site this made a dramatic improvement to load average, and although it
won't cure your bottleneck it should at least free up some disk I/O
capacity for DB access.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: Trouble logging with syslog

2004-01-07 Thread Simon Brady
On Wed, 7 Jan 2004, Jason Williams wrote:

 However, the only snag I have run into is getting syslog to log information 
 to /var/log/imapd.log, as suggested on the cyrus web site.
 
 Here is what I have in my syslog.conf file:
 
 local6.debug/var/log/imapd.log
 auth.debug  /var/log/auth.log
 
 auth.log works no problem, but not imapd.log

Are you certain the FreeBSD package logs to local6? For example, the
Debian package is modified to log to mail instead. If you can't find
anything in the docs, a crude approach would be catch *.debug then
selectively add ;foo.none on to the end of it until you find which one
imapd is using.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: Cyrus IMAPd 2.2.2-BETA Released

2003-10-29 Thread Simon Brady
On Wed, 29 Oct 2003, Simon Matter wrote:

 Okay, I will do the same. Means, if there is need to run the auto* stuff,
 I do it on a box with the right auto tools and then create a patch of the
 results to have them available for builds where the auto* tools are
 missing.

RH7.3 (but not 7.2) supplies more recent auto* tools in the automake15 and
autoconf253 RPMs. How about adding a weak dependence on those, for cases
where users need to run them?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: Cyrus IMAPd 2.2.2-BETA Released

2003-10-29 Thread Simon Brady
With my brain disengaged I wrote:

 RH7.3 (but not 7.2) supplies more recent auto* tools in the automake15 and
 autoconf253 RPMs. How about adding a weak dependence on those, for cases
 where users need to run them?

Of course it would help if I'd read the version numbers in the original
message properly - while you might get away with autoconf 2.53 for 2.57,
automake 1.5 is probably too far behind 1.7 to be of use.

Then again, maybe it's worth testing - if it does work out then I'll claim 
to have known so all along :-)

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: Quota problems running 2.1.15

2003-10-20 Thread Simon Brady
On Mon, 20 Oct 2003, Alec H. Peterson wrote:

 I've got a situation where quota -f deletes all quotas for mailboxes beyond 
 a certain point in the alphabet even though they exist.  This in turn 
 causes mail delivery to fail with a System I/O error for those mailboxes 
 until I re-create the quotas. However, running quota -f again causes the 
 deleting to take place again.
 
 Any ideas of how I can fix this behavior?

I've seen this happen after a mailbox was manually deleted from the spool 
directory. quota would run through the users in order until it hit the 
corrupt one then abort, leaving all the remaining quotas in limbo.

Since reconstructing the mailbox didn't work, our get things going before
the users lynch us fix was to export the mailboxes db with ctl_mboxlist
-d, delete all references to the broken directory, rebuild with
ctl_mboxlist -u then rerun quota -f.

You should be able to identify the offending mailbox by observing where
quota gets up to before dying.

Good luck!

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: cyrus auto account provisioning

2003-07-29 Thread Simon Brady
On Tue, 29 Jul 2003, joe ritter wrote:

 I wanted to see how people were doing any kind of auto
 account provisioning using Cyrus IMAP . We currently
 do not have any automated system for adding users or
 deleteing them etc. We use a combination of Cyrus and
 squirrelmail for our users. I know some posters on
 this list are from an ISP sytle environment and I
 wanted to find out what was working
 opensource/freeware, homegrown scripts, web font ends
 etc? 

My site (a university with ~20K users) uses a suite of locally-written 
perl wrapped around the Net::IMAP module. There's a shell frontend 
for helpdesk staff to manually create staff mailboxes, and a 
bulk-processing frontend which picks up batches of students to add or 
remove as provided by our student management system (while only the latter 
counts as auto-provisioning, the common backend simplifies manual 
operations too).

As well as mailbox creation the scripts do things like LDAP updates and
delivery of standard welcome messages - it's highly site-specific and more
than a little crufty, but it does the job for us.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: quota warning problem - Is it a bug of cyrus imap?

2003-06-30 Thread Simon Brady
On Tue, 1 Jul 2003, [gb2312] Daniel Qian | ǮʡÄÏ wrote:

 I have installed cyrus imap 2.13 recently and it works very well ecept
 when the mailbox data reaches big volume. On a 10G quota for a mailbox
 the server starts warning the outlook user and closes the connection
 when the used space is just al little over 15% of the quota.  But for
 small quota the server works fine- it starts the warning at 90% of the
 quota limit. Is there anyone on this list aware of this problem? 

I haven't looked at the code, but this sounds like what you'd see if quota 
math was being done in 32 bits: 10G mod 4G = 2G, and 90% of 2G is 18% of 
the original 10G figure.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: problems with imap and imsp installation

2003-06-27 Thread Simon Brady
On Fri, 27 Jun 2003, Richard Hopkins wrote:

 And I'd also recommend a --without-ldap on Solaris systems unless you 
 intend trying to use it (and failing miserably). I run configure with a...
 
 ./configure --with-auth=unix --without-ldap

I've got --with-ldap working on Solaris 8 (SASL 2.1.10) but it was a
painful process - not because of the Cyrus code, but because of problems
with the supporting infrastructure. This page was a lifesaver for me,
especially the gem about libtool and runpaths:

   http://www.bolthole.com/solaris/LDAP.html

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Cyrus on Red Hat Enterprise Linux

2003-06-10 Thread Simon Brady
Hello world,

We're planning to retire our Solaris mail server at the end of the year
and move Cyrus to Linux. I'd intended to move to RH7.3, which we use
widely and understand quite well, but Red Hat's support policies have
killed that idea.

Is anyone currently running Cyrus on Red Hat Enterprise, either out of the
box or self-installed? If so, have you encountered any issues beyond those
to be expected on 7.x? Having built from source on Solaris I was looking
forward to using Simon Matter's RPMs, but I don't know how they'll
interact with the RH Network all your server are belong to us madness.

[OT: Yes, I'm aware that there other other Linuces beyond RH, but we're
committed to HP hardware which is only certified for RH and SuSE (one of
my colleagues has been told by an HP engineer that they support Debian but
I've yet to see anything official). We have zero SuSE experience in-house,
so RH kind of have us by the danglies...]

Thanks for any feedback,
Simon

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: Message with attachments shows up totally empty.

2003-06-03 Thread Simon Brady
On Mon, 2 Jun 2003, Mark London wrote:

 Using the latest version of cyrus, one of our users received a message
 with several attachments.  Using different mail clients, they all show
 only the header of the message, no body or attachments, even though they
 are there (the person was able to forward the message to others,
 including themselves). Squirrelmail displays it without a problem.  Is
 this a known problem, and if not, is there anything I can do to help
 debug the source of the problem (i.e. turning on a debug switch?)  

This is almost certainly a client issue - my guess would be the sending 
software is munging the MIME structure of the message (Eudora is 
notorious for this). The best way to isolate the culprit is to cat the 
raw message in the server's spool directory - look for the MIME 
Content-Type header at the top of the message, e.g.

  Content-Type: multipart/mixed; boundary=--rhubarbrhubarbrhubarb--

then check the individual parts, each of which should start with the 
above boundary string followed by its own header info:

  rhubarbrhubarbrhubarb--
  Content-Type: text/html
  Content-Disposition: attachment; filename=attached.html

You'll need to consult the MIME spec (RFC2045) if you want all the gory
details, but even a cursory look at the raw message should give you an
idea of what's coming through.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



RE: Questions about LDAP schema and Multi-Domain IMAP

2003-03-05 Thread Simon Brady
On Wed, 5 Mar 2003, Howard Chu wrote:

 I suggest you ditch OpenLDAP 2.0.27 and update to the latest 2.1 release.
 Then you ditch saslauthd  PAM and have SASL authenticate directly against
 LDAP. Note that OpenLDAP 2.0.X does not work with Cyrus SASL 2.1.x anyway, so
 you need OpenLDAP 2.1 if you're already using SASL 2.1.

Just to clarify, does the last sentence refer to OpenLDAP authenticating
against SASL or SASL authenticating against OpenLDAP? Like others on the
list I've got SASL 2.1.10 authing quite happily to OpenLDAP 2.0.27 via
saslauthd, so I assume you mean the former. This may be where the
confusion is arising.

 There are a number of advantages to using this approach over any other one:
   saslauthd only supports plaintext login, and plaintext logins are
   inherently insecure.

Unless you're using (only) TLS, in which case they seem to be a _lot_ 
simpler to set up from scratch than some of the other mechanisms (judging 
by the frequent requests for help I see on the SASL list). Of course, if 
you can't enfore strong transport-layer encryption then your point stands.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: Deleting records from db

2003-02-26 Thread Simon Brady
On Wed, 26 Feb 2003, Sebastian Konstanty Zdrojewski wrote:

 Hi,
 
 I tried with this:
 
 localhost.localdomain reconstruct user.mailboxname
 reconstruct: Operating System Error
 
 The imapd.log file appended the following line:
 
 Feb 26 16:08:48 nexus imapd[30158]: Reconstructing 
 'user.massimiliano_rovatti' (not recursive) for user 'postmaster'
 
 Sorry for boring... I would like to get out of this problem without 
 deleting the entire database...

I encountered this exact situation (directories manually deleted, 
leading to an unreconstructable mailbox) about a month ago. The solution 
we chose was to export the mailboxes db (ctl_mboxlist -d), delete the 
bogus entries from the textfile, then import it back in.

However, this may not be the Right Way to fix it. In our case the server 
was down anyway (the damaged mailbox had killed an overnight quota check 
and corrupted the quota db), so this brute force and ignorance approach 
made sense at the time.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand



Re: compiling error - cannot find -lssl

2003-02-18 Thread Simon Brady
On Tue, 18 Feb 2003, Mark Wessel wrote:

 Tried autoconf 2.13... same config error.
  [...]
 On Tue, 18 Feb 2003, Mark Wessel wrote:
 
  ok, I've updated to autoconf-2.53-8, and have the latest configure.in  file
  (1.232.4.38). Now there seems to be an issue with SASL, I think...
 
  After running:
 
  ./configure --with-auth=unix --with-sasl=/usr/lib/sasl
  --with-bdb-incdir=/usr/local/BerkeleyDB.4.1/include
 
  I get receive this error:
 
  ./configure: line 10055: syntax error near unexpected token '2,1,7'
  ./configure: line 10055: 'CMU_SASL2_REQUIRE_VER(2,1,7)'

Hmm, are you sure you've rebuilt configure using 2.13? I can't recreate 
this error with either version of autoconf on RH7.3, but I notice that my 
2.13-generated configure is only 7389 lines long (vs 12902 for 2.53).

To be absolutely certain, I'd suggest completely nuking your build dir, 
untarring the source distro again, then copying in the configure.in from 
CVS and rerunning autoconf 2.13. Yes, I know it sounds superstitious, but 
it's got me out of a few holes in the past :-)

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: compiling error - cannot find -lssl

2003-02-18 Thread Simon Brady
On Tue, 18 Feb 2003, Jeremy Rumpf wrote:

 Dunno what Mark's root problem is, just wanted to share the above tidbits...

Thanks for the info Jeremy - I'm still not brave enough to touch RH8, so 
this is good to know.

I may be wrong, but I believe the problem Mark is having is documented in

  http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=1798

This is due to an OpenSSL detection patch I submitted, which unfortunately
lost a line somewhere between submission and application...

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





Re: compiling error - cannot find -lssl

2003-02-17 Thread Simon Brady
On Sun, 16 Feb 2003, Mark Wessel wrote:

 After running 'make all CFLAGS=-O' I eventually get this error:
 
 Running Mkbootstrap for Cyrus::IMAP ()
 chmod 644 IMAP.bs
 rm -f blib/arch/auto/Cyrus/IMAP/IMAP.so
 LD_RUN_PATH=/usr/local/lib:/lib gcc -shared -L/usr/local/lib IMAP.o -o
 blib/arch/auto/Cyrus/IMAP/IMAP.so ../../lib/libcyrus.a -lsasl2 -lssl 
 -lcrypto
 /usr/bin/ld: cannot find -lssl
 collect2: ld returned 1 exit status
 make[2]: *** [blib/arch/auto/Cyrus/IMAP/IMAP.so] Error 1

Are you building imapd-2.1.12? If so, replace the top-level configure.in 
with the latest mainline version (currently 1.259) from CVS:

 http://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/configure.in

Obviously you'll need to reconfigure after this. If this doesn't help, can 
you please post some more details about your configuration?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





Re: compiling error - cannot find -lssl

2003-02-17 Thread Simon Brady
On Mon, 17 Feb 2003, Mark Wessel wrote:

 After replacing 'configure.in' with the latest version, and running 
 './configure' without any errors, I can no longer run 'make depend'
 
 Error message after running 'make depend':
 
 cd .; rm -f config.cache; autoconfig
 /bin/sh: line 1: autoconf: command not found
 make: *** [configure] Error 127

That's OK, you just need to install autoconf (the tool that parses
configure.in and creates a new configure script). It'll be available as an
RPM on the RedHat CDs (if it's already installed, then make sure it's in 
your path).

However, if you're relatively new to Linux then a better option might be 
to install Cyrus directly from the RPMs that contributors to this list 
have created. I don't run Cyrus on Linux myself, so I'm not sure whether 
the RPMs will work on RH8, but a troll through the list archives should 
answer that question.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: Issues with Install

2003-02-04 Thread Simon Brady
On Tue, 4 Feb 2003, lists wrote:

 I am using redhat 8.0 and I get this error
 [cyrus@imap tools]$ ./mkimap
 reading configure file...
 done

Have you set the 'configdirectory' parameter in your /etc/imapd.conf file? 
It looks like mkimap isn't seeing one, so it's trying to use an empty 
string instead. This will also explain messages such as

 Feb  4 20:55:26 imap ctl_cyrusdb[15487]: DBERROR db4: 
 /var/imap/db/__db.001: No such file or directory

(in this case master is trying to use /var/imap as the config directory).

As well as this there are other things in imapd.conf you need to set - 
the imapd.conf manpage is a must-read for this.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand







Re: PHP 4.2.3 for Redhat 7.3

2003-01-28 Thread Simon Brady
On Tue, 28 Jan 2003, crab ravi wrote:

 I'm in a process of setting up Postfix-Cyrus-Web-Cyradm on Redhat 7.3
 Server. As per Mr Luc's Howto, PHP-4.2.3 is needed, but RH 7.3 is
 shipped with PHP-4.1.2-7. I could not able to find RPM in the internet.

The easiest solution is to build PHP from source. This is what I do on our 
RH7 webservers, and it works fine with the Apache RPMs (although you will 
need to install apache-devel so PHP can find the APXS tool, and you may 
need other -devel packages so it can find headers).

An alternative would be to take an SRPM from another distribution
(www.rpmfind.net lists a few) and do an rpm --recompile, but that may
introduce subtle problems of its own.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: PHP 4.2.3 for Redhat 7.3

2003-01-28 Thread Simon Brady
On Tue, 28 Jan 2003, Brian wrote:

 ... and first try rebuilding the php from Rawhide:
 
 rpmbuild --rebuild
 ftp://rpmfind.net/linux/rawhide/1.0/SRPMS/SRPMS/php-4.2.2-13.src.rpm

Except that according to the original message Web-Cyradm requires 4.2.3, 
which is why I didn't suggest this as an option.

That nitpick aside, I take your point - in an ideal world we'd all have 
time to do things properly, and the fruits of our packaging labours would 
flow back to the rest of the community. However, PHP is big and hairy and 
insanely configurable, and even if I took the time to package it up would 
anyone else actually be happy with my configuration choices?

I know this attitude is short-sighted, and I have a lot of respect for 
people who do put the effort into package creation. But sadly, in an 
environment where time is (someone else's) money, kludges sometimes make 
the best sense.

Until you go to upgrade, of course :-)

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





Re: Problem after removing quota

2003-01-25 Thread simon . brady
On Sat, 25 Jan 2003, John A. Tamplin wrote:

 Why did you delete the quota file in the first place?  If you want to 
 remove the quota, do it via IMAP.  You can't expect to just go delete 
 files maintained by Cyrus and have everything continue to work (see the 
 long discussion on virus scanners deleting message files from the 
 mailstore).

Fair enough, but based on recent experience I share Denis's feeling that
quota handling is somewhat fragile.

What happened locally was a blatant case of administrator error: one of my 
colleagues (not me!) tried to clean out a particularly large mailbox 
hierarchy by manually deleting it then doing a reconstruct. Yes, I know, 
no comment required...

Historical practice here (which dates from 1.5.19 days and may no longer
be appropriate) is to run a nightly quota -f over our ~27000 mailboxes.  
The molested mailbox was still in the mailboxes db, and when quota
discovered that its expected subfolders were missing it immediately died.  
One could argue it should have skipped the problem entry and kept going,
but OTOH perhaps it is better to say oi! something's broken! and bail
out.

Problem is, in bailing out it lost ALL the quota files for the remaining
unprocessed users - but only the quota files, not whatever magic flag says
this mailbox has a quota file. The resulting inconsistency led to all
sorts of problems, the cutest of which was POP clients repeatedly
downloading the same messages because pop3d couldn't close cleanly. In the
end we fixed things by shutting Cyrus down, doing a mailboxes
export/edit/import to expunge the broken mailbox, then doing a complete
quota rebuild.

As I said, the root cause was meatware failure. I fully agree that it's
unreasonable to mess with the spool and not expect things to break.
However, IMAO the original mistake was amplified out of all proportion by
the quota program's response - a single corrupt mailbox shouldn't lead to
it losing half the quota files.

Humans will do dumb things. Third-party software will run amok. Smelly
stuff will happen. I'm quite happy for Cyrus to shut down when it detects
a this can't happen situation, but personally I wish quota could have
done so in a less drastic fashion. Maybe it's worth considering how quota 
management can be made more robust?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





Re: lmtpd NUL rejection

2003-01-13 Thread simon . brady
On Fri, 10 Jan 2003, Igor Brezac wrote:

 Have you tried F=1 mailer flag?  This may only work in Sendmail 8.12.x.
 
 From op.txt:
 
 1   Don't send null characters ('\0') to this  mailer.

We've gone live with 8.12.7, and F=1 is doing just what we hoped for: 
silently deleting NULs in transit. Thanks again for the pointer.

Along the way, I've discovered a related bug in lmtpd (2.1.11). If a
misconfigured client sends a message with lines longer than 8190 chars,
the following code in imap/lmtpengine.c (lines 672-675) will incorrectly
report the presence of NULs:

 else if (p[0] != '\n') {
 /* line contained a \0 not at the end */
 r = IMAP_MESSAGE_CONTAINSNULL;
 continue;

(It assumes all lines end with '\n', which prot_fgets() doesn't 
guarantee.)

I'm not sure what the right fix is for this. It's trivial to identify the 
long-line case, but should lmtpd return an error (and if so, what?), or 
should it try to handle this pathological case?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: saslauthd performance anxiety

2003-01-11 Thread simon . brady
On Fri, 10 Jan 2003, Paul M Fleming wrote:

 Personally I have issues with dumping the contents of a password cache
 to a file. Especially in this case, they WILL be stored in cleartext. I
 had planned on keeping somes stats (hits,misses,etc)

Fair enough. Even if you don't dump passwords, it might be useful to show
info like the username, when its entry was added to the cache, and when it
was last hit. This could be handy when answering questions like why did
user X experience an auth failure at time T?.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: lmtpd NUL rejection

2003-01-11 Thread simon . brady
On Fri, 10 Jan 2003, Igor Brezac wrote:

 Have you tried F=1 mailer flag?  This may only work in Sendmail 8.12.x.
 
 From op.txt:
 
 1   Don't send null characters ('\0') to this  mailer.

Aha, this sounds like just what we need - and yes, it was added in 8.12.0 
so it looks like we've got another upgrade to do. Fortunately the new 
edition of the bat book has come out just in time...

Thanks!

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: saslauthd performance anxiety

2003-01-09 Thread simon . brady
On Wed, 1 Jan 2003, Igor Brezac wrote:

 On Wed, 1 Jan 2003 [EMAIL PROTECTED] wrote:
  [...]
  Can anyone offer advice on tuning the saslauthd pool? Are there particular
  options, either on the command line or in saslauthd.conf, which I should
  be looking at?
 
 Try using 'ldap_auth_method: custom'.  It is up to three times faster
 than the 'bind' method.

Thanks for the suggestion. Unfortunately 'custom' wasn't an option for
us, although we certainly could have benefited from it. The reason we
can't use it is that to support password migration our shell back-end does
mad things like:

   try binding to new server;
   if (failure) {
  try binding to old server;
  if (success)
 update user password in new server for next time;
   }

Don't look at me, I just inherited it :-)

This logic (to use the term loosely) makes it impossible to return a
sensible response to a search on userPassword. Instead, I committed a
gross hack and implemented a new method called auth_fastbind. It does away
with the search and extra anonymous bind in auth_bind by making two
assumptions:

   1. Expanding the ldap_filter expression gives the fully-qualified DN
   2. There is no cost to staying bound as a named user

These held for our shell back-end, but I don't know how applicable they 
are to wider use. Still, if anyone's interested I've attached the patch 
(against 2.1.10).

Simon Brady mailto:[EMAIL PROTECTED]
Systems Specialist  Ph. +64 3 479-5217
ITS Technical Services  Fax +64 3 479-5080
University of Otago, Dunedin, New Zealand   Mobile +64 27 411-6045

diff -ru cyrus-sasl-2.1.10.orig/saslauthd/lak.c cyrus-sasl-2.1.10/saslauthd/lak.c
--- cyrus-sasl-2.1.10.orig/saslauthd/lak.c  Fri Dec  6 02:54:58 2002
+++ cyrus-sasl-2.1.10/saslauthd/lak.c   Fri Jan 10 00:19:45 2003
@@ -70,6 +70,7 @@
 static int lak_search(LAK *, const char *, const char **, LDAPMessage **);
 static int lak_auth_custom(LAK *, const char *, const char *, const char *);
 static int lak_auth_bind(LAK *, const char *, const char *, const char *);
+static int lak_auth_fastbind(LAK *, const char *, const char *, const char *);
 static int lak_result_add(LAK *lak, const char *, const char *, LAK_RESULT **);
 static int lak_check_password(const char *, const char *, void *);
 static int lak_check_crypt(const char *, const char *, void *);
@@ -179,6 +180,8 @@
} else if (!strcasecmp(key, ldap_auth_method)) {
if (!strcasecmp(p, custom)) {
conf-auth_method = LAK_AUTH_METHOD_CUSTOM;
+   } else if (!strcasecmp(p, fastbind)) {
+   conf-auth_method = LAK_AUTH_METHOD_FASTBIND;
}
} else if (!strcasecmp(key, ldap_timeout)) {
conf-timeout.tv_sec = lak_config_int(p);
@@ -917,6 +920,24 @@
 }
 
 
+static int lak_auth_fastbind(LAK *lak, const char *user, const char *realm, const 
+char *password) 
+{
+   int rc;
+   char *dn = NULL;
+
+   rc = lak_filter(lak, user, realm, dn);
+   if (rc != LAK_OK || dn == NULL) {
+   syslog(LOG_WARNING|LOG_AUTH, lak_filter failed.);
+   return LAK_FAIL;
+   }
+
+   rc = lak_bind(lak, LAK_BIND_AS_USER, dn, password);
+
+   free(dn);
+   return rc;
+}
+
+
 int lak_authenticate(LAK *lak, const char *user, const char *realm, const char 
*password) 
 {
int rc;
@@ -932,8 +953,10 @@
 
if (lak-conf-auth_method == LAK_AUTH_METHOD_BIND) {
rc = lak_auth_bind(lak, user, realm, password);
-   } else {
+   } else if (lak-conf-auth_method == LAK_AUTH_METHOD_CUSTOM) {
rc = lak_auth_custom(lak, user, realm, password);
+   } else {
+   rc = lak_auth_fastbind(lak, user, realm, password);
}
 
return rc;
diff -ru cyrus-sasl-2.1.10.orig/saslauthd/lak.h cyrus-sasl-2.1.10/saslauthd/lak.h
--- cyrus-sasl-2.1.10.orig/saslauthd/lak.h  Fri Oct 18 10:30:58 2002
+++ cyrus-sasl-2.1.10/saslauthd/lak.h   Fri Jan 10 00:19:45 2003
@@ -53,6 +53,7 @@
 
 #define LAK_AUTH_METHOD_BIND 0
 #define LAK_AUTH_METHOD_CUSTOM 1
+#define LAK_AUTH_METHOD_FASTBIND 2
 
 typedef struct lak_conf {
 char   *path;



Re: saslauthd performance anxiety

2003-01-09 Thread simon . brady
On Wed, 1 Jan 2003, Kervin L. Pierre wrote:

 Maybe you should seriously consider moving from back-shell to back-perl, 
 which you can optimize much more and is probably quicker right of the 
 bat, since it does not spawn a separate process for the interpreter.
 
 Better still, have you thought of back-meta or back-ldap?  These were 
 designed for ldap routing.

Hmmm, good suggestions. I'd looked at the OpenLDAP 2.1 Admin Guide early 
on, and nothing in it jumped out at me as justifying going beyond 2.0.27 
(the words latest stable have a warm glow about them...). If I'd known 
about these other back-ends I would probably have decided differently.

Given that 2.1 is a big unknown to us (and it seems to want things like 
DB4 which we don't have experience with), I'm not keen on dropping it into 
production without some serious testing. Still, it sounds like just what 
we need.

(On the subject of OpenLDAP and back-shell, I should also mention that we 
uncovered a nasty race condition that can lead to deadlocked shell 
subprocesses. The bug report and fix are at 
http://www.OpenLDAP.org/its/index.cgi?findid=2262)

 I suspect you're optimizing the the wrong bottleneck.

You are sooo right - I upped the saslauthd pool enough to prevent
exhaustion, and we were fine for the first few days until the users came
back. By about 9am Friday the CPU overhead of spawning all those perl
processes had authentication timing out horribly (and the above-mentioned
deadlock issue didn't help either). And that was just the start...

In the end I reduced the load by hacking saslauthd (as described in my 
reply to Igor), which bought us time to write and test a C replacement for 
the perl. Of course the perl is much easier to understand and maintain, so 
going to a smart back-end is a better long-term solution.

Oh well, live and learn... Thanks for the advice all the same!

Simon Brady mailto:[EMAIL PROTECTED]
Systems Specialist  Ph. +64 3 479-5217
ITS Technical Services  Fax +64 3 479-5080
University of Otago, Dunedin, New Zealand   Mobile +64 27 411-6045





lmtpd NUL rejection

2003-01-09 Thread simon . brady
I've recently upgraded imapd from 1.5.19 to 2.1.11, and instead of having
sendmail invoke deliver it now talks to lmtpd over a Unix socket. All is
well, except that lmtpd is much more scrupulous about checking its input
than deliver was - in the space of a week, it's detected three otherwise
normal messages containing embedded NULs and has rejected them with DSN
554 5.6.0 Message contains NUL characters (status
IMAP_MESSAGE_CONTAINSNULL in imap/lmtpengine.c).

OK, fair enough, except that sendmail responds to the bounce by trying to 
copy the message to postmaster. Via lmtpd. Oops.

Clearly the input is bad and lmtpd is justified in rejecting it. However,
broken mail clients (or whatever - we haven't identified any common factor
yet) are a fact of life, and having mail stuck in a non-delivery loop
isn't very helpful for our users.

What's the Right Thing to do here? Should sendmail (8.11.2) be configured
to somehow report the failure without forwarding the message, or perhaps
do NUL filtering on the fly? Or is there some way of configuring the lmtp
mailer definition to get around this problem?

And out of general curiosity, have other sites moving to lmtpd encountered 
this, or are we just particularly weird?

Thanks...

Simon Brady mailto:[EMAIL PROTECTED]
Systems Specialist  Ph. +64 3 479-5217
ITS Technical Services  Fax +64 3 479-5080
University of Otago, Dunedin, New Zealand   Mobile +64 27 411-6045




Re: Problems with authentication

2003-01-02 Thread simon . brady
Michael Obster wrote:

S: L01 NO Login failed: generic failure
Authentication failed. generic failure
Security strength factor: 0

Are you logging cyrus (local6.debug) and SASL (auth.debug)? If so, what do 
they say?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: make depend problems

2002-12-31 Thread simon . brady
On Tue, 31 Dec 2002 [EMAIL PROTECTED] wrote:

 I am trying to compile the latest version of Cyrus IMAPD and have some
 problems already with the make depend, as you can see here in my output:
 
 gmake[1]: Leaving directory `/opt/source/cyrus-imapd-2.1.11/imtest'
 ### Making depend in /opt/source/cyrus-imapd-2.1.11/perl
 gmake[1]: Entering directory `/opt/source/cyrus-imapd-2.1.11/perl'
 ### Making depend in /opt/source/cyrus-imapd-2.1.11/perl/sieve
 gmake[2]: Entering directory `/opt/source/cyrus-imapd-2.1.11/perl/sieve'
 gmake[2]: *** No rule to make target `depend'.  Stop.
 gmake[2]: Leaving directory `/opt/source/cyrus-imapd-2.1.11/perl/sieve'
 gmake[1]: *** [depend] Error 1
 gmake[1]: Leaving directory `/opt/source/cyrus-imapd-2.1.11/perl'
 gmake: *** [depend] Error 1
 
 Does anyone have an idea what the problem could be ??? I am using standard
 GNU tools on Solaris 9.

Are you using the --disable-sieve configure option? It's broken and will
produce this error, but the attached patch (tested on Solaris 8 with GNU
tools) fixes the problem.

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand

diff -ru cyrus-imapd-2.1.11.orig/configure.in cyrus-imapd-2.1.11/configure.in
--- cyrus-imapd-2.1.11.orig/configure.inSun Nov 17 05:31:54 2002
+++ cyrus-imapd-2.1.11/configure.in Wed Jan  1 13:37:32 2003
@@ -871,7 +871,10 @@
EXTRA_OUTPUT=${EXTRA_OUTPUT} timsieved/Makefile notifyd/Makefile
 
PERL_SUBDIRS=${PERL_SUBDIRS} sieve
+   PERL_DEPSUBDIRS=sieve
EXTRA_OUTPUT=${EXTRA_OUTPUT} perl/sieve/Makefile perl/sieve/lib/Makefile
+else
+   PERL_DEPSUBDIRS=none
 fi
 
 dnl Check for MD5 functions
@@ -941,6 +944,7 @@
 dnl   fi
 dnl ], perl=$PERL; with_perl=$with_perl; prefix=$prefix; SASL_LIB=$LIB_SASL; 
SASL_INC=$SASLFLAGS; export SASL_LIB SASL_INC)
 AC_SUBST(PERL_SUBDIRS)
+AC_SUBST(PERL_DEPSUBDIRS)
 AC_SUBST(PERL)
 
 dnl make sure that Makefile is the last thing output
diff -ru cyrus-imapd-2.1.11.orig/perl/Makefile.in cyrus-imapd-2.1.11/perl/Makefile.in
--- cyrus-imapd-2.1.11.orig/perl/Makefile.inTue Jul 23 04:16:05 2002
+++ cyrus-imapd-2.1.11/perl/Makefile.in Wed Jan  1 13:37:55 2003
@@ -44,7 +44,7 @@
 
 SUBDIRS = @PERL_SUBDIRS@
 
-DEPSUBDIRS = sieve
+DEPSUBDIRS = @PERL_DEPSUBDIRS@
 
 srcdir = @srcdir@
 VPATH = @srcdir@
@@ -107,11 +107,13 @@
rm -f Makefile Makefile.bak config.status config.log config.cache
 
 depend::
-   @for d in  $(DEPSUBDIRS); \
-   do \
-   (cd $$d; echo ### Making depend in `pwd`;   \
-   $(MAKE) $(MFLAGS) DESTDIR=$(DESTDIR) depend) || exit 1; \
-   done
+   @if test $(DEPSUBDIRS) != none; then \
+   for d in  $(DEPSUBDIRS); \
+   do \
+   (cd $$d; echo ### Making depend in `pwd`;   \
+   $(MAKE) $(MFLAGS) DESTDIR=$(DESTDIR) depend) || exit 
+1; \
+   done; \
+   fi
 
 checkdepend::
 



Re: [Annoyed] Cyrus-imapd/sasl upgrade and lmtpd behaviour...

2002-12-31 Thread simon . brady
On Mon, 30 Dec 2002, Scott Smith wrote:

 What you probably want to do is add your MTA to the same group as Cyrus,
 at most.  [...]  Cyrus / SASL peeps, can you DOCUMENT this, since I
 personally know of several people that have had this problem, including
 myself when I upgraded from 1.5.x :)

Not wanting to sound patronising, but doc/install-configure.html already 
says this:

  * Edit /etc/group and add user daemon to the mail group. This 
will permit sendmail to run the deliver (LMTP client) program to
deliver mail to the IMAP server.

Mind you, it's under Sendmail 8.9.x - perhaps it need to be moved to an
outer scope?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: SASLAuthD keeps dying.

2002-12-31 Thread simon . brady
On 29 Dec 2002, Oliver Jones wrote:

 I think it is the TLS support of pam_ldap that is causing the core
 dump.  I ran saslauthd in a debugger and it seg faulted in
 libcrypto.so.2 which is part of openssl.
 
 The interesting thing is that all my other pam related apps arn't
 dumping core or crashing and they are all using TLS to talk to the LDAP
 server via pam_ldap.  So it must be some interaction between pam, LDAP
 and saslauthd that is causing the issue.

This doesn't directly solve your problem, but have you considered dropping 
PAM and just using saslauthd's built-in ldap support? I'm running it in 
production (2.1.10) talking to ldap://localhost, and have tested it 
successfully with ldaps://remotehost. What does PAM-LDAP buy you that the 
native mechanism doesn't?

(Although as an aside, I strongly second Henrique's advice about checking
which libraries it's picking up: I sweated several hours over a different
saslauthd problem only to find that ldd told me the answer instantly.)

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





Re: someoone are using saslauthd+ldaps?

2002-12-31 Thread simon . brady
On Sat, 28 Dec 2002, Felix Cuello wrote:

   Someone are using saslauthd + ldaps?, because my saslauthd works fine
   with LDAP but not with LDAPS...

I can confirm that it works with SASL 2.1.10 / OpenLDAP 2.0.27 on 
Solaris 8. What sort of problems are you getting? Does ldaps in general 
work from the machine, e.g. can you do an ldapsearch to ldaps://host?

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand





saslauthd performance anxiety

2002-12-31 Thread simon . brady
I've just upgraded to imapd 2.1.11, and while it's going fine so far I'm a
bit concerned about how it will cope under load when our students return.

The problem is that I'm using the saslauthd native LDAP mechanism, and the
directory it's binding to is quite slow (it's actually a slapd instance
running a shell backend which routes bind requests to different places
depending on the usercode - don't ask...). Because saslauthd makes
synchronous bind calls, it's doesn't take many concurrent requests to
exhaust the default pool of five instances.

Since it's hard to predict peak usage, I'm tempted to run the daemon with 
the -n0 option so it can spawn as required. However, a colleague has 
pointed out that if something blows up then spawn-on-demand could kill the 
server - with a fixed-size pool, auth requests would fail but the system 
would keep going.

Can anyone offer advice on tuning the saslauthd pool? Are there particular
options, either on the command line or in saslauthd.conf, which I should
be looking at?

Many thanks...
Simon

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand




Re: Question: How to specify path to saslauthd mux socket inimapd.conf?

2002-12-10 Thread simon . brady
On Mon, 9 Dec 2002, Kevin M. Myer wrote:

 What I still need to figure out is how to specify which saslauthd mux
 socket for each domain's imap process to connect to. [...] I can't seem
 to find that documented anywhere (probably because its only in this
 special case scenario that you'd even need to use it :)
 
What you're describing sounds almost like a solution to a problem I'm
grappling with, but I suspect not quite - any suggestions on the Right Way
to proceed here would be welcomed:

I'm upgrading an inherited 1.5.19 installation to 2.1.11, but have to cope 
with some local authentication madness. For historical reasons we 
authenticate to different LDAP servers depending upon the usercode (staff 
and students), and the 1.5 code has been horribly hacked to support this.

Taking the LDAP madness as read, it sounds like the way to solve this in
Cyrus is to go to impad-2_2 in CVS and use named virtual domains, with
users logging in as user@staff or user@student. Alas, that's not a
realistic short-term option for me (as a Cyrus newbie I'd rather stay away
from CVS, and the upgrade has to be invisible to users wherever possible).

So what I've been considering is applying a similar patch to
saslauthd/lak.c to choose an LDAP config based on the usercode. Obviously
I'd prefer not to mutilate the code if I could avoid it - if I instead ran
two saslauthd instances with their own configs, is there a way to make
imapd 2.1 dynamically choose a socket based on some per-user property?  
(This property needn't be the usercode - for example, we keep staff and
students in separate partitions.)

Like I said, any advice would be gratefully received!

 Also, is it reasonable to think that most major IMAP clients could
 handle talking to a server that only listens on imaps (basically my
 forcing of TLS idea above)?  I know my webmail client, IMP, can handle
 that but can most other standalone clients handle imaps well and will
 they barf over self-signed certificates?

Following up on Rob's list, my tests show that Outlook 2000, Pine 4.44 and 
Mozillia 1.0.1 complain about self-signed certs but let you use them. 

Eudora 5.2 is more interesting: they've fixed the broken STARTTLS support
that plagued 5.0/5.1 so that it really does start TLSv1 on port 143 (their
IMAPS (port 993) support is for TLSv1 with an SSLv2 hello). However, it 
will fail the connection if you're using a self-signed cert, and Qualcomm 
support doc 2323HQ tells you to go into Certificate Manager and add the 
cert to your trusted list.

Which is all nice and sensible, except that running it in Light mode you
have to open the Last SSL Info dialog to get to Cert Manager, and you
can't open that dialog until you've had a successful SSL connection.
Hmm...

--
Simon Brady mailto:[EMAIL PROTECTED]
ITS Technical Services
University of Otago, Dunedin, New Zealand