Re: sieve vacation reposnse gives no response even after end of days specified
Wesley Craig wrote: On 24 Nov 2008, at 01:51, ram wrote: Is there a patch that fixes this value to 1 Find one attached. Is this in bugzilla? I couldn't find it... If someone puts it there and marks it as a blocker, it will be fixed in the next release. :wes At that time I just reported the bug here. I wasn't even sure it was a bug. .TM. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: sieve vacation reposnse gives no response even after end of days specified
ram wrote: I have a default cyrus installation with cyrus-imapd-2.3 on centos 5 People use sieve to set their vacation responses As per the configuration no response is sent for 1 day - cat /usr/sieve/i/it/default.script require vacation; # Vacation vacation :days 1 :addresses [EMAIL PROTECTED] this is a vacation response; -- The first vacation response seems to go perfectly fine , but even after 1 day the response is not sent again What could be the issue ? how do I debug this ? http://tools.ietf.org/html/rfc5230#section-4.1 The minimum value used for this parameter is normally 1. Sites MAY define a different minimum value as long as the minimum is greater than 0. I think your server has the minimum = 3 days. See also: http://lists.andrew.cmu.edu/pipermail/info-cyrus/2007-August/026550.html .TM. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: LARGE single-system Cyrus installs?
Andrew McNamara wrote: Note that ext3 effectively does the same thing as ZFS on fsync() - because the journal layer is block based and does no know which block belongs to which file, the entire journal must be applied to the filesystem to achieve the expected fsync() symantics (at least, with data=ordered, it does). Well, does not know which block belongs to which file sounds weird. :) With data=ordered, the journal holds only metadata. If you fsync() a file, ordered means that ext3 syncs the data blocks first (with no overhead, just like any other filesystem, of course it knows what blocks to write), then the journal. Now, yes, the journal possibly contains metadata updates for other files too, and the ordered semantics requires the data blocks of those files to be synced as well, before the journal sync. I'm not sure if a fsync() flushes the whole journal or just up to the point it's necessary (that is, up to the last update on the file you're fsync()ing). data=writeback is what some (most) other journalled filesystems do. Metadata updates are allowed to hit the disk _before_ data updates. So, on fsync(), the FS writes all data blocks (still required by fsync() semantics), then the journal (or part of it), but if updates of other files metadata are included in the journal sync, there's not need to write the corresponding data blocks. They'll be written later, and they'll hit the disk _after_ the metadata changes. If power fails in between, you can have a file whose size/time is updated, but contents not. That's the problem with data=writeback, but it should be noted that's pretty normal for other journalled filesystems, too. It applies only to files that were not fsync()'ed. I think that if you're running into performance problems, and your system is doing a lot of fsync(), data=orderer is the worst option. data=journal is fsync()-friendly in one sense, it does write *everything* out, but in one nice sequential (thus extremely fast) shot. Later, data blocks will be written again to the right places. It doubles the I/O bandwith requirements, but if you have a lot of bandwidth, it may be a win. We're talking sequential write bandwidth, which is hardly a problem. data=writeback is fsync() friendly in the sense that it writes only the data blocks of the fsync()'ed file plus (all) metadata. It's the lowest overhead option. If you have a heavy sustained write traffic _and_ lots of fsync()'s, then data=writeback may be the only option. I think some people are scared by data=writeback, but they don't realize it's just what other journalled FS do. I'm not familiar with ReiserFS, it think it's metadata-only as well. data=ordered is good, for general purpose systems. For any application that uses fsync(), it's useless overhead. I've never hit performance problems, my numbers are 200 users with 2000 messages/day delivered to lmtp, _any_ decent PC handles that load easily, and I've never considered turning data=ordered to data=writeback for my filesystems. Now that I think about it, I've also forgot to set noatime after the last HW upgrade (what a luxury!). /me fires vi on /etc/fstab and adds 'noatime' .TM. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Sieve vacation
Ethariel wrote:
Hello,
I've installed cyrus-imapd (2.2.13 on MDV 2007). Sieve scripts are
working fine except 'vacation'.
User 1 has a vacation script with days: 1
User 2 sents an email to user1 it receives an answer (the correct
vacation message).
Then after User2 can send email the same day or three days after, it
will not receive anymore automatic answer.
If I delete user1 script, and recreate a new one with another vacation
alert, then it works fine for one more mail.
The Sieve draft describes 'Day' as the period between two answers, but
this value can be bypass by 'SITE' value.
Where do I check site value ? Or are they any known bugs in 2.2.13 Sieve
implementation ?
(I've searched the archive without success).
Thks to all
Ethariel
Hi,
I've been hit by the same problem, recently. I've positively verified
that sieve pushed an entry into the deliverdb (which is used to remember
whom the autoresponse has already been sent to) with a timestamp of
(now + 3 days) instead of +1.
I've been digging through the source, both of 2.2.12 (which I'm running)
and of 2.3.8. The current minimum for :days seems to be hardcoded to 3 days.
I'm not totally sure this is intented, tho. It smells like an 'unwanted
feature'. Here's what I've found:
- in the sieve source, you've got a function named
sieve_register_vacation(), which take a C struct as an argument. This
struct has 4 member, of which two have inspiring names: min_response and
max_response. The relevant part of the code follows:
if (v-min_response == 0) v-min_response = 3;
if (v-max_response == 0) v-max_response = 90;
so, if the caller doesn't specify a min_response, the default is 3.
We're here inside the sieve parser, so this min_response is what
according to the RFC is a limit set by the implementation, and it's
totally fine, per spec, for the parser to accept an user script with
:days 1 but set it to the minimum value, 3.
You can see the source here:
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/sieve/interp.c?rev=1.24
- in the same place you find the source of sievec, which is the
command-line compiler for sieve scripts. This program only parses and
compliles a sieve script, and never actually executes it. Sieve scripts
are executed by lmtpd (part of the cyrus imapd), as a step of the
delivery process. Inside sievec.c you find:
sieve_vacation_t vacation = {
0, /* min response */
0, /* max response */
(sieve_callback *) foo,/* autorespond() */
(sieve_callback *) foo /* send_response() */
};
The foo() function is a dummy one, the two callback are called only at
execution time, not at parse/compile time, of course. Note that as far
as sievec is concerned, it does not specify limits, so the library
defaults, 3 and 90, come into play.
- in the cyrus-imapd source, you find timsieved, which provides a way
for remote users to update their sieve scripts. It seems it does not use
sievec to compile the script, it links to the sieve library instead.
Unsurprisingly, you see:
sieve_vacation_t vacation = {
0, /* min response */
0, /* max response */
(sieve_callback *) foo,/* autorespond() */
(sieve_callback *) foo /* send_response() */
};
Scripts are compiled by timsieved exactly in the same way sievec does.
See:
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/timsieved/scripttest.c?rev=1.24
- now the fun starts. There's another place where you can find a sieve
parser. It's inside lmtpd. If I'm not mistaken, this parser is not used
to parse/compile scripts, as lmtpd now looks for pre-compiled scripts
only. Anyway there is it, and it also calls sieve_register_vacation(),
but now the struct is different:
/* vacation support */
sieve_vacation_t vacation = {
1, /* min response */
31, /* max response */
autorespond, /* autorespond() */
send_response, /* send_response() */
};
Now, I think this struct is not actually been used any more. But it's
interesting to note that here the min response is set to 1. This
overrides the library default, and becomes the new implementation minimum.
See:
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/imap/lmtp_sieve.c?rev=1.15
I'd hardly call this a bug, but suspect that originally the intended
behaviour of sieve inside lmtpd (that is, cyrus imapd), was to set the
minimum of :days to 1, and not 3. By moving to precompiled scripts, this
setting has been lost, since all the other compilers now relay on the
library default, which is 3.
Of course I'm not much familiar with all the source, and I may be
missing something.
.TM.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mail vanishing
On Thu, 2005-06-30 at 15:04 -0400, websrvr wrote: I'm having a problem with mail vanishing. I can send mail, recieve mail but if I try to read the mail it seems to vanish. It doesn't matter if I use POP or IMAP, the results are the same, any ideas on a solution?? http://www.chiark.greenend.org.uk/~sgtatham/bugs.html .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Tunning for large number of files in INBOX
On Thu, 2005-06-30 at 13:22 -0300, Andreas Hasenack wrote: On Wed, Jun 29, 2005 at 02:56:58PM -0600, Michael Loftis wrote: clients retrofitted to squak IMAP. Get a real IMAP client like Mulberry that takes advantage of server side sorting, threading, and searching to allow for (nearly) limitless mailboxes but not download each and every header. Is there a real IMAP client which is free software? I have seen this downloading *all* headers behaviour with every free imap client I have tried. Not that I'm suggesting it, but pine doesn't show the all headers behavior, and it's free. :-) Too bad they forgot that an imap folder can hold both messages and subfolders, and they had to add a late hack to allow the correct browsing of a Cyrus server. No client is perfect. Being an old-time user of Pine, it's always a pain to use Thunderbird or Evolution, clients so feature-full but w/o decent imap behavior: sometimes I have to switch back to Pine to be able to handle 50k+ new messages per folder in a decent time (Pine takes negligible time to open them). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Tunning for large number of files in INBOX
On Wed, 2005-06-29 at 14:56 -0600, Michael Loftis wrote: --On June 29, 2005 4:30:06 PM -0400 Joel Nimety [EMAIL PROTECTED] wrote: Hello, I'm trying to set up cyrus-imap as a backend for an email archiving solution. I'm creating one account on the imap server for each customer domain(s) we'll be archiving mail for. I'm concerned that the number of emails that will end up in each INBOX will reach some limit (ext3 fs limit, practical limit, etc.) With EXT3 there are definite limits, not hard ones, but practical ones for time to traverse/read the inode and list. Use ReiserFS. For Mail clients most can't handle big folders because many of them are just POP3/NNTP clients retrofitted to squak IMAP. Get a real IMAP client like Mulberry that takes advantage of server side sorting, threading, and searching to allow for (nearly) limitless mailboxes but not download each and every header. With ReiserFS and UFS+Hashdirs (Linux and FreeBSD respectively) I personally have many mailboxes that are well over 20k or 30k messages, and have a few in the 200k range, haven't run into any performance problems. With EXT3 I had serious problems in the 5k range, or less. When was that? AFAIK, EXT3 on Linux has indexed directory access since version 2.5.42 (11 Oct 2002). Any 2.6 kernel and any 2.4.20+ kernel includes it. You should see no problem in handling directories with 100k files (biggest IMAP folder I've seen so far was about 50k messages). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: torture continues! Another install error with cyrus imap
On Tue, 2005-06-07 at 18:15 +0100, Imran Aziz wrote: The server does not crash now, logs show no errors that can crash the serve, but still having issues to login to the server, and as I have listed in my earlier message, getting maildrop folder error. Is the mailbox actually there, on the filesystem? It should be under the directory you specified as partition-default: in your imapd.conf. Does cyradm work? I mean, are you able to see the mailboxes using the lm command in cryadm after you created them? How did you create the mailbox, BTW? Are you using an autocreate patch of some kind? Anything in the logs? Do you get a successful authentication message or a failure? Something like this: Jun 5 22:59:30 localhost imap[24937]: login: host.example.com [x.x.x.x] username CRAM-MD5+TLS User logged in .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: torture continues! Another install error with cyrus imap
On Mon, 2005-06-06 at 17:53 +0100, Imran Aziz wrote: Can anyone please save my life, and sort this out. I promise I will never come into Unix territory and stay with easy installs of windows. Since it seems you're using a RPM-based system, why don't you install the cyrus binary rpms? On Fedora Core (and on many other distributions) installing cyrus-imapd is much easier than any other Windows add-on package, since it's already on the DVD/CD and you don't even need to download or compile it. Just choose the right option at install time, or install later with rpm -i. Of course, the Fedora cyrus rpm is made to work with the rest of Fedora rpms. If you replaced some of them with third party rpms, you may run into problems. If you like binary installs, why are you trying building from source? Unexperienced users and building a large piece of software from source don't go together well, on _any_ system. I really doubt that compiling and building a package under Windows is much easier, especially one offering the broad variety of configuration options of cyrus. To be fair, you should compare apples with apples. Anyway, I've read you mail with Subject: incorrect version of Berkeley db error. How happened that cyrus was compiled with a different version of BDB? It takes having two versions installed on the same system, compiling against one and running using the other one. Why? I've had look at your former messages. I may be missing something, but don't get why you're not using the system libraries and are compiling against stuff in /usr/local in the first place. On your very first mail for example, I see: --with-bdb-libdir=/usr/local/BerkeleyDB.4.3/lib. Why are you using /usr/local/BerkeleyDB.4.3? If you're building all libraries as well, no wonder this is going to be a pain, you're making your own Linux distribution, building it from sources, and resolving dependencies along the way! I still strongly advice for using a binary rpm made for your distribution. If you really want to build from sources, try and stick using your system libraries as close as possible. Unfortunately, it seems you already installed a lot of stuff, you should now pay attention to what is found in /usr/local and what in /usr. If you run a ./configure without giving any explict path (i.e., no --with-bdb-libdir, no --with-bdb-incdir, no --with-sasl, and so on), what happens? Does it find the right things in /usr/lib and /usr/include? .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: torture continues! Another install error with cyrus imap
On Tue, 2005-06-07 at 12:50 +0100, Imran Aziz wrote: Hello Marco, Thanks a lot for sending a detailed email. The main reason that I am not using an RPM is because I have Red Hat Linux 3 ES which unfortunately does not comes with Cyrus RPMS. That answers the first question of why I am doing this. Second I need to use postfix to integrate with cyrus imap so I have to configure everything with PAM for authentication , reasons being that I am going to use Opengroupware to use Cyrus IMAP. I'm used to Fedora's cyrus RPM (which is of course a close cousin of RHL one), and it has pam. It's true that for the configuration part you have to do it alone, both for cyrus and for postfix. I use sendmail btw. Now the second issue is with the version of Berkely database on my system. I already have the RPM 4.1.25 installed on my system and I cannot uninstall it because it has dependencies on the system. So I tried to use ver 4.3.8 but had all sorts of issues with it. ON compiling with already installed RPM 4.1.25 I get errors related to the post here https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=91933#c0 which is the latest situation. So I cannot use the installed version of RPM on the system because it will not work as redhat themselves state in the above post. Pardom me, but what CPU are you using? After a quick reading, it seems that the problem is in the i386 glibc packages in the end (sorry there is too much Fedora/RedHat mixing in the threads to follow correctly all the issues at a first glance). All CPUs excluding i386 should be NPTL- enabled by now? Am I missing something? The comment from RedHat is: And what I'm saying -- to make it perfectly clear -- is that this bug will be closed WONTFIX because no Red Hat platform is non-NPTL. Are you running RHL3ES on a non-NPTL platform? I'm asking only to gain a better comprehension of the problem. Anyway, this is drifting off-topic, since it seems a bug with the CPU/kernel/glib/db4 combo that does not depend on cyrus at all (other applications linking db4 are effected as well). And accordingly to my little knowledge of linux if I try to upgrade this RPM I am not sure if the other dependent packages like Apache will crash, so now I don't know what to do. The issue surly is Berkeley DB now, but should I try to install and upgrade on the current RPM or should I install a second version of Berkley DB on an alternate location and try to compile everything with it. Oh I see your problem now. The problem is that even if you compile cyrus with the db4 installed in /usr/local (which I assume you compliled with NPTL disabled), that's not enough. You have to _run_ it with with same db4. Remember that (very likely) you're using db4 as shared libraries, and by default it will look for shared object (.so files) in the system directories first and will find a different version (see some previous message of yours). And even if the version is the same, they still are the wrong libraries (those with NPTL enabled). You need to tell your cyrus executable where to find the right .so. I'm not _that_ expert of the cyrus build process to give you the definitive answer here (I used to maintain my own rpms, but that was 2 years ago). In general, you can do either at compile time (or better link time) with a bunch of compiler options, or at run time with the LD_LIBRARY_PATH environment variable (everytime you run cyrus). Try: # export LD_LIBRARY_PATH=/usr/local/bdb/lib or wherever you installed the libdb-4.x.so files, before running your cyrus imapd. It that works, maybe someone else on the list will provide you with the right flags to get your rpath correctly set. As a wild guess, try and do: $ export LD_RUN_PATH=/usr/local/bdb/lib (same path as above, it that worked) before compliling cyrus. I've seen you added --with-sasl=/usr/local/lib to your ./configure flags. Have you build sasl as well? Why? .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: torture continues! Another install error with cyrus imap
On Tue, 2005-06-07 at 15:32 +0100, Imran Aziz wrote: Hello Marco I am going to adopt and try your suggestion. Going to built everything again with a separate version of Berkeley DB at a location like /usr/local/db4.2 and compile and link with the libraries in that specific folder. I am useing 4.2 version because postfix docs say that latest version of Berkeley DB is not supported or has problems so going to use an older version. I don't get it... Are you compiling postfix as well? Please tell us what CPU the system is running on. Any modern system should not have any NPTL related problems... Anyway, I think you have to decide if: - you consider the system db4 broken on that host, then best thing to do is to replace the system one. On the thread you read on bugzilla, someone posted links to rpms that will replace entirely your db4. Remove the old rpms and install the ones without NPTL. That may break other applications, but we're assuming db4 is broken, aren't we? So _any_ application that uses it may break any time. - or you consider the system db4 broken _for cyrus only_, then rebuild only cyrus with the new db4 and let the rest of the system work with the old one. If you see no problems on the standard postfix with the standard db4, keep using those. I think postfix and cyrus may run at the same time each with its own version of db4. All you have to do is to tell cyrus to use the one in /usr/local. In the end of all this, I'd say that using a distribution that supports your hardware may be easier of course. B-) .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: EMBARRASSING TO THE LIST: Re: *WARNING* Your EmailAccount Will Be Closed
On Wed, 2005-06-01 at 18:51 +0200, Ulf Rasch wrote: Marco Colombo wrote: [snip] Interesting theories. But they seem to backfire on you, since, according to Mr. Jeffrey T Eaton [EMAIL PROTECTED] this list _is_ closed: The list is not now, nor ever has been, open-posting. Posting is restricted to subscribers, or to those users local to cmu.edu Then it must be one of those lists that let you configure your sending address, which is good. I must have done that at subscription time, and later forgot about it. Anyway it doesn't make much a difference, as you can see. Open or close is just a policy. If you think of it as an anti spam measure, it doesn't work. Sometimes, even advanced measures fail. Anyway, given that the list is restricted, there's no point in discussing... .TM. Well, if this list is closed then maybe the subscriber with the address: [EMAIL PROTECTED] [EMAIL PROTECTED] could be dropped from the list. If you don't like their policy, please bring the issue with the list management. You know, they can be easily reached at owner-info- [EMAIL PROTECTED] as per the Sender: header. Addressing the list (or me) on this matter is pointless, since we - the other subscribers and me - are not more capable of changing list policies than you are. Your message was just received by thousands (?) people who are not able to fulfill your request, no matter what they think about it. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: EMBARRASSING TO THE LIST: Re: *WARNING* Your EmailAccount Will Be Closed
On Tue, 2005-05-31 at 11:46 -0700, Jules Agee wrote:
Kern, Tom wrote:
I'm subscribed to the postfix and spamassassin mailing lists and they are
closed.
I think those 2 lists have a lot to do with email.
This is the only list i'm on that got hit by that german spam bot..
maybe you shouldn't discount every option to filter spam as not worth the
effort or they can get around it somehow.
you'd be surprised at how far just basic checks and filters can go..
Seconded.
info-cyrus is the only list I'm subscribed to that allows posting by
non-subscribers. Maybe it's not a coincidence that it's also the only
list that I get spam viruses from on a regular basis.
This makes no sense. As I said before, it takes a close-to-zero effort
to forge headers. Subscribers-only or open, it's a matter of list
_policy_. It may have a minimal effect of reducing spam, but that's not
the point. If you want to stop spam, use a specific filter. Bypassing
the subscribers-only check is trivial.
Spam coming through the list is more likely to bypass filters since it's
origin is slightly obfuscated and the headers added by the list software
add a small measure of authenticity to the message. My Bayes filter
thinks those headers look like legit mail.
Bayesian filters do not work like that. They don't care about headers
more than any other word in the messages (they may account the fact that
a word appears in a certain header, but there's not special meaning
attached to the fact). The relevance of a word (or combination of words
if you're running something fancy), either as a spam-indicator or ham-
indicator depends on the history of _your_ setup, not on the meaning we
human beings assign to it. So it may happen that the best indicators are
NOT the usual suspects ('buy', 'viagra', ...) but something surprising.
The only thing that counts is how often a word appeared in spam and in
ham you received.
A nice reading is:
http://www.paulgraham.com/spam.html
Those filters are sensitive to quick changes in message patterns. If
some german spam suddenly starts arriving, and most of the spam you
received is not german, the filters need some time to learn the german
words that mark spam.
Anyway this is drifting off-topic.
It's not a major influence,
but it does have some effect, so I think it's reasonable for the list
admins to assume some small measure of responsibility for the junk that
gets relayed through their system.
Again, offering email services is not part of the job of running a
mailing list. If you want to filter your email, filter at your client or
on the server you read it from. A mailing-list is just one of the source
of e-mail.
If you want to filter the water you drink, do you run all over the
planet and put a filter on every water source, or do you filter it as
soon as it enters your house? Why e-mail should be different?
I'm not asking for 100% accountability, and it's not that big a deal
anyway. There will always be asshats, and there will always be a way to
screw up a list if someone's really trying. Fortunately, those real
asshats are relatively rare.
All I'm saying is that it would be nice to see measures in place that
seem to be pretty common on other lists, like restricting posting to
subscribers. What would it hurt to implement that? Why NOT?
It's just a matter of policy. Which in turn depends on the target of the
list. A list aimed at a group of people, with some implicit commitment
to some subject, such as a SIG, a devel team, and so on is way different
from a list aimed at receiving random info or help requests, or bug
reports. If all I need is some info about a program, having to go
through the whole subscribing process (exchanging mails with majordomo,
including learning the basic syntax, or the web-mail-web cycle of
mailman) is just annoyance. I'd rather look at some other software with
easier means to access information.
This very list, while named info-cyrus, is not aimed at random users.
They'll likely complain to the mail system administrator (the guy
running cyrus software) who in turn will ask the list. If you're in
charge of running such a service, subscribing to this mailing list is
just natural.
The only annoyance of a subscriber-only filter is when it's badly
implemented, and doesn't allow you to set alternate email addresses.
It's customary (at least for me) to subscribe with a different address,
but still posting with my usual one. This allows me to implement a local
redistribution list (just an alias, actually) to reach all people
interested (i.e. the cyrus admins). Should the admins change, all I have
to touch is the local list. I have about 30 lists set up like that, and
only 2 or 3 of them require the users to manually adjust their From:
before posting.
Have a nice day,
.TM.
--
/ / /
/ / / Marco Colombo
___/ ___ / / Technical Manager
Re: EMBARRASSING TO THE LIST: Re: *WARNING* Your EmailAccount Will Be Closed
On Wed, 2005-06-01 at 09:42 -0400, Etienne Goyer wrote: Marco Colombo wrote: On Tue, 2005-05-31 at 11:46 -0700, Jules Agee wrote: Seconded. info-cyrus is the only list I'm subscribed to that allows posting by non-subscribers. Maybe it's not a coincidence that it's also the only list that I get spam viruses from on a regular basis. This makes no sense. As I said before, it takes a close-to-zero effort to forge headers. Subscribers-only or open, it's a matter of list _policy_. It may have a minimal effect of reducing spam, but that's not the point. If you want to stop spam, use a specific filter. Bypassing the subscribers-only check is trivial. What you are saying is technically correct. But there is a caveat. I am subscribed to about two dozens list (more ? I lost count), and the only one I am getting spam from is info-cyrus. It's also the only one that I aware permit posting by non-subscriber. So there is a correllation, but are there a causation ? I do not know, it might be a coincidence. But it's one heck of a coincidence. Interesting theories. But they seem to backfire on you, since, according to Mr. Jeffrey T Eaton [EMAIL PROTECTED] this list _is_ closed: The list is not now, nor ever has been, open-posting. Posting is restricted to subscribers, or to those users local to cmu.edu Then it must be one of those lists that let you configure your sending address, which is good. I must have done that at subscription time, and later forgot about it. Anyway it doesn't make much a difference, as you can see. Open or close is just a policy. If you think of it as an anti spam measure, it doesn't work. Sometimes, even advanced measures fail. Anyway, given that the list is restricted, there's no point in discussing... .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: EMBARRASSING TO THE LIST: Re: *WARNING* Your Email Account Will Be Closed
On Mon, 2005-05-30 at 16:01 -0400, Mark Nernberg wrote: The problem is that this crap comes through the Cyrus lists all the time! If it were once, I'd keep my mouth shut. And, don't give me that TrendMicro/ClamAV crap -- quality attachment filtering and simply disallowing attachments to the list would nip that in the bud. Server-side global content-based filtering is silly, unless of course it's your (private) server. Users are expected to do their own filtering, otherwise they're exposed anyway. Server-side filtering (on public servers) is just false sense of security. Moreover, this is not a US-only list. In some countries unattended filtering of messages addressed to third parties might be illegal per se, virus, spam or whatever included (at most, you can add a header to the message if you think it's spam or virus, but you _have_ to deliver it). Mailing lists may be special cases (think of moderators), but still you should never rely on the server, you should implement a strong client- side filter anyway. And once you have a strong local filter, additional security provided by a remote (weaker) one is negligible. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: EMBARRASSING TO THE LIST: Re: *WARNING* Your Email Account Will Be Closed
On Tue, 2005-05-31 at 09:59 -0400, Joseph Brennan wrote: --On Tuesday, May 31, 2005 11:47 AM +0200 Marco Colombo [EMAIL PROTECTED] wrote: Server-side global content-based filtering is silly, unless of course it's your (private) server. Users are expected to do their own filtering, otherwise they're exposed anyway. Server-side filtering (on public servers) is just false sense of security. I strongly disagree. Users just want spam to go away. They do not want to configure filters. They're not very good at it either: they usually just add the sender address to a blacklist, and that does almost nothing for them. It's not a security issue. It's annoyance reduction. Configure? Manual blacklisting? What are you referring to? I've being using both Evolution and Thunderbird, and both filter SPAM (and thus most viruses of course) like a charm, and I've configured nothing. All I have to do is to hit 'Junk' instead of 'Delete' (like I used do to before having any filter) on spam. Once in a while, I quickly look at the Junk folder, and very rarely recover a false positive. No configuration needed at all. Anyway it seems we have a different meaning for 'users'. If you mean employees of a company, well for sure they'll get filters on their (company) server. If you mean customers of an ISP, they may get filtering as well (but I'd prefer marking, or storing to a special folder, instead of silently dropping). My point being: the purpose of the mailing list software is not to provide a safe email service to 'customers' or 'employees'. That's someone else's job. The software might place restrictions (on message size, attachments, and so on) but it's only to enforce _list_ policies, not end-user security (or comfort). For example, a list with 100,000 subscribers may sensibly avoid forwarding 10MB in a single message. But that's another matter. If this list could possibly restrict posting to subscribers that would go a long way. That is pretty routine for lists. And pretty useless. Address forging can be easily automated. More than 1/2 of the spam I see on our servers already forges the sender domain. A nice fraction of it learned how to forge our staff's address already, so I got some forged messeges telling me that _I_ have locked my own account down, for example. As for it being 'routine', I'm currently subscribed to about 20 lists, and only 2 of them are subscribers-only. Not surprisingly, both have nothing to do with e-mail software. IMHO, any list that aims at random users (info, bug reports, and so on), should minimize the annoyance of posting a single message. It may be different for -devel or SIGs lists, tho. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: [info-cyrus] put on the subject line. was: spam
On Tue, 2005-05-17 at 07:43 -0500, Greg Harris wrote: My MacOSX Mail application cannot sort mail from here into the right postboxes. Then you must not be doing it right. Any message coming through the list has in the headers: Sender: [EMAIL PROTECTED] I haven't had a problem with sorting on that with any tool that I have tried. While I see how some people may want to have some special identifier in the Subject of the message, I think the claims that they can't get things auto-sorted are exagerations. More often it's the case they don't want to auto-sort, but be able to quickly pick things out of their unsorted incoming message list. Getting the Subject line tags right can be a problem too. Some clients mess up various replies so you get things like Re: [info-cyrus] RE: [info-cyrus] FWD:Re: [info-cyrus] put on the subject line. -_Gene While I agree with you that sorting is probably possible on any client, it is really nice when the tags are in place. They make things much cleaner when the occasional e-mail jumps through the hoops because of a slight change in the addressing scheme. If I can set my filter to look for any instance of [Info-Cyrus] in the subject line, then everything that is supposed to go to that folder goes and nothing else. While some subject lines do get a little messy, it is easier for a person to jump past all of the junk in the subject line than to move messages between folders. This is the only list that I know of that does not add a tag to the subject line and I have always thought that it was a little weird and wished it was there. Well, it would be the only one that _does_ for me. There are some (mailman based) lists that by default do Subject: mangling, but luckily you can turn that off. I'm against any kind of header mangling by list software, unless there's a technical reason for it (like Sender:, which does make sense to change). If anyone is going to change current behaviour, please make it so it's user configurable. And I really can't think of any reason why those occasional e-mail jumps through the hoop should ever change the Sender: header. But I'm all for a List-Id: header if Sender: is not enough/good for most uses. Just my 0.02c. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Auto-Archiving, and Unread Threshold Alert
On Tue, 2005-05-10 at 22:41 +0800, Senandung Mendonan wrote: Hi, I need to do the following in Cyrus IMAP 2.2.12:- 1. Daily auto-archiving - On 12am each day, an IMAP/Cyrus agent runs through each user's mailbox, and sort out all existing mails into today, yesterday, last week, and last month folders. The best way to achieve this, by far, is client-side. See VFolders in evolution (those are not real IMAP folders, and the server is totally unaware of them; it's only a matter client interface). But I understand sometimes you can't choose the client and you have to band-aid on the server side. 2. Unread threshold alerts - Every 5 minutes, an IMAP/Cyrus agent runs through each user's mailbox (or any other alternative ways), and upon finding new mails that are unread since last poll, alert the sysadmin. 5 mins? Wow. Are you sure you need this? The implications are: - time to read and process a single message is 5 mins - rate of message arrivals is predictable, and is the rate at which messages are processed - ever; - _all_ messages are to be processed (or deleted), no matter if they are not relevant with the activity (based on the subject) - no one can say ok, this one i'll read later. It the above conditions are not strictly met, all you have is a nice entropy generator (the sysadmin will receive almost random and useless notifications). Anyway, the quickest way to implement your agent is a Perl or Python script. Here's one (quick and dirty) that prints the number of messages in each user's INBOX that are both unread and older than 5 minutes. It assumes you can do PLAIN authentication over SSL to imapd running on localhost: #!/usr/bin/env python import sys import re import imaplib import time import email.Utils # authentication credentials for the admin user auth = admin password = *** def check_one(user, threshold = 300): conn = imaplib.IMAP4_SSL() rc, data = conn.authenticate(PLAIN, lambda x: \0.join((user, auth, password))) if rc != OK: sys.exit(1) rc, data = conn.select(readonly = 1)# INBOX if rc != OK: sys.exit(1) rc, data = conn.uid(SEARCH, UNSEEN) if rc != OK: sys.exit(1) uids = ,.join(data[0].split()) if not uids: return rs, data = conn.uid(FETCH, uids, INTERNALDATE) if rc != OK: sys.exit(1) patt = re.compile(r'\(UID ([0-9]+) INTERNALDATE ([^]*)\)') unread_messages = [] for line in data: m = patt.search(line) if not m: continue uid = m.group(1) internaldate_mk = email.Utils.parsedate(m.group(2)) if not internaldate_mk: continue internaldate_ts = time.mktime(internaldate_mk) now_ts = time.time() if internaldate_ts now_ts - threshold: unread_messages.append(uid) conn.logout() return unread_messages # the list of users to check users = (user1, user2, user3) for user in users: mesgs = check_one(user) if mesgs: print user %s has %d messages. % (user, len(check_one(user))) You can do the same in Perl of course. And moving messages around should be possible too, for your daily auto-archiving. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Auto-Archiving, and Unread Threshold Alert
On Wed, 2005-05-11 at 14:38 +0200, Marco Colombo wrote: [...] for user in users: mesgs = check_one(user) if mesgs: print user %s has %d messages. % (user, len(check_one(user))) ^^^ Well, make the last line look like this, there's no need to check the same user twice: print user %s has %d messages. % (user, len(mesgs)) .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [POLL] unexpunge behavior
On Tue, 2005-05-03 at 15:15 -0400, Derrick J Brashear wrote: On Tue, 3 May 2005, Ken Murchison wrote: Igor Brezac wrote: How does quota work? I assume that a 'deleted message' (before it is cyr_expire'd) does not count against quota. Correct. And since the user can't get it back without intervention it's not secretly limitless quota, though it does make knowing how much disk space you've allocated less deterministic. What prevents an user from exceeding his quota by a huge amount of space? Say I create 100MB of messages, delete them, and repeat? True, sooner or later the expire daemon will kick in, but what about if the user is fast enough to fill your disk up _before_ that happens? I perfectly understand that the other way (deleted messages counted) prevents the user from freeing space in his mailbox in a timely manner. Maybe an extra quota? Something like the deleted space is 1/2 or the same of the user quota. That would put a limit on the allocated space (1.5 or 2 times the quota value). Most users won't notice, and runaway (or malicious) cases have a predictable limitation. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus POP3 Issue
Rob Siemborski wrote: On Fri, 11 Mar 2005, Marco Colombo wrote: Ok technically speaking SSL/TLS is not part of SASL. But the two are related. Maybe I'm biased by the fact that most of the connections I see are SSL+plaintext. So I was referring to SSL keys actually. Sure, or, say, kerberos keys. For what SASL is using it for, its a far lesser sin. I have to say I'm not familiar with CRAM-MD5/DIGEST-MD5. But in the latter the channel can be encrypted, so I guess at some point a shared session key is generated. Yes, there is a session key here, but the information it is based off of is the nonces (as I said, they need to be sent in the clear anyway, so coming from urandom doesn't matter that much), the shared secret, and some static text. See RFC 2831. Well, now I _am_ confused. The nonce is sent in plain, but the RFC says: The contents of the nonce are implementation dependent. The security of the implementation depends on a good choice. It is RECOMMENDED that it contain at least 64 bits of entropy. So, the RFC is saying you'd better get the nonce value (and cnonce too) from /dev/random, at least 8 bytes. Then you can encode them in hex or base64 (which will result in longer string of course, but still with 64 bits of entropy in it). As I read it, it is _not_ recommended to use /dev/urandom, which may let you generate a nonce value with 0 bits of entropy. That's true also for the client and cnonce value. Now the problem is different. I agree someone attacking the kernel PRNG is not pratical a threat. But if you want to confrom to RFC 2831, you need to understand the full implications: [RFC 2119] 3. SHOULD This word, or the adjective RECOMMENDED, mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course. Now, can you claim conformance to RFC 2831 if you're using /dev/urandom? Does the fact that your cyrus server is heavily used fall under those particular circumstances? Or is it normal operations, instead? What are the valid reasons you found not to use /dev/random, in your _particular_ case? In general: we've got here a security mechanism that apparently is, by design, based on a combination of entropy and crypto techniques. If we substitute the entropy with something that is protected by another crypto technique, what claims can we make on the final result? Do the two different techniques mix together in a cryptographicly secure way? Or have we introduced some subtle weakness in the system? The brief history of computer cryptography is full of examples of combinations of different algorithms with no added strengh or, even worse, less strengh than the components alone. Now, I'm not being theoretical. I'm being bureaucratic. :-) As far as I'm concerned, the whole purpose of this thread was to make steps towards fully understanding the implications. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus POP3 Issue
Rob Siemborski wrote: On Mon, 14 Mar 2005, Marco Colombo wrote: Now, can you claim conformance to RFC 2831 if you're using /dev/urandom? Does the fact that your cyrus server is heavily used fall under those particular circumstances? Or is it normal operations, instead? What are the valid reasons you found not to use /dev/random, in your _particular_ case? That the server will basicly fail to function if /dev/random is blocks indefinately? If a site feels they need more entropy, they can always use /dev/random (or any other source of entropy). We originally had that as a default configuration, but in a large number of deployments, it caused more problems than it solved. I understand that. I've replied only because you mentioned RFC 2831. I'm not happy to hear there is a 'large number of deployments' where RFC 2831 recommandation is violated. The admins of those site should consider either getting more resources (entropy, in this case) or stop running any strong but demanding SASL mechanism (or SSL/TLS). Once again, by definition, a large number does not mix well with the particular circumstances mentioned in the RFC. The problem of those site was not our default configuration. Their problem is that they're using a solution they can't afford. Running out of entropy should raise a flag to them, and make them switch to less demanding mechs, or consider an upgrade. What's the point in using any strong auth mech in a way that violates its RFC recommandations? Moreover, is it ok for any software having a _default_ configuration that acts against some RFCs? Now we have a large number of sites that runs a stripped-down version of DIGEST-MD5 that RFC 2831 _barely_ allows. I just think they should be made aware of the implications _before_ they do. Let their servers block, let them learn why, let them ponder on implications, and let them make a decision. Be it not using strong mechs, or getting a decent source of entropy, or using /dev/urandom I dont' care. Just let them hit the problem and decide. Having said that, now I'll let this thread die, I promise. :-) .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus POP3 Issue
Rob Siemborski wrote: SASL doesn't generate *keys* using this, it generates *nonces*, which are known to the attacker anyway, since they are transmitted in the clear anyway. It just matters that they don't repeat often enough to bother precomputing values for. If SASL was using this for key generation, then yes, most of the comments in this thread have merit. Ok technically speaking SSL/TLS is not part of SASL. But the two are related. Maybe I'm biased by the fact that most of the connections I see are SSL+plaintext. So I was referring to SSL keys actually. I have to say I'm not familiar with CRAM-MD5/DIGEST-MD5. But in the latter the channel can be encrypted, so I guess at some point a shared session key is generated. -Rob (Hmmm, its possible that the SRP plugin is using this for something else, I'm not familiar enough with SRP and would have to ask Ken). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus POP3 Issue
unnamed OSes used a few years ago. :-) the problem is not to predict the output. Once we know the state, the output is _computable_. The problem is knowing the state by observing the output. Given numbers N1, N2, and N3, say, how many possible states may produce that sequence? I state the number is low. If it were big, the sequence (N1, N2, N3) would be _very_ likely to happen, and that's not a good PRNG for sure. [...] At least so far. Someone was talking about removing that useless feature, so keep an eye for it to make sure they don't. Well something must be done about the feature. If all you need is a PRNG whose seed it randomly changed now and then (ala /dev/urandom, when the pool is almost empty), you'd better implement your own in user space, and _always_ use that. I'd rather have it fixed where it will do the most good, which is in the kernel. I don't want to see what happens in the Windows world happening in the Unix world... Let me explain better. Let's say your cyrus server needs 100k/bits random bits per second. The kernel gets 25kbit/s (_you_ should underestimate them as well, btw, adding less to the counter). So, every 10 bits That source has H 0.998, and I am already underestimating it to H=0.99, which is good enough for my needs. I could be really paranoid and use 0.75 or even 0.5, but it is a slow source, so that would hurt. Interesting enough. :-) How do you know the real H? How do you measure it? Are you sure you're underestimating it? But are you talking about a fully-compromised /dev/urandom above, or about a non-compromised /dev/urandom? For a fully-compromised urandom, I agree with you. All it takes is to break SHA1. Ok, recent Linux kernel has a complicated multi-level PRNG. But it's just a kind of (not-so-tested) crypto. I'm not sure it counts as additionaly security. [...] Secondly, assuming you're fine with it, the question is: why don't you _always_ drain 25kbit/s only, if that is enough for you? I draw less than that most of the time, and for *my* case telling SASL to use /dev/random is fine. Heck, I have always about 100k bits of entropy bufferized and waiting for a load spike, and I could easily make that 100M bits if I needed to and wanted to waste that much real memory (pages with random data are obviously locked and cannot be swapped). Why? You can have an userspace daemon buffer it. Writes on /dev/random block as well, if I recall right. The daemon will awake everytime the entropy count is low. [...] Why don't you use your private PRNG, extract 100kbit/s from it, and get only 25kbit/s from the kernel, _always_? By using /dev/urandom, you're asking for more than, by definition, you need. Good point. But I would actually argue that we should enhance /dev/urandom to be configurable to be able to do exactly that. Why bloat the kernel with something that can easily be userspace? The only kernel functionality it needs is already provided by /dev/random. So yes, I am one who thinks that /dev/urandom is a useless feature (almost). Run a user space daemon instead, which applications can register to, and ask for the quality they need, and no more. I'd say that /dev/urandom *would* be an useless feature if we could trust people to know what they are doing when writing their PRNGs. We cannot, so we better improve urandom to have better resource reservation semanthics. It *could* be teached to set a maximum output of real entropy per calling process, if one is willing to write the code. That would fix the issue you talk about. And IMHO it would be a safer (in the real world) way to go for it. But yes, it does not have to be done in kernel space. In which case we are probably better off removing urandom or making it a named pipe managed by that user space daemon. *And* teaching SASL to use the daemon instead of mucking with /dev/random (and there we go arguing it all again ;-) ). No, here we agree. :-) It's _your_ system, _you_ are setting policies. The deamon would provide SASL with _exacly_ the level of security you want/are fine with. I think we're slightly drifting off-topic. I'd end this thread here if you don't mind. For most cases, a DoS is worse, in practice, than a remote chance of total break at SASL/SSL level. An interesting reading is: http://www.ussg.iu.edu/hypermail/linux/kernel/0208.2/0347.html and the whole thread (both previous and following mails). Yes, the idea of userspace /dev/urandom is not new. :-) Years ago I've played with /dev/random stuff. The result was this project: http://freshmeat.net/projects/random_tools/ AFAIK, no one uses it, and it's unmaintained at all. Modern PCs seem to collect enough entropy for my needs. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http
Re: Cyrus POP3 Issue
Henrique de Moraes Holschuh wrote: On Thu, 03 Mar 2005, L. Mark Stone wrote: The POP server component is giving us a problem. It often fails to respond to connection requests in a timely manner, if at all. IMAP Disable APOP, or get SASL to use /dev/urandom like it should be doing in any sane distribution (SASL is not generating long-term keys which would be a good reason to use /dev/random). You do want to use /dev/random for your session keys. _They_ are likely going to attack your session keys, not your master key. The whole point is to guess as many bits as possible of the kernel PRNG state. /dev/random blocks when the kernel thinks the PRNG state could have already been guessed, in pure theory. So the attacker would be able guess its _next_ output. By reading from /dev/urandom, you're accepting the risk (the output is protected by a crypto-strong hash function, such as SHA1, anyway - but you have to trust it). Once the attacker knows the output of your /dev/urandom, he knows all the session keys your host will generate. Note that since you've generated the master keys offline, in an unknown time on an unknown computer, the attacker knows _nothing_ about the PRNG state of the machine you used. You could have used /dev/urandom for them w/o any risk, unless the attacker was able to observe you in that moment[*]! The only thing that could help him is a bug in the PRNG and the hash function, so that some bit strings are impossible (or unlikely) to be generated. That would allow him to discard them in a brute force search. It happened in the past, some software used to generate 128 bit keys was buggy and the actual number of possible combinations was much much smaller (say 2^20). But the same PRNG and the same hash function are used by /dev/random as well, so it isn't much safer in this respect. If your kernel runs out of random bits, you should find a reliable source of randomness and feed it into the kernel. .TM. [*] assuming the keys you generated are not longer than the kernel internal pool, in bits. If the PRNG state is 4096 bits, and you are generating a 8192-bit key out of /dev/urandom, the attacker needs only to try 2^4096 possible combinations, not 2^8192. Usually the kernel pool is large enough (and can be enlarged at run time in modern Linux kernels). -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus POP3 Issue
First of all, let me state that I don't really believe that attacking the internal state of the kernel PRNG is pratical at all. It is possible, in theory. Using /dev/urandom is pretty safe for many real-world uses. We're discussing theory here. Henrique de Moraes Holschuh wrote: On Fri, 04 Mar 2005, Marco Colombo wrote: You do want to use /dev/random for your session keys. _They_ are likely going to attack your session keys, not your master key. The whole point is to guess as many bits as possible of the kernel PRNG state. Which, in a Cyrus server, won't be helpful since there is a lot of disk activity and the PRNG state changes quite often, provided you don't keep it drained (and thus without activity because nobody can talk to the server!). Using random instead of urandom goes a long way into helping to keep the pool drained (the kernels usually protect a small amount of entropy won't let urandom drain it completely). Well, the problem is simple. Let's say your server generated 100 session keys in the last minute. If the system collected at least 100*key size random bits, you could have used /dev/random. If it not, and you used /dev/urandom, the attacker has fewer bits to guess. Let E be this number. Assuming that he knows the pool state _now_ (that is, he collects enough keys and is able to break SHA1 and reverse the PRNG algorithm to obtain its state), in order to get the pool state _one minute_ ago, he has to guess E bits only, which it is easier than guessing all 100 keys. Every _single_ bit you got out of /dev/urandom for your session keys when /dev/random would have blocked _halves_ the attacker's effort. /dev/random blocks when the kernel thinks the PRNG state could have already been guessed, in pure theory. So the attacker would be able guess The hard facts are that most kernel entropy accounting is not anywhere close to exact enough, so we cannot rely on it on anything close to a border condition (such as the entropy pool being empty, which is the only time urandom differs too much from random). You *are* already depending on the security of the PRNG for small ammounts of time (when the entropy in the pool is close to zero). Well, the accounting needs not to be accurate at all. _Underestimating_ the randomness in the pool is perfectly safe, and that's what the kernel does. You can add 1024 bits of truly random data to the pool, and account only 128 entropy bits. So, even if your truly random bits were not so random, you're ok, as long as _at least_ 128 were truly random. Since there's no way to make sure bits are truly random at all (all tests may fail, all devices may be somehow biased, and so on) you can only reach some level of confidence. Extracting N bits from a good source, injecting them into the pool, and accounting only M entropy bits, where MN, is the only thing you can do. The smaller M, the better. It's a tradeoff: your confidence vs. the cost of those N bits. The kernel is quite conservative in estimating randomness, that's why the pool empties so easily, expecially on headless servers. And no, you're not depending on the security of the PRNG+hash when you're using /dev/random, assuming they're not buggy. Given that fact, IMHO you are much better using urandom anyway, because the system will still work enough close to an empty entropy pool to regen that pool a bit. I'm not following you. Unless you're implying that the only source of randomness is the result of the activity the _needs_ the random bits. Such as a dedicated headleass cyrus server with no other entropy source but disk activity. If all new connections blocks on /dev/random, the disk activity will be lower. Well, first I think disk activity is much bigger for already connected clients. I know some clients out there make a new connections to the server everytime they check for new mail (once a minute?) but those are broken clients. I agree that with those, to block connections means to block activity. But most clients I know open _one_ connection to the server and keep it open. Sane clients generate most of their activity well after the connection has been established (and the session key exchanged). Blocking connections does not block their activity at all. Second, giving up security only to squeeze some more entropy bits from your disks isn't a winning strategy, IMHO. anyway - but you have to trust it). Once the attacker knows the output of your /dev/urandom, he knows all the session keys your host will generate. If, and only if, you have no new entropy going in. If you have a few bits of entropy going in, they have a reduced search space (which IS already a very bad thing)... unless the kernel does a catastrophic reseed of the PRNG. Which on newer Linux kernels for example, it does as often as it possibly can. I know that every single bit of entropy going in changes the state, but the attacker needs only to try the possible values. The reduced search space _is_ the problem. Whatever the kernel does
Re: IMAP Implementation People Time Estimates
Christopher T. Beers wrote: We are in the process of evaluating our current student email solution here at the University and I have been asked to develop some estimates of the time it takes to research, test and implement this. The new system must support 35,000 users with about 1500 concurrent connections. More than likely the setup will have redundant web front ends (2 servers running SquirrelMail, IMP, etc), 1 mupdate server, 2 frontend machines and 2 backend Cyrus mail stores. It will run on top of Linux (RHEL 4 probably). We are familiar with UW IMAP server so we have to learn the differences to go to Cyrus, especially the aggregator, backup/recovery, cyradm, etc. So keep that in mind. If anyone has any data that would even remotely be helpful I would greatly appreciate it. We are comparing Exchange, MiraPoint and this (which is much cheaper) but people now want to compare accurate estimates of people resources to implement it. To me, that is like asking me how long it would take me to learn [place any abstract task here that you have never done]. Precisely. Learning time depends 20% on how complex the task is and 80% on how skilled the people involved are. I understand you're having an hard time extimating the first figure, since you don't know cyrus-imapd, but how do you expect us to extimate the second one? Not to mention resources depend also on how long you are planning to test the system, before you feel comfortable enough to put it in production. Moreover, I'm sure the cyrus solution offers _a lot_ of different options, and you may want to evaluate them carefully. Just think of the different authentication models: are you going to use LDAP? Kerberos? Choosing the right options is key for a stable system and happy users. Most commercial softwares do not offer the same amount of options. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [RFC] EXTERNAL auth choosing between CN and email address?
Kevin P. Fleming wrote: Marco Colombo wrote: What field is that, exaclty? v3 extension? I'm not sure... it's in the OpenSSL headers files as NID_pkcs9_emailAddress. Oh, I know nothing of OpenSSL API. It seems too me (but I'm not sure) it's the emailAddress attribute in the DN. Some time ago I did some research, and found this: RFC 2459 4.1.2.6 Subject [...] In addition, legacy implementations exist where an RFC 822 name is embedded in the subject distinguished name as an EmailAddress attribute. The attribute value for EmailAddress is of type IA5String to permit inclusion of the character '@', which is not part of the PrintableString character set. EmailAddress attribute values are not case sensitive (e.g., [EMAIL PROTECTED] is the same as [EMAIL PROTECTED]). Conforming implementations generating new certificates with electronic mail addresses MUST use the rfc822Name in the subject alternative name field (see sec. 4.2.1.7) to describe such identities. Simultaneous inclusion of the EmailAddress attribute in the subject distinguished name to support legacy implementations is deprecated but permitted. So it seems its usage is deprecated. If you are to code a patch, you may look into the alternative name(s). Those are standard v3 extensions. As I understand it, comforming applications should look there in order to find email addresses (of type rfc822Name). Of course, since you're using your own CA, you could use whatever field/attribute, but keeping an eye on standards won't hurt, IMHO. And after all your own mail was an RFC. :-) .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Compacting mailboxes...
Forrest Aldrich wrote: I'm concerned about disk space consumption of these messages in-limbo. Is there some mechanism that can force a compact of the mailbox to remove these deleted messages - or is there another method to manage this scenario? Use quotas as a LART. http://www.catb.org/~esr/jargon/html/L/LART.html As soon as users realize they mailboxes are full because their client is not compacting folders, they'll learn how to configure it properly. A help page with directions for common clients might help, tho. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [RFC] EXTERNAL auth choosing between CN and email address?
On Thu, 24 Feb 2005, Kevin P. Fleming wrote: I'm working on a webmail system using client certificates for authentication. I have Cyrus IMAP working fine with Cyrus SASL and AUTH=EXTERNAL after negotiating TLS... the IMAP daemon authenticate the user properly. However, it chooses the CN from the client cert as the authentication identity. With a bit of hacking to imap/tls.c I was able to convince it to use the email address instead, but I'd rather not keep it this way... ^ What field is that, exaclty? v3 extension? Anyway, the goal of authentication is to identify users not email addresses. The whole idea of using certs is broken, unless you use the cert itself. No CA makes any attempt to provide _unique_ information. And the uniqueness of an email address it pretty weak. The only unique info you can extract from a cert is the public key, which is what you're actually using to identify the remote party. There should be a way to associate public keys with cyrus usernames. Of course, if your server trust only _one_ CA, and you have control on how that CA works, you can use certs safely. You can make sure CN data (or any data) is unique. BTW, I've used EXTERNAL myself, but only for lmtp, and to identify servers. And I used an internal CA. CN was server name, and I'm pretty sure there's no other cert with that CN data. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus in ISP environment?
Jure Pe_ar wrote: On Wed, 16 Feb 2005 16:21:09 +0100 Attila Nagy [EMAIL PROTECTED] wrote: Amavisd was slow like hell, but cyrus could easily put email down to disk at a rate of 10-15 MBps. Take the above numbers with a grain of salt, because the testing was pretty lame 10-15MBps ... then add a few hundred concurrent pop imap sessions plus some monitoring/statistical script walking your spool doing various operations and see this number fall down dramatically ... Because with random i/o ops you increase time disk heads travel around and add latency to the whole setup. The only thing that helps here is having lots and lots of disks or hw raid controllers with nice big caches. Just what 'lots' and 'big' means depends much on your actual needs. Consider splitting the SMPT incoming part from the IMAP/POP serving one. Have the SMTP server receive, queue, scan messages. Once messages are in the queue, use a queue runner to deliver message the IMAP server via LMTP. Unless you are willing to accept mail for unknow users (and discover that later at LMTP level) you may need to teach your SMTP server how to recognize valid users. The splitted approach scales much better, and reduces I/O load a lot (as you said, it's the _combined_ I/O load that hurts you). You can identify the overloaded part easily, and take countermeasures. Adding a second SMTP server should be easy. Adding a second IMAP backend can be easy or hard depending on your setup (e.g., if you can split messages based on, say, domain, it's just a matter of configuring your MTA to deliver to the right LMTP server for a given domain). The only difference from a standard installation (where both the MTA and IMAP servers run on the same host) is LMTP authentication (you need to set it up - the defaul preauthentication based on Unix permissions of the local socket won't do of course) and authorization (the MTA needs write access to all mailboxes). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus, NFS and mail spools
Terry.Poperszky wrote: With Courier, I have the ability to spread access to the mail spool directory across several incoming smtp servers, is Cyrus able to do something similar? What I am referring too, is having multiple incoming email servers, with their mail spools being NFS mounts to a single box using Maildir format, access to which is served by courier Imap. This configuration belongs to a local ISP and I am looking at porting it to my corporate network, but I want to look at other options than Courier. Thanks Is NFS a requirement? If you need to split the SMTP part, you can run multiple instances of any SMTP server (e.g. sendmail), all of them delivering mail via LMTP to a single server running Cyrus. Cleaner, much more efficent and scalable, IMHO. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Why doesn't Cyrus put mail into boxes?
Mike Jones wrote: On Mon, 2004-08-16 at 15:37, Mike Jones wrote: [...] However, no mail ends up in the mailboxes underneath the /var/spool/cyrus/mail directory. [...] Start with the logs. What does Postfix tell you? I think on Debian the logging goes in /var/log/mail.log - does it look like Postfix has successfully handed off to Cyrus? Thanks for responding. This is what regularly appears in the mail.log file: Aug 16 18:35:20 hostname master[15129]: about to exec /usr/sbin/ctl_cyrusdb Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: checkpointing cyrus databases Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: archiving database file: /var/lib/ cyrus/annotations.db Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: archiving log file: /var/lib/cyrus/ db/log.01 Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: archiving database file: /var/lib/ cyrus/mailboxes.db Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: archiving log file: /var/lib/cyrus/ db/log.01 Aug 16 18:35:20 hostname ctl_cyrusdb[15129]: done checkpointing cyrus databases Aug 16 18:37:28 hostname master[15130]: about to exec /usr/lib/cyrus/bin/pop3d Aug 16 18:38:46 hostname master[15133]: about to exec /usr/lib/cyrus/bin/pop3d Aug 16 18:40:04 hostname master[15139]: about to exec /usr/lib/cyrus/bin/pop3d Aug 16 18:40:46 hostname master[15141]: about to exec /usr/lib/cyrus/bin/pop3d Aug 16 18:40:47 hostname pop3[15141]: login: remote client cyrus plaintext User logged in Aug 16 18:40:50 hostname pop3[15141]: Unable to locate maildrop for cyrus: Mailbox does not exist Aug 16 18:42:36 hostname master[15142]: about to exec /usr/lib/cyrus/bin/pop3d Aug 16 18:45:36 hostname master[15144]: about to exec /usr/lib/cyrus/bin/imapd Aug 16 18:46:22 hostname master[15150]: about to exec /usr/lib/cyrus/bin/imapd Aug 16 18:46:27 hostname imap[15150]: login: hostname cyrus plaintext User logged in An exception to the regularity is the 18:40:50 intry, which was caused by a message sent to cyrus@mailserver. There is no Cyrus mailbox, so that went all right. Hence I assume this is the normal sequence of log entries you would expect? Does this mean that the mailbox directory structure is simply a 'database' of mailboxes only used for look-up purposes? No. AFAIK, there's a separate db for mailboxes. But of course, it has to agree with the directory structure for everything to work. What exactly do the authors mean by 'checkpointing'? I don't know exaclty, but cyrus manages a set of databases, not only the mailboxes one. You can choose their format at ./configure time. One possible format is BerkeleyDB, and I think 'checkpoint' is normal maintainance for BDB. Of course, database checkpointing is a general concept, and it may apply to other formats as well. Please see: http://www.sleepycat.com/docs/api_c/txn_checkpoint.html How do the cyrus user names correlate with email addresses? I wonder if this is where the problem is - is mailbox 'peter' the same as peter@mailhostname or peter@domain ? What controls that? That's the job of the MTA (Postfix). It accepts a message, check if it's local, if so does all the local processing (aliases to name one thing), and ends up with one or more local mailboxes to deliver the message to. The message along with the list of mailboxes is handed to cyrus. Assistance much appreciated. I would like to get this set-up to work :-). Thanks Mike .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: URGENT: Naming conflict in Cyrus-IMAP 2.2 vs. leafnode 1.9
[recipient list purged] On Mon, 3 May 2004, Rob Siemborski wrote: On Mon, 3 May 2004, Henrique de Moraes Holschuh wrote: I appreciate the problems with the namespace conflict, but if we were to do this for all of our binaries every time a conflict was discovered, I There is one answer for that, which is to prefix everything that is not a service with cyr_ for 2.3, and have all services installed in a separate directory (which is already possible, but it is not strongly suggested or anything). After all, services are to be run by master, only... I'm not a big fan of prefixes for executables: we already have file namespace separation thanks to directories. Not precisely knowing _where_ to put them is not an excuse. /usr/libexec/cyrus-imapd seems fine to me. Most executables are neither general purpose (/usr/bin) nor for sysops (/usr/sbin) as they are required to run as the 'cyrus' (or whatever) user. Adding /use/libexec/cyrus-imapd to $PATH for the cyrus user only sounds sane enough. If a command can be sensibly used by any user, it belongs to the system bin directory, but the prefix in the name is natural in this case: cyradm comes to mind, it's so natural that it already has it. B-) The problem is more the man pages, really (as it is in this case). That's close to impossible to solve for developers. It has to be dealt at package creation time (sed/perl). It's more a deficiency in the old glorious UNIX man system than anything else. But recent man executables have provisions to handle this nicely btw, just few distros fully make use of them. man on fedora has MANPATH and MANPATH_MAP. MANPATH_MAP fits the separate directory approach well. Just install manpages in /usr/share/cyrus-imapd/man or something. Just my 2c. FWIW, Bug 2425 is open now to address this. I'm not sure when a good time to do so is, but certainly 2.3 is the earliest possible time. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: Cyrus logging, /var/log/imapd.log
On Thu, 15 Apr 2004, Doug Koobs wrote: Hi, I'm using 2.2.3 and had no trouble with syslogd, I use local6.* instead of local6.debug. By the way local0 through local7 are made available by syslog to applications wishing to have a separate log file (man syslog.conf should tell you more). If local6.* doesn't change anything you should try to restart syslogd using /etc/init.d/syslog restart. Hope this help. françois Francois, Thanks for the tip. I changed the lcal6.debug to local.* and restarted syslog, but still nothing got written to imapd.log. I read the man page for syslog, and it explains what the localx entries are for, but not where they are defined. Anyone know where it is determined that local6 is logging for cyrus? Well, you may want to check if your syslog is doing its job correctly first: $ logger -i -p local6.info this is just a test the message should appear in the right file. If not, you've got a problem with your syslog configuration. There is no 'definition' of what goes into which localx, or any other facility. Applications ask for a certain message to be logged with a certain facility and a certain priority tagged to it. That's all. As for where local6 is defined in cyrus: $ cd cyrus-imapd-2.2.3 # this is the first tree i stepped into $ ./configure --help | fgrep syslog --with-syslogfacility=FACILITY set the syslog facility to use (default LOCAL6) It's a compile time option you can set with configure. In some previous releases, LOCAL6 was simply hardcoded if I remember well. Doug Confidential: This electronic message and all contents contain information from Financial Credit Services which may be privileged, confidential or otherwise protected from disclosure. The information is intended to be for the addressee only. If you are not the addressee, any disclosure, copy, distribution or use of the contents of this message is prohibited. If you have received this electronic message in error, please notify the sender and destroy the original message and all copies. [what's the purpose to correctly format your message for 80 cols if you append this single-line statement in the end?] (sorry couldn't resist) B-) --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED] --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Problem with cyradm crashing
I've got a weird problem with cyradm crashing (SIGSEGV) while performing
a simple 'lm'. After some ugly dedugging, I've isolated the problem
in lib/imclient.c.
It seems that the call to sasl_decode() in imclient_input() sets
wrongly 'plainlen' (to some value 100). Here's the code:
if (imclient-saslcompleted == 1) {
/* decrypt what we have */
if ((result = sasl_decode(imclient-saslconn, buf, len,
plainbuf, plainlen)) != SASL_OK) {
(void) shutdown(imclient-fd, 0);
}
if (plainlen == 0) return;
} else {
plainbuf = buf;
plainlen = len;
}
Is it a known problem of SASL 2.1.12? If so, please ignore this
bug report.
Some other facts:
1) cryadm crashes only on our production mailboxes database, which
is being converted both via ctl_mboxlist -d / ctl_mboxlist -u and
cvt_cyrusdb. That initially made me focus on format problems
(tried skiplist, db3, flat), conversion problems
(tried db3 tools, too), as a fresh install (empty mailboxes.db)
always worked.
In the end I've discovered it depends on the size of the server
response to the LIST command, i.e. on the number of folders and
their depth.
2) the old cyradm binary, from a 2.0.16 installation, never crashes.
(that made me turn to the client side instead of the server side)
The old binaries were compiled vs. SASL v.1.5.26 (shared), which has
been recently upgraded to 1.5.28.
3) behaviour depends on the SASL mech used to authenticate:
LOGIN: crashes
CRAM-MD5: crashes
DIGEST-MD5: works well
I can't test PLAIN, since cyradm can't do TLS. SRP crashes, too, but
it may be another kind of problem (see below). Please note DIGEST-MD5
it's the only mech in which sasl_decode is expected to do something.
I've traced it, it gets like 2kB of input and returns about 3kB of
'plainbuf'. AFAIK, for the other mechs sasl_decode() should only copy
the input as is.
4) forcing the else part in the above code:
plainbuf = buf;
plainlen = len;
fixes the problem. Of course it is possible only if SASL uses
a mechs which does not provide any protection (when sasl_decode()
is just a noop, I think). This 'fix' works with CRAM-MD5 and LOGIN,
and of course breaks completely DIGEST-MD5 (and others, too, I guess).
Anyway it isolates the bug in sasl_decode(), apparently.
5) by extensive Debugging with printf (TM), I've discovered
the sasl_decode() misbehaviour (plainlen scrambling). It happens
if and only if the buffer is large enough. This happens only when
the server response is large, of course. A short answer (2 or 3 test
mailbox) never triggers it.
large enough means both 4096 (the default) and 2048. 1440 works well.
1024 works too. I'm sorry I can't be more precise, but debugging
a perl application that gets some .so binary modules loaded which in
turn call some shared library functions (SASL) is beyond what I can
do in a decent timeframe B-).
6) the following patch fixes the problem, for all mechs but SRP.
I've never used SRP before, I've tried it only to collect more details
on this problem, so it may be a completely unrelated problem. All other
mechs do work.
--- cyrus-imapd-2.1.12/lib/imclient.c.buf1024 Wed Nov 6 21:43:26 2002
+++ cyrus-imapd-2.1.12/lib/imclient.c Thu Mar 27 20:40:47 2003
@@ -86,7 +86,7 @@
#include iptostring.h
/* I/O buffer size */
-#define IMCLIENT_BUFSIZE 4096
+#define IMCLIENT_BUFSIZE 1024
/* Command completion callback record */
struct imclient_cmdcallback {
sasl_decode() now gets called with a smaller buffer and plainlen
is always ok. Of course this is not a real fix, it just makes
cyradm work with the buggy sasl_decode().
7) the above (problems and fixes) has been tested on both a Red Hat
Linux 7.3 and 8.0. The SASL library has been compiled by me and upgraded
to 2.1.12 on both systems.
8) I haven't made any further investigation since I know very little of
SASL itself. I think that, if this isn't a known bug, I'd have to code
a little test program in C.
.TM.
--
/ / /
/ / / Marco Colombo
___/ ___ / / Technical Manager
/ / / ESI s.r.l.
_/ _/ _/ [EMAIL PROTECTED]
Re: Problem with cyradm crashing
On Fri, 28 Mar 2003, John Alton Tamplin wrote: Marco Colombo wrote: 2) the old cyradm binary, from a 2.0.16 installation, never crashes. (that made me turn to the client side instead of the server side) The old binaries were compiled vs. SASL v.1.5.26 (shared), which has been recently upgraded to 1.5.28. Cyrus 2.x requires SASL 2.x -- if you are running it with SASL 1 perhaps that is your problem. See install-prereq.html in the documentation. Ehm. The *old* binary, running on a completely different system, RHL62, *never* crashes. The working cyramd is the old one, v.2.0.16, compiled with SASL 1.5.26, now running with 1.5.28. What I do is just type the password and then 'lm'. I've setup 2 test systems (reading install-prereq.html, yes), RHL7.3 and RHL8.0, with SASL 2.1.12 and imapd 2.1.12. On both the new test systems, cyradm crashes as described in my previous message. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Moving to a different arch
I'm planning to move one of our server from a Linux/SPARC box to an Intel (well, AMD) based one. I'm also upgrading from 2.0.16 to 2.1.12. I've copied both my configdirectory and partition-default to the new box. When I start the daemon I get (I had to remove the db/__db.* files): Mar 19 16:40:43 devel ctl_cyrusdb[4987]: recovering cyrus databases Mar 19 16:40:43 devel ctl_cyrusdb[4987]: DBERROR db3: Ignoring log file: /home/from-backup/var/cyrus-imapd/db/log.06: magic number 88090400, not 40988 Mar 19 16:40:43 devel ctl_cyrusdb[4987]: DBERROR db3: Invalid log file: log.06: Invalid argument Mar 19 16:40:43 devel ctl_cyrusdb[4987]: DBERROR db3: PANIC: Invalid argument Mar 19 16:40:43 devel ctl_cyrusdb[4987]: DBERROR: critical database situation Mar 19 16:40:43 devel master[4985]: process 4987 exited, status 75 I have the strong feeling B-) that this is an endianess problem of Berkeley db3 (see the magic). I guess I need to dump the database into some text format on the sparc box and restore it on the intel one. Now the questions: 1) is there a way to dump the db using cyrus utilities? 2) if I have to use db3 utils, can someone provide me with hints? (e.g., db_dump works on a file, which files should I dump? is it just the mailboxes.db file? - if so, I think ctl_mboxlist is all I need) 3) am I expected to run into other endianess problems? (cyrus.* files come into my mind). I *think* I remember of being able to move across different archs in the past (1.6.x), when the mailbox file was just a text file. But maybe I'm wrong. TIA, .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: problem installing cyrus imap 2.0.16
On Fri, 11 Jan 2002 [EMAIL PROTECTED] wrote: sageetha satheesan schrieb am Thu, Jan 10, 2002 at 09:06:51PM -0800: * hi, *I had a problem installing cyrus imap. i already * installed cyrus-sasl-1.5.27. * I followed the instructions as per given in the Cyrus * IMAP HOWTO --- in the site : * http://www.linuxdoc.org/HOWTO/Cyrus-IMAP.html#toc6 * * After creating a default user called cyrus and * assigning the password. I proceeded to build files * using the following instructions : * *make depend *make all CFLAGS=-O *make install * * The error occured after typing : * make all CFLAGS=-O * * error was: * * cyrusdb_db3.c:44:16: db.h: No such file or directory * * what should i do now? Please help. Thank you. Please provide system and (if possible) distribution used, e.g. by issuing a uname -a. I can only speak for linux, and only a limited number of distros. When using RedHat/SuSE/Mandrake, it may be the case that you do not have the appropriate db*-devel-RPMs installed. This leads to the effect that the lib is there (which cyrus' configure script will happily notice) but not the corresponding include files, so any make will fail. Which is, technically, a buglet in cyrus' configure script... B-) it should be looking for include files instead of library files (or both). Anyway, the problem could be: - wrong Berkeley DB (vendor package) version; - Berkeley DB installed in some non standard location (like /usr/BerkeleyDB); - missing -devel packages (in Red Hat, for example), as you stated. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: NFS Cyrus (urgent!)
On Fri, 14 Dec 2001, Eranga Udesh wrote: Hi All, With further investigation I managed to shortlist the file systems that I can (as I hope) in this environment. Now the problem is to coose the correct one. I will publish the list, so if anybody know their prons and crons, please advise me. 1. GFS (www.globalfilesystems.org) - commercial product, seems very interesting 2. InterMezzo (www.inter-mezzo.org) - somewhat new comer, has some interesting developments coming up 3. Coda (http://www.coda.cs.cmu.edu) - Excellent, but no news about co-existance with Cyrus Mail Spool 4. Parallel Virtual File System (http://www.parl.clemson.edu/pvfs) - Excellent, but no news about co-existance with Cyrus Mail Spool Thanks in advance, Eranga I'm not an expert, but don't 2. and 3. both have a last-close-wins semantic opposed to the last-write-wins found on local (UNIX) filesystems? Doesn't this raise issues when a shared file is been updated? .TM.
Re: Changing filesystems, backup strategies
On Fri, 21 Sep 2001, Paul Vallee wrote: No, unless that's the default... No, it's not the default. # mount /dev/hda1 on / type ext2 (rw) none on /proc type proc (rw) /dev/hdb1 on /u01 type ext2 (rw) /dev/hda5 on /u03 type ext2 (rw) /dev/hdd1 on /u04 type ext2 (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) Hmm... noatime Do not update inode access times on this file system (e.g, for faster access on the news spool to speed up news servers). Seems interesting... So while running find, for example, this would save overhead? I'm willing to give this a try if that's what it would mean. I've seen significative load decrease by using it, but it was on a Web server (using apache). I think cyrus can benefit from it, but I'm not sure whether it needs it or not to work properly. stat (and thus find) do not change access time, AFAIK, but cpio does, unless you use --reset-access-time. I suppose the --atime-preserve option of tar serves the same purpose. I believe things are worse if you use the above options, since they'll cause double-writing the atime. You need the -o noatime mount option to get rid of atime updates at all, and then you don't need the two options above. Use strace(1) to see what exaclty happens when you perform your backup. If it creates a lot of write traffic (excluding backup output, of course), that explains why load explodes. Cyrus already generates a good mix of read/write traffic. If you're using ext2, have a look at chattr(1). The A attribute seems promising, but I'm not sure it's implemented (and besides, maybe you need to set it on every file, and I'm not aware of any way to have it already set on file creation). And maybe turn off -S if you set it (following docs). It doesn't turn ext2 into a journaled FS, use ext3 (or one of the others) if you're really concerned about data integrity. Otherwise, turn it off, and enjoy performance, at somewhat increased risk if the system crashes. BTW, if that happenes, hope someone managed to flush your disk controller buffers to plates in time - no FS feature will do it. (sorry I have no direct experience on other FS - and the cyrus server I run isn't that loaded, I've never bothered to try noatime. And yes, I've blindly followed instruction and set -S - so take all the above as completely untested with cyrus) .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: Changing filesystems, backup strategies
On Wed, 19 Sep 2001, Paul Vallee wrote: Hello, We are having performance problems with our mailserver, especially when we try to back it up. I am attempting to set up an incremental backup of the imap spool, and preferably to include only files since the last incremental backup. I am a reasonably advanced shell scripter, and I don't need help with coding that. My problem is that basic filesystem commands on this mount point are intolerably slow, and if I dare to run even find on this filesystem the load shoots up high enough to affect service. is it mounted with -o noatime ? .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: SASL re-entrancy crisis (was: OpenLDAP 2.0.x + pam_ldap +cyrus-imapd-2.0.x)
On Wed, 8 Aug 2001, Devdas Bhagat wrote: On Wed, 08 Aug 2001, Marco Colombo spewed into the ether: snip And BTW, why don't you remove SASL from OpenLDAP, instead? You're just asking CMU people to remove SASL from their Cyrus IMAPD so that OpenLDAP 2 can use it to implement the encrypted connection (to the LDAP server) you need. Ask the OpenLDAP people to replace the SASL library they use with PAM... Ok, I have rather simple requirements. I need to store AAA details for multiple users and services in a single place. Since many applications directly support getting user data from LDAP, I'll go along with that. now both cyrus and OpenLDAP use SASL. What I would like to do is get cyrus to use OpenLDAP, like all my other applications. I would prefer not to have to do this through PAM, snice it involves running running a process as root. SASL supports my requirements of having encrypted passwords. However, cyrus does not make it easy for me to use ldap directly. I need to use an external process to do this. Also, even if I choose to use an external process, cyrus makes it pretty hard. I don't think that the complaints are about cyrus SASL, or SASL support in cyrus. They are about the implementation of SASL support in cyrus, which assumes that sasldb will be used, and usernames will reside on the same machine as cyrus. This is a completely different issue. David Wright is proposing to *remove* SASL from Cyrus IMAPd in favor of a PAM-only solution, and I was answering to him. I don't want SASL to be removed from IMAPd, because that's the right place to use SASL. Despite of PAM not being a replacement for SASL, of course. I think that OpenLDAP requirement for a modular, configurable network security layer (SASL itself) is weaker than the IMAPd one. So IFF you need to remove SASL from one of the two, OpenLDAP is a better candidate. You're asking for the opposite: to remove PAM from SASL (for LDAP auth, of course) and implement a direct SASL - LDAP solution. As I said, a complete different issue. snip Anyway, SASL is a very good ad interim solution. Just try to write a SASL application to understand how useful it can be to you. Just one point. My SASL database does *NOT* reside on the same machine as cyrus-imap. I see no reason to expose user data directly to the internet. This is a big flaw in SASL from my POV, it is not a server (It was never supposed to be one, so I would consider this a design flaw, at most). SASL provides support for many machs. Some of them require cleartext passwords to be available to the server. Where those mechs get the passwords from is a mere implementation detail from the SASL engine standpoint. sasldb is one easy way for mechs to get the passwords. Someone please correct me if I'm wrong, but SASL focuses on providing generic authentication hooks for client/server applications, allowing them to support different authentication models / protocols in a transparent and easy-to-manage way. Mechs themeselves focus on protocol properties (thus, PLAIN is weak, CRAM-MD5 better, DIGEST-MD5 offers channel encryption, IIRC, TLS (crypto auth) is an option as EXTERNAL mech...). Finally, mechs need to get their passwords (can be auth tokens, certs, ...) - or they just need to validate them. Strictly speaking, it's not SASL job, it's up to the mechs. The mechs provided with the SASL library may be naive, compared to your needs, but I wouldn't call that a design flaw in SASL. Does this make some sense? Essentially, I'm trying to store user data at a single point. This includes passwords. Now I have services running on other machines, which need to access this data. LDAP provides a convenient API for this, SASL doesn't. Consider LDAP as a means of accessing SASL on a remote machine. Now can we please get good support for multiple authentication methods into imapd? LDAP API doesn't offer anything of what the SASL API does, and of course the converse is also true. So you can't compare the two, as the provide *completely* different services to the application. Otherwise please explain me why OpenLDAP itself uses SASL. SASL already offers good support for multiple authentication methods. So good that even the OpenLDAP developers (as many others) chose to use it to fulfill thier multiple authentication methods needs. Devdas Bhagat -- Bowie's Theorem: If an experiment works, you must be using the wrong equipment. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: SASL re-entrancy crisis (was: OpenLDAP 2.0.x + pam_ldap +cyrus-imapd-2.0.x)
On Thu, 9 Aug 2001, Devdas Bhagat wrote: On Thu, 09 Aug 2001, Marco Colombo spewed into the ether: snip BTW, if really OpenLDAP 2 is build on SASL, you can't really get rid of it. You'll have an IMAPD - LDAP - SASL (for authentication of the LDAP client to the LDAP server) solution. This is what I'm asking for. Quite a few people are looking for something like this, from the traffic about LDAP. Plain Text The current implementation of SASL does not support remote connectivity. [for password retrival / validation, you mean - it's true, that's PAM or the pwcheck daemon job] Of course SASL *implements* remote connectivity. Even OpenLDAP supports remote connectivity *by means* of SASL. What most people are looking for is a way to connect to a remote SASL database with minimal configuration. The problem with the current design of imapd is that it assumes that SASL will be available locally in some form, ignoring that it may not be available there. Uh? SASL knows nothing of where passwords are stored. It's an API for client/server application to perform authentication over the network. The client declares itself as a SASL client, the server as SASL server, and the authentication happens in a transparent (modular, runtime configurable) way. How an auth method validates the passwords the SASL engine doesn't know/care. Do the pwcheck daemons provide support for this? The SASL database is not directly available locally, so a client-server type of application is required which can access SASL. If yes, I'll go with pwcheck or similar, else either a server has to be hacked into SASL and a client in the implementation (not what I like, I would be using kerberos if that was ok), or change imapd so that hooks for other methods can be easily added in. Why change imapd? Why change SASL? Just write a pwcheck daemon that speaks LDAP. First, it's much easier. Second, all SASL applications will (automagically) use it, even OpenLDAP itself. If you change IMAPd, I'll need to change sendmail, because I share the passwords between the two. SASL is *precisely* an API to isolate applications from authentication methods details, and you're proposing to hack imapd to implement hooks for other methods can be easily added in, which is *exacly* what SASL does. IMAPd already has those hooks, the SASL API. Complaining that the default mechs provided with SASL are too naive for you its just like complaining that the example html pages included into the apache distribution do not fit your needs. SASL allows you to write your own auth mech as a plug-in, and add it to all SASL applications (IMAPd, sendmail, OpenLDAP, whatever), without even recompiling them. The default mechs support naive /etc/sasldb, system getpwent(), PAM, a simple pwcheck daemon as password validation systems. Thru PAM you can authenticate over the network (NIS, LDAP, other directory systems). If PAM isn't enough, link the pwcheck daemon against whatever library you like. The default one uses getpwent() system libraries, and on some systems it can be used to query an LDAP server, too. /Plain Text I cannot make it simpler than the above. Me neither. snip Or they can be clients of a simple local /etc/sasldb database. The point being that I see no design flaw here. My point being that it is not easy to hook these into imapd. My point being that IMAPd uses SASL hooks. You hook new mechs into SASL, (place them into the plugin directory) and suddenly all SASL applications become aware of it. How cat it be better that this? snip So use/write a pwcheck daemon. Or a PAM module, it that fits better. I'll probably have to do one of these things :(. The PAM module it's already there. Still I see no reasons you should modify all SASL mechs to be LDAP clients instead of using one of the above methods - but you can do it. Thats not what is being asked for. Let me put this in another way: From my POV SASL is currently like MS Access, great for everything on one machine, but not for much else. SASL is all about client / server dialog. What I need is a slightly more featured software, like MS-SQL server, which can be contacted from over a network. You want LDAP, because the passwords are stored on the network, but you don't realize that LDAP libraries are just a SASL application in order to perform network authentication. How can you say that SASL is everything on one machine when even the solution you're advocating uses SASL to implement network connectivity (well, the initial part of it)? Hopefully, someone has already implemented this, else thats yet another thing to be done. You just need the pam_ldap bug being fixed. Probably a SASL bug, possibly triggered by either IMAPd or OpenLDAP unexpected behaviour (unexpected by the Cyrus SASL library developers, of course). I see no (other) need to modify IMAPd. Devdas Bhagat -- Genius is pain. -- John Lennon .TM
Re: SASL re-entrancy crisis (was: OpenLDAP 2.0.x + pam_ldap +cyrus-imapd-2.0.x)
On 9 Aug 2001, Julio Sanchez Fernandez wrote: Marco Colombo [EMAIL PROTECTED] writes: because that's the right place to use SASL. Despite of PAM not being a replacement for SASL, of course. I think that OpenLDAP requirement for a modular, configurable network security layer (SASL itself) is weaker than the IMAPd one. So IFF you need to remove SASL from one of the two, OpenLDAP is a better candidate. No, OpenLDAP needs to provide SASL. Without it, it is severely crippled. Some people hold the opinion that OpenLDAP is not compliant with the V3 specs if compiled without SASL. The mechanisms implemented may be trivial, but SASL has to be there. Ok, I wrote if and only if. SASL is a Good Thing(TM). The fact I can write a little client/server application that supports many different mechs, from weaker ones to stronger ones, *without* almost any knowledge of them is great. I even support *future* mechs, and users can add them with no need to recompile... .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: SASL and SHADOW
On Thu, 9 Aug 2001, Tyrone Vaughn wrote: I did search the archives and the closest solution I can find is to abandon checking the shadow file via PAM and run the program pwcheck as the root user -- something I don't want to do. If you know the answer, would you please forward it on to me? Simple: you need read access to /etc/shadow to check passwords. So either you arrange /etc/shadow permissions so that the imapd process (which does not run as root) can read it, or you need some root process to read it instead and provide the answer (which is what pwcheck is designed for). As someone else already suggested, if you don't want to run pwcheck as root (the yet another root daemon running on my system syndrome), arrange permissions so that only the imapd process can read /etc/shadow. $ ls -al /etc/shadow -r--r-1 root cyrus 11736 Aug 6 15:20 /etc/shadow I've used both the pwcheck and the above solution successfully. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: SASL re-entrancy crisis (was: OpenLDAP 2.0.x + pam_ldap +cyrus-imapd-2.0.x)
On Wed, 8 Aug 2001, David Wright wrote: First off, thanks to you, Lawrence, and the many others who helped clarify why OpenLDAP 2.0.x + pam_ldap + cyrus-imaps-2.0.x won't play together out-of-the-box. For those just tuning in to this thread, it's because the SASL routines are (1) used both by cyrus-imapd and OpenLDAP and (2) not re-entrant, so memory gets deallocated by one while the other is still trying to use it. Poof -- core dump. CMU people, please don't take the following personally; your work is impressive and appreciated. But some things about it are less than useful off the CMU campus, and I think SASL is one of them. My reasoning is simple: A non-re-entrant security layer is contradiction in terms. Security layers are designed to be ubiquitous -- if SASL works out as planned, every application involved in authentication will link to it. Problems like these would be a dime a dozen (which is one reason SASL will not become ubiquitous). SASL is not a security layer the same way PAM is. PAM is just a library. A network connection (if any) is made only for AAA purpose, and PAM handles ONLY the client side. Conversely, SASL (see RFC) allows you to extend existing network protocols, adding authentication and authorization to them. The SASL *library* handles BOTH client and server side, allowing you to write secure applications with little knowledge of the security protocols involved. While of course it should be reentrant, it is understandable that some problems occur when an application violates the basic client-server model, and creates (via an external library) a second SASL client instance in the middle of the execution of the first one (SASL hides the security protocol state machine away from the programmer). I think for most applications PAM is a much better alternative. It is inherently simpler. It can support ticket systems by using Kerberos. It can support access restrictions based on time-of-day, IP-address, and such, which (correct me if I'm wrong) SASL cannot. If is far more widely used and easily understood. Of course. Whenever an application wants to do some (client-side) AAA PAM is a good framework. But you don't implement a network server with PAM. BTW, it follows that it makes perfect sense for the *server* side of SASL to use PAM (as it does) for AAA. Of course, one can always claim that SASL doesn't hurt anything, since it can call PAM. But my experience has proven the falacy of the doctrine of harmless layers. It turns out to be even more difficult than planned for me to avoid the re-entrancy problem, because the LDAP encyption of OpenLDAP 2.0.x gets broken when compiled --without-cyrus-sasl, and OpenLDAP 1.x doesn't have any encryption, and I need my LDAP communications encrypted (which is why the sasl-ldap patch also isn't an option). Implementation details. PAM doesn't offer a *network* security layer, so it can't simply replace SASL. Which leads me to repeat my earilier question: how hard would it be to replace SASL with PAM, thus producing, IMHO, a more useful imap server for sites other than CMU. Basically this comes down to the question: how many calls to SASL library APIs are there in cyrus-imapd? 10? 100? 1000? If it's less than 100 I'd give it a try myself. You need to re-implement all the client-server authentication dialog. And be sure to interoperate with different clients. With SASL, I can hope to talk to RFC-aware clients, at least. And BTW, why don't you remove SASL from OpenLDAP, instead? You're just asking CMU people to remove SASL from their Cyrus IMAPD so that OpenLDAP 2 can use it to implement the encrypted connection (to the LDAP server) you need. Ask the OpenLDAP people to replace the SASL library they use with PAM... I'm not a big SASL fan myself. Were I to replace SASL *now*, I'd limit network security to SSL/TLS (instead of them being just a external method), with either plaintext password (over encrypted channel) or X.509 client certs, ala HTTP. But in the end SASL lets me do that anyway, and I guess that in the future, when IPSec/IPv6 are widely deployed, we'll just ask the network layer to provide enough security, with no need for another security layer on top of it. Anyway, SASL is a very good ad interim solution. Just try to write a SASL application to understand how useful it can be to you. .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: Signalled to death by 11 on imapd?
On Mon, 2 Jul 2001, William K. Hardeman wrote: Howdy all, Has anyone found a solution or workaround to the 'signalled to death by 11' issue with imapd processes? This seems to occur randomly for us, although more frequently when using a webmail interface than with mulberry or outlook 2000. I did a gdb core-file on one of the cores produced, but all I knew that it told me was that the process had seg faulted. I had a similar problem, when upgrading from 2.0.12 to 2.0.14. Reverted to 2.0.12 (no time to debug now). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
Re: Cyrus/NFS/SMB
On Mon, 20 Nov 2000, Tristan Ball wrote: Note: this is not something I have tried, or even wish to, as my largest cyrus email server handles 250 users at the moment, easily handled by a Sun E450. :-) I've seen the messages poo-pooing using NFS for cyrus, and the reasons make sense, my question is would another shared filesystem, like samba/smb/smbfs under linux be acceptable, or would I have I be better off using a proper distributed filesystem like coda? Just curious. If you choose to respond, would you be so kind as to CC my personal address, as I don't read this list as often as I would like. Thank you. My understanding is that Cyrus IMAPD itself is designed as a central mailboxes (a special kind of files, in a way) server. So it makes little sense to put the mailboxes themselves on a different (central) server. Having a NFS server re-exporting a SMB-mounted FS is quite a nonsense... (unless you're really forced to do so, but be also prepared to pay a price in performance and functionality). Think of IMAP just like another network file service. Building it on top of other one is bad practice. See also: http://asg.web.cmu.edu/cyrus/ag.html (I can't find a way to reach this document from http://asg.web.cmu.edu/cyrus, but there's a link in the doc directory of the source distribution). .TM. -- / / / / / / Marco Colombo ___/ ___ / / Technical Manager / / / ESI s.r.l. _/ _/ _/ [EMAIL PROTECTED]
