Re: Bingo!
-- Gabor Gombas <[EMAIL PROTECTED]> is rumored to have mumbled on 17. November 2007 14:34:02 +0100 regarding Re: Bingo!: I'm not sure what to make of that. I would assume that we've got a blocking BIO, because it is - d'oh - blocking. But I don't see how you influence what kind of BIO you use. A BIO is non-blocking if the underlying file (socket) descriptor has the O_NONBLOCK flag set, either during open or by a previous call to fcntl(). And yes, if O_NONBLOCK was not set on a socket, then any OpenSSL operations can block pretty much indefinitely. Thanks for the explanation. -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpV6drjZVE6e.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
-- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on 17. November 2007 11:21:38 -0500 regarding Re: One more attempt: stuck processes: Here's a patch that seems to fix the problem. I did some basic testing (Linux only) to make sure that it doesn't break anything else, but its always possible that it has some unforseen side effects. Keep an eye on it and let me know if you see anything unusual. Thanks, it seems to be working fine so far! Enjoy your weekend now ... -- Sebastian Hagedorn - Postmaster - RZKR-R1 (Flachbau), Zimmer 18 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 pgpkpMVeIVDp2.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Help with xfermailbox
Dan White wrote: > Wesley Craig wrote: >> If I recall correctly, this is a bad interaction/bug between Cyrus IMAPd >> and Cyrus SASL. I see you're running IMAP 2.3.10. What version of SASL? >> > > 2.1.22 from Debian etch with a couple of customizations to > ldapdb, which itself it compiled against openldap 2.3.30. > > I've also compiled by sasl against heimdal libraries rather than > the (debian) default mit. I just recompiled sasl on both backends to use mit libraries, for gssapi. No luck. Same two errors (syslog and auth). - Dan Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Help with xfermailbox
If I recall correctly, this is a bad interaction/bug between Cyrus IMAPd and Cyrus SASL. I see you're running IMAP 2.3.10. What version of SASL? :wes On 17 Nov 2007, at 10:27, Dan White wrote: > Nov 17 09:25:02 neo cyrus/imap[11281]: encoded packet size too big > (4156 > 4096) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
Sebastian Hagedorn wrote:
> -- Ken Murchison <[EMAIL PROTECTED]> is rumored to have mumbled on
> 16. November 2007 15:54:50 -0500 regarding Re: One more attempt: stuck
> processes:
>
>> That's exactly what Gary is seeing.
>
> Right. Apparently stripped binaries aren't any good for straces.
>
>> Its blocking in SSL_accept().
>> Apparently the client connects to port 995, and then either sends
>> nothing, or goes away and leaves the socket open.
Here's a patch that seems to fix the problem. I did some basic testing
(Linux only) to make sure that it doesn't break anything else, but its
always possible that it has some unforseen side effects. Keep an eye on
it and let me know if you see anything unusual.
--- prot.c.~1.93.~ 2007-11-17 10:02:49.0 -0500
+++ prot.c 2007-11-17 11:21:05.0 -0500
@@ -58,6 +58,7 @@
#ifdef HAVE_UNISTD_H
#include
#endif
+#include
#include
#include
#include
@@ -214,10 +215,20 @@
*/
int prot_settimeout(struct protstream *s, int timeout)
{
+struct timeval tv;
+
assert(!s->write);
s->read_timeout = timeout;
s->timeout_mark = time(NULL) + timeout;
+
+/* Set a read timeout on the socket itself. This primarily prevents
+ * OpenSSL calls such as SSL_accept() from blocking indefinitely.
+ */
+tv.tv_sec = timeout;
+tv.tv_usec = 0;
+setsockopt(0, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(struct timeval));
+
return 0;
}
--
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Help with xfermailbox
Wesley Craig wrote: > If I recall correctly, this is a bad interaction/bug between Cyrus IMAPd > and Cyrus SASL. I see you're running IMAP 2.3.10. What version of SASL? > 2.1.22 from Debian etch with a couple of customizations to ldapdb, which itself it compiled against openldap 2.3.30. I've also compiled by sasl against heimdal libraries rather than the (debian) default mit. I've temporarily disabled my ldapdb canon_user plugin on the two servers it was enabled on, and I'm getting the same error. I'm using GSSAPI to authenticate. >> Nov 17 09:25:02 neo cyrus/imap[11281]: encoded packet size too big >> (4156 > 4096) - Dan Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Help with xfermailbox
Wesley Craig wrote: > On 16 Nov 2007, at 15:53, Dan White wrote: >> Nov 16 13:44:57 neo cyrus/imap[6171]: decoding error: generic >> failure; SASL(-1): generic failure: , closing connection > > A fuller version of this error is probably recorded in your auth log. > > :wes Here's from my syslog.conf: Nov 17 09:25:02 neo cyrus/imap[11281]: decoding error: generic failure; SASL(-1): generic failure: , closing connection and from my auth.log Nov 17 09:25:02 neo cyrus/imap[11281]: encoded packet size too big (4156 > 4096) - Dan Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: One more attempt: stuck processes
On Fri, Nov 16, 2007 at 06:37:52PM +0100, Sebastian Hagedorn wrote: > OK. Still the symptom seems to be different from what I'm seeing. It may be. As I said I had no time so far to investigate it in depth, I just wanted to say "mee too" for the hung process problem. > Could it be that you have a process limit in /etc/cyrus.conf? Of course. And when the number of hung processes reaches that limit users start to complain that they can not read their mail. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Collaboration replacement via Toltec/Bynari (was How many people to admin a Cyrus system?)
Hi Rob and all I have been considering doing this in my environment. From the user perspective having to launch messages that should be automatically processed (like read acks and task/appointment updates) confuses them a little. I don't know if it is worth it for acks though as this only displays in the original outbound messages if you leave it in the PST sent box. We move sent items to the IMAP store with a rule, as well as manually file into a "public folder" system for group visibility. I think though that with this method any ack/update mails will still appear in the "IMAP" Inbox as unread. It almost makes me want to rewrite the mail header at postfix time (or use a sieve rule) and maintain a separate POP mailbox for the purpose... Cheers Bob Rob Mueller wrote: So just mapping the INBOX to display will result in these special emails never being "processed". However POPing them will. I do have a hacky workaround suggestion which might work. 1. Display all folders including INBOX 2. Setup a "dummy" POP account that points to the same cyrus server 3. Make sure the "leave on server" option is set 4. Setup a filter rule in Outlook to just delete all the POPed email Then emails arriving in the IMAP inbox will appear as normal, but they'll also be POPed regularly and processed by the Outlook transport mechansim. Unfortunately this solution wont currently work because (1) is impossible in Toltec at the moment. Rob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
On Fri, Nov 16, 2007 at 11:17:23PM +0100, Sebastian Hagedorn wrote: > I haven't yet found what BIO stands for According to Wikipedia it's "an > abstraction library used by OpenSSL to handle communication of various > kinds, including files and sockets, both secure and not". You can think about a BIO as an object that wraps around a file (socket) descriptor, has an internal buffer and some methods for reading and writing. I.e. instead of calling the read() syscall on the socket, you call the ->read method of the BIO, and that ->read method may do whatever it wants including decrypting the raw SSL stream. Of course this description is not entirely correct but is enough to give you the idea. > I'm not sure what to make of that. I would assume that we've got a blocking > BIO, because it is - d'oh - blocking. But I don't see how you influence > what kind of BIO you use. A BIO is non-blocking if the underlying file (socket) descriptor has the O_NONBLOCK flag set, either during open or by a previous call to fcntl(). And yes, if O_NONBLOCK was not set on a socket, then any OpenSSL operations can block pretty much indefinitely. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Bingo!
I'm not sure of you're aware of it, so I'll point it out: you tried
something different from what Ken and I tried. It doesn't explain
everything, but at least some of what you see.
-- Alain Spineux <[EMAIL PROTECTED]> is rumored to have mumbled on 17.
November 2007 00:46:43 +0100 regarding Re: Bingo!:
I tested the timeout using a SSL connection and a normal connection
and get two different result !
The SSL was working as expected but the normal one "timeout" in a
different way, it get a SIGTERM !
I will repeat the test
Here is the normal one :
# telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
+OK eg01.emailgency.loc Cyrus POP3 v2.3.9-openpkg server ready
user [EMAIL PROTECTED]
+OK Name is a valid mailbox
pass
+OK Mailbox locked and ready
list
+OK scan listing follows
1 1779
.
# I dont send anymore command for 10min and wait for the timeout !
Connection closed by foreign host.
That's OK. Just to make sure: the problem I had noticed on my server and
which we were trying to nail down did *not* occur for sessions where a
login had been done. It only happens for sessions where the client does
nothing or goes away *before* a login (or actually a successful SSL
negotiation) has happened.
Here is the strace from the list command
read(0, "list\r\n", 4096) = 6
time(NULL) = 1195253508
write(13, "<1195253508<", 12) = 12
write(13, "list\r\n", 6)= 6
open("/kolab/var/imapd/msg/shutdown", O_RDONLY) = -1 ENOENT (No such
file or directory)
select(1, [0], NULL, NULL, {0, 0}) = 0 (Timeout)
time(NULL) = 1195253508
write(13, ">1195253508>", 12) = 12
write(13, "+OK scan listing follows\r\n1 1779"..., 37) = 37
write(1, "+OK scan listing follows\r\n1 1779"..., 37) = 37
time(NULL) = 1195253508
select(1, [0], NULL, NULL, {600, 0}
)= ? ERESTARTNOHAND (To be restarted)
#
# HERE IT IS WAITING FOR 10min AS EXPECTED
--- SIGTERM (Terminated) @ 0 (0) ---
Process 5720 detached
I dont understand why it got a SIGTERM !
Neither do I. I tested the same thing and I got this trace:
Process 18811 attached - interrupt to quit
select(1, [0], NULL, NULL, {565, 38}) = 0 (Timeout)
time(NULL) = 1195297357
close(9)= 0
munmap(0xb47a7000, 4096)= 0
unlink("/var/lib/imap/proc/18811") = 0
...
So everything looks fine here.
--
Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10
Zentrum für angewandte Informatik - Universitätsweiter Service RRZK
Universität zu Köln / Cologne University - Tel. +49-221-478-5587
pgp3STziVdp5B.pgp
Description: PGP signature
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Collaboration replacement via Toltec/Bynari (was How many people to admin a Cyrus system?)
> So, the problem has nothing to do with IMAP, and everything to do with > message handling before delivery to the mailbox. If I've assimilated everything right, I think the summary of the problem is: Outlook handles some email messages specially (the example Joon has used is iTIP emails). To actually process these special emails, they have to pass through the Outlook "transport" mechanism. An email appearing in an IMAP folder is not processed by the Outlook transport mechansim. However an email POPed from a POP server to a local folder is. There's no api way to just "invoke" the Outlook transport system on an email, it's a black box thing that just happens on certain transports. So just mapping the INBOX to display will result in these special emails never being "processed". However POPing them will. I do have a hacky workaround suggestion which might work. 1. Display all folders including INBOX 2. Setup a "dummy" POP account that points to the same cyrus server 3. Make sure the "leave on server" option is set 4. Setup a filter rule in Outlook to just delete all the POPed email Then emails arriving in the IMAP inbox will appear as normal, but they'll also be POPed regularly and processed by the Outlook transport mechansim. Unfortunately this solution wont currently work because (1) is impossible in Toltec at the moment. Rob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: LARGE single-system Cyrus installs?
> This is where I think the actual user count may really influence this > behavior. On our system, during heavy times, we can see writes to the > mailboxes file separated by no more than 5-10 seconds. > > If you're constantly freezing all cyrus processes for the duration of > those writes, and those writes are taking any appreciable time at all, > you're going to have a stuttering server with big load averages. This shouldn't really be a problem. Yes the whole file is locked for the duration of the write, however there should be only 1 fsync per "transaction", which is what would introduce any latency. The actual writes to the db file itself should be basically instant as the OS should just cache them. Still, you have a point that mailboxes.db is a global point of contention, and it is access a lot, so blocking all processes on it for a write could be an issue. Which makes me even more glad that we've split up our servers into lots of small cyrus instances, even less points of contention... Rob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
