Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi:
On Tue, 2010-10-26 at 15:46 +0200, Ralf Hildebrandt wrote:
on the morning I was running dovecot-lda for local delivery, but
then switched to the old 1.2.x deliver for performance reasons.
If the slowness is caused by config parsing, that's actually easy to
avoid:
service config {
unix_listener config {
mode = 0666
}
}
(Of course assuming there isn't anything secret in the config.)
Now dovecot-lda and others will read the config from the socket rather
than executing doveconf.
I will test this immediately!!
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On 3.11.2010, at 7.51, Ralf Hildebrandt wrote:
service config {
unix_listener config {
mode = 0666
}
}
Now dovecot-lda and others will read the config from the socket rather
than executing doveconf.
I will test this immediately!!
Forgot to mention: This requires that base_dir hasn't been changed from the
built-in default. But that can also be worked around with symlinks, e.g.:
ln -s /var/run/dovecot /usr/local/var/run/dovecot
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi:
On 3.11.2010, at 7.51, Ralf Hildebrandt wrote:
service config {
unix_listener config {
mode = 0666
}
}
Now dovecot-lda and others will read the config from the socket rather
than executing doveconf.
I will test this immediately!!
Forgot to mention: This requires that base_dir hasn't been changed from the
built-in default. But that can also be worked around with symlinks, e.g.:
ln -s /var/run/dovecot /usr/local/var/run/dovecot
Well, dovecot itself seems to be working. Would it log problems
accessing base_dir?
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On 3.11.2010, at 10.32, Ralf Hildebrandt wrote: Forgot to mention: This requires that base_dir hasn't been changed from the built-in default. But that can also be worked around with symlinks, e.g.: ln -s /var/run/dovecot /usr/local/var/run/dovecot Well, dovecot itself seems to be working. Would it log problems accessing base_dir? No. If it can't read the config socket (which it can't by default), it silently fallbacks to executing doveconf.
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi:
On 3.11.2010, at 7.51, Ralf Hildebrandt wrote:
service config {
unix_listener config {
mode = 0666
}
}
Now dovecot-lda and others will read the config from the socket rather
than executing doveconf.
I will test this immediately!!
Forgot to mention: This requires that base_dir hasn't been changed from the
built-in default. But that can also be worked around with symlinks, e.g.:
ln -s /var/run/dovecot /usr/local/var/run/dovecot
ln -s /usr/dovecot-2/var/run/dovecot /var/run/dovecot
(since /var/run/dovecot didn't exist here, I have everything below
--prefix=/usr/dovecot-2)
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: No. If it can't read the config socket (which it can't by default), it silently fallbacks to executing doveconf. :) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On 3.11.2010, at 10.34, Ralf Hildebrandt wrote: ln -s /var/run/dovecot /usr/local/var/run/dovecot ln -s /usr/dovecot-2/var/run/dovecot /var/run/dovecot That symlink probably gets deleted at system bootup. (since /var/run/dovecot didn't exist here, I have everything below --prefix=/usr/dovecot-2) If you haven't changed base_dir, then the symlink isn't necessary. In the example above I was assuming base_dir had been changed to /var/run/dovecot from the built-in /usr/local/var/run/dovecot.
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: On 3.11.2010, at 10.34, Ralf Hildebrandt wrote: ln -s /var/run/dovecot /usr/local/var/run/dovecot ln -s /usr/dovecot-2/var/run/dovecot /var/run/dovecot That symlink probably gets deleted at system bootup. (since /var/run/dovecot didn't exist here, I have everything below --prefix=/usr/dovecot-2) If you haven't changed base_dir, then the symlink isn't necessary. In the example above I was assuming base_dir had been changed to /var/run/dovecot from the built-in /usr/local/var/run/dovecot. I left base_dir at it's default, which is $prefix/var/run/dovecot So, no harm, no foul -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Tue, 2010-10-26 at 15:46 +0200, Ralf Hildebrandt wrote:
on the morning I was running dovecot-lda for local delivery, but
then switched to the old 1.2.x deliver for performance reasons.
If the slowness is caused by config parsing, that's actually easy to
avoid:
service config {
unix_listener config {
mode = 0666
}
}
(Of course assuming there isn't anything secret in the config.)
Now dovecot-lda and others will read the config from the socket rather
than executing doveconf.
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi:
My idea is to turn back on logging and then trace the dovecot/log
process using
strace -p PID -c
-tt is also nice.
I traced around a bit today, in dovecot-lda.
A pattern I'm seeing quite often is this:
0.000853 open(/usr/dovecot-2/lib/dovecot/settings,
O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 7
0.000463 fcntl64(7, F_GETFD) = 0x1 (flags FD_CLOEXEC)
0.000555 getdents64(7, /* 8 entries */, 32768) = 360
0.023219 getdents64(7, /* 0 entries */, 32768) = 0
0.001545 close(7) = 0
two consecutive getdents64 calls on the same FD, one fast, one slow.
WTF?
# ll /usr/dovecot-2/lib/dovecot/settings
total 88
-rw-r--r-- 1 root root 24766 Okt 21 21:27 libmanagesieve_login_settings.a
-rwxr-xr-x 1 root root 1107 Okt 21 21:27 libmanagesieve_login_settings.la
-rwxr-xr-x 1 root root 23291 Okt 21 21:27 libmanagesieve_login_settings.so
-rw-r--r-- 1 root root 11630 Okt 21 21:27 libmanagesieve_settings.a
-rwxr-xr-x 1 root root 1065 Okt 21 21:27 libmanagesieve_settings.la
-rwxr-xr-x 1 root root 12858 Okt 21 21:27 libmanagesieve_settings.so
And this is happening once more in the same trace:
0.002036 open(/usr/dovecot-2/lib/dovecot,
O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 8
0.000468 fcntl64(8, F_GETFD) = 0x1 (flags FD_CLOEXEC)
0.003281 getdents64(8, /* 85 entries */, 32768) = 3736
0.150525 getdents64(8, /* 0 entries */, 32768) = 0
0.000490 close(8) = 0
on fast call, one slow call.
Sorting all traces by execution time:
% cat dovecot-lda.* | sort -n -k1 |less
0.094459 fstat64(8, {st_mode=S_IFREG|0755, st_size=1870288, ...}) = 0
0.095051 brk(0x8d73000)= 0x8d73000
0.098191 getdents64(8, /* 0 entries */, 32768) = 0
0.098912 time(NULL)= 1288084930
0.103059 getdents64(8, /* 0 entries */, 32768) = 0
0.110251 getdents64(8, /* 0 entries */, 32768) = 0
0.115710 getdents64(8, /* 0 entries */, 32768) = 0
0.117688 time(NULL)= 1288084937
0.128360 getdents64(8, /* 0 entries */, 32768) = 0
0.130091 mprotect(0xb75b4000, 4096, PROT_READ) = 0
0.131681 rt_sigaction(SIGALRM, {0xb76cb490, [], SA_SIGINFO}, NULL, 8) = 0
0.133327 getdents64(8, /* 0 entries */, 32768) = 0
0.134047 stat64(/home/c/o/cousername/Maildir/dovecot-uidlist,
{st_mode=S_IFREG|0600, st_size=1130, ...}) = 0
0.135077 rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTART}, NULL, 8) = 0
0.150525 getdents64(8, /* 0 entries */, 32768) = 0
0.153457 open(/home/f/k/fkusername/Maildir/dovecot-uidlist,
O_RDWR|O_LARGEFILE) = 11
0.170190 getdents64(8, /* 0 entries */, 32768) = 0
0.188536 getdents64(8, /* 0 entries */, 32768) = 0
0.193665 umask(077)= 0177
0.270003 getdents64(8, /* 0 entries */, 32768) = 0
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* Ralf Hildebrandt ralf.hildebra...@charite.de: I traced around a bit today, in dovecot-lda. A pattern I'm seeing quite often is this: I also actived system accounting and found this: # sa -l |egrep (deliver|dovecot-lda) 17365 277.05re 5.98u 159.69s 0avio 1489k dovecot-lda 17951 179.22re 2.04u 74.71s 0avio 1360k deliver on the morning I was running dovecot-lda for local delivery, but then switched to the old 1.2.x deliver for performance reasons. Both are using sieve etc. (I converted the config when switching to 2.0.x). It seems the old deliver is using less real time, and only 50% system time. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Tue, 2010-10-26 at 11:44 +0200, Ralf Hildebrandt wrote: I traced around a bit today, in dovecot-lda. A pattern I'm seeing quite often is this: 0.000853 open(/usr/dovecot-2/lib/dovecot/settings, O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 7 0.000463 fcntl64(7, F_GETFD) = 0x1 (flags FD_CLOEXEC) 0.000555 getdents64(7, /* 8 entries */, 32768) = 360 0.023219 getdents64(7, /* 0 entries */, 32768) = 0 0.001545 close(7) = 0 two consecutive getdents64 calls on the same FD, one fast, one slow. WTF? I don't know.. Is that directory on local filesystem? Are your mails on the same disk? And this is happening once more in the same trace: 0.002036 open(/usr/dovecot-2/lib/dovecot, O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 8 0.000468 fcntl64(8, F_GETFD) = 0x1 (flags FD_CLOEXEC) 0.003281 getdents64(8, /* 85 entries */, 32768) = 3736 0.150525 getdents64(8, /* 0 entries */, 32768) = 0 0.000490 close(8) = 0 on fast call, one slow call. v1.x was also doing this. It's for loading plugins. 0.098912 time(NULL)= 1288084930 time(NULL) should always be a really fast operation. On my somewhat old machine it takes 0.16 seconds. So if it's taking this long, it seems that the system in general is under heavy load and the individual system call times don't really tell anything. 0.193665 umask(077)= 0177 This should be about 0.16 seconds too.
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: On Tue, 2010-10-26 at 11:44 +0200, Ralf Hildebrandt wrote: I traced around a bit today, in dovecot-lda. A pattern I'm seeing quite often is this: 0.000853 open(/usr/dovecot-2/lib/dovecot/settings, O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY|O_CLOEXEC) = 7 0.000463 fcntl64(7, F_GETFD) = 0x1 (flags FD_CLOEXEC) 0.000555 getdents64(7, /* 8 entries */, 32768) = 360 0.023219 getdents64(7, /* 0 entries */, 32768) = 0 0.001545 close(7) = 0 two consecutive getdents64 calls on the same FD, one fast, one slow. WTF? I don't know.. Is that directory on local filesystem? Yes. Are your mails on the same disk? No! We have: /dev/sda1 on / type ext3 (rw,errors=panic) /dev/mapper/volg1-logv1 on /home type ext4 (rw,noexec,nodev,noatime,errors=panic,data=writeback) time(NULL) should always be a really fast operation. On my somewhat old machine it takes 0.16 seconds. So if it's taking this long, it seems that the system in general is under heavy load and the individual system call times don't really tell anything. Yes, but where does the heavy load come from :) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: With v1.2 all logging also went through dovecot master process, OK. I never bothered, because I never had problems :) I think dovecot could benefit from a diagram like the Postfix big picture where all daemons are listed and their relations/dependencies. each using their own pipe fd. With v2.0 each service has their own fd (e.g. all imap processes log via the same shared pipe fd, all pop3 processes log via another fd, etc). So it should be better in v2.0 than v1.2.. Yes. My idea is to turn back on logging and then trace the dovecot/log process using strace -p PID -c but not today, I'd rather do this on the weekend. It would be possible to make all processes log directly though, by adding -L parameter to the service executables. That would be interesting. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Fri, 2010-10-15 at 09:51 +0200, Ralf Hildebrandt wrote: * Timo Sirainen t...@iki.fi: With v1.2 all logging also went through dovecot master process, OK. I never bothered, because I never had problems :) I think dovecot could benefit from a diagram like the Postfix big picture where all daemons are listed and their relations/dependencies. Yeah, I guess I should draw something some day. :) My idea is to turn back on logging and then trace the dovecot/log process using strace -p PID -c -tt is also nice. but not today, I'd rather do this on the weekend. Each log line is written in a separate write() call. This would be a bit tricky to change to buffer the writes.. But I did get rid of some unnecessary time() + stat(/etc/localtime) calls with: http://hg.dovecot.org/dovecot-2.0/rev/4933c3095ee2 http://hg.dovecot.org/dovecot-2.0/rev/e68366e88099 http://hg.dovecot.org/dovecot-2.0/rev/80097e5c38e9
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: OK. I never bothered, because I never had problems :) I think dovecot could benefit from a diagram like the Postfix big picture where all daemons are listed and their relations/dependencies. Yeah, I guess I should draw something some day. :) Postfix is a bad example. I had to redraw Wietse's diagram for a course on Postfix, since he kept adding new daemons :) Each log line is written in a separate write() call. This would be a bit tricky to change to buffer the writes.. But I did get rid of some unnecessary time() + stat(/etc/localtime) calls with: http://hg.dovecot.org/dovecot-2.0/rev/4933c3095ee2 http://hg.dovecot.org/dovecot-2.0/rev/e68366e88099 http://hg.dovecot.org/dovecot-2.0/rev/80097e5c38e9 I love you man :-) I'll try those. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* Ralf Hildebrandt ralf.hildebra...@charite.de: Colleague returned from vacation today and gave some ideas. Today I turned off the mail_log and notify plugins and the load dropped considerably. Both plugins were used with default settings. I left all settings unchanged and examined the machine. Load stays low, so something the way how logging is done in 2.0.x must have changed. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Thu, 2010-10-14 at 13:42 +0200, Ralf Hildebrandt wrote: * Ralf Hildebrandt ralf.hildebra...@charite.de: Colleague returned from vacation today and gave some ideas. Today I turned off the mail_log and notify plugins and the load dropped considerably. Both plugins were used with default settings. I left all settings unchanged and examined the machine. Load stays low, so something the way how logging is done in 2.0.x must have changed. Load as in I/O load or CPU load?.. If CPU, which process was eating it? I can't think of any good reason why mail_log would be doing anything that would cause such problems (other than maybe just logging too much stuff and causing problems because of that).
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: On Thu, 2010-10-14 at 13:42 +0200, Ralf Hildebrandt wrote: * Ralf Hildebrandt ralf.hildebra...@charite.de: Colleague returned from vacation today and gave some ideas. Today I turned off the mail_log and notify plugins and the load dropped considerably. Both plugins were used with default settings. I left all settings unchanged and examined the machine. Load stays low, so something the way how logging is done in 2.0.x must have changed. Load as in I/O load or CPU load?.. If CPU, which process was eating it? CPU, specifically system (not user, not idle, not wait). Process: No special process was peaking. How is the logging implemented? Each imap process is logging by itself? I can't think of any good reason why mail_log would be doing anything that would cause such problems (other than maybe just logging too much stuff and causing problems because of that). That kind of logging was on with 1.2.x, unless the defaults changed between 1.2.x and 2.0.x, it logged the same amount of data... -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Thu, 2010-10-14 at 15:47 +0200, Ralf Hildebrandt wrote: Colleague returned from vacation today and gave some ideas. Today I turned off the mail_log and notify plugins and the load dropped considerably. Both plugins were used with default settings. Load as in I/O load or CPU load?.. If CPU, which process was eating it? CPU, specifically system (not user, not idle, not wait). Process: No special process was peaking. How is the logging implemented? Each imap process is logging by itself? dovecot-lda logs directly, with everything else it goes through dovecot/log process. Are you logging to syslog? Hmm. System CPU usage is about time spent in syscalls.. Is it doing too many tiny write()s or something? .. Maybe I should strace the processes and see if something's too slow there.
Re: [Dovecot] Config review (2.0.5)
* Timo Sirainen t...@iki.fi: Process: No special process was peaking. How is the logging implemented? Each imap process is logging by itself? dovecot-lda logs directly, OK! with everything else it goes through dovecot/log process. Interesting: 1400+ processes logging through one process... Has this changed from 1.2? Are you logging to syslog? yes! Hmm. System CPU usage is about time spent in syscalls.. Exactly. Is it doing too many tiny write()s or something? I was thinking of this as well, because the total amount of disk IO didn't change (between 1.2 and 2.0) only the time spent in system... .. Maybe I should strace the processes and see if something's too slow there. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On Thu, 2010-10-14 at 16:47 +0200, Ralf Hildebrandt wrote: with everything else it goes through dovecot/log process. Interesting: 1400+ processes logging through one process... Has this changed from 1.2? With v1.2 all logging also went through dovecot master process, each using their own pipe fd. With v2.0 each service has their own fd (e.g. all imap processes log via the same shared pipe fd, all pop3 processes log via another fd, etc). So it should be better in v2.0 than v1.2.. It would be possible to make all processes log directly though, by adding -L parameter to the service executables.
Re: [Dovecot] Config review (2.0.5)
* Daniel L. Miller dmil...@amfes.com: Now wait a minute! You said you found the problem and it was exactly what I suggested! I've already received my prize for most intelligent @ss in a discussion group - you can't take that away from me! Yes I can. *it gone* What changed? You turned off FTS, performance was better - and then it developed the problem again? You didn't specify. Colleague returned from vacation today and gave some ideas. Today I turned off the mail_log and notify plugins and the load dropped considerably. Both plugins were used with default settings. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* Ralf Hildebrandt ralf.hildebra...@charite.de: I wonder if the old files from 1.2.5 were still valid with 2.0.5... But given identical clients and user behaviour (all I changed was the dovecot version), one would expect both 1.2.x and 2.0.x behave identical in terms of load. I put some more effort into the analysis of the performance problems: It's not the FTS plugin, since re-enabling doesn't change a thing. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On 10/12/2010 4:25 AM, Ralf Hildebrandt wrote: * Ralf Hildebrandtralf.hildebra...@charite.de: I wonder if the old files from 1.2.5 were still valid with 2.0.5... But given identical clients and user behaviour (all I changed was the dovecot version), one would expect both 1.2.x and 2.0.x behave identical in terms of load. I put some more effort into the analysis of the performance problems: It's not the FTS plugin, since re-enabling doesn't change a thing. Now wait a minute! You said you found the problem and it was exactly what I suggested! I've already received my prize for most intelligent @ss in a discussion group - you can't take that away from me! What changed? You turned off FTS, performance was better - and then it developed the problem again? You didn't specify. -- Daniel
Re: [Dovecot] Config review (2.0.5)
On 8.10.2010, at 12.55, Ralf Hildebrandt wrote: Has the implementation of FTS changed in some way? Reading the two wikis doesn't yield any important difference. My clients/users surely have not changed! No.. I haven't touched Squat for a long time. Then again, I haven't really tried it much either for a long time. I guess I'll have to try.
Re: [Dovecot] Config review (2.0.5)
* Daniel L. Miller dmil...@amfes.com:
# FullTextSearch
fts = squat
I'm probably talking out of the wrong hole again - but have you tried
removing squat from your plugin list to see if it makes a difference?
Oh my god, load is down this morning. WOW.
But what has changed? My old 1.2.x config ALSO had:
mail_plugins = quota imap_quota trash mail_log fts fts_squat zlib autocreate
and
plugin {
fts = squat
...
Has the implementation of FTS changed in some way? Reading the two
wikis doesn't yield any important difference. My clients/users surely
have not changed!
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
# FullTextSearch fts = squat I'm probably talking out of the wrong hole again - but have you tried removing squat from your plugin list to see if it makes a difference? Oh my god, load is down this morning. WOW. But what has changed? Perhaps: The initial Squat index building for large mailboxes can be very CPU and memory hungry. - http://wiki2.dovecot.org/Plugins/FTS/Squat
Re: [Dovecot] Config review (2.0.5)
* m...@eulberg.name m...@eulberg.name: Oh my god, load is down this morning. WOW. But what has changed? Perhaps: The initial Squat index building for large mailboxes can be very CPU and memory hungry. - http://wiki2.dovecot.org/Plugins/FTS/Squat Should that info be logged? Also: since the LDA doesn't update the search index, repeated logins should cause repeated rebuilds of the index. BTW, where's the index stored? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
* m...@eulberg.name m...@eulberg.name: Oh my god, load is down this morning. WOW. But what has changed? Perhaps: The initial Squat index building for large mailboxes can be very CPU and memory hungry. - http://wiki2.dovecot.org/Plugins/FTS/Squat Should that info be logged? Also: since the LDA doesn't update the search index, repeated logins should cause repeated rebuilds of the index. BTW, where's the index stored? I guess it will not be logged. Wiki says: The Squat indexes are stored among Dovecot's other index files. They're called dovecot.index.search and dovecot.index.search.uids. It's safe to delete both of these files to trigger a rebuild.
Re: [Dovecot] Config review (2.0.5)
* m...@eulberg.name m...@eulberg.name: I guess it will not be logged. It would probably happen too often. Wiki says: The Squat indexes are stored among Dovecot's other index files. They're called dovecot.index.search and dovecot.index.search.uids. It's safe to delete both of these files to trigger a rebuild. I wonder if the old files from 1.2.5 were still valid with 2.0.5... But given identical clients and user behaviour (all I changed was the dovecot version), one would expect both 1.2.x and 2.0.x behave identical in terms of load. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebra...@charite.de | http://www.charite.de
[Dovecot] Config review (2.0.5)
Since I have these performance problems after migration to 2.0.5 I'm
publishing my config for review. Can anybody spot any obvious signs of
trouble?
# 2.0.5: /usr/local/etc/dovecot.conf
# OS: Linux 2.6.32-24-generic-pae i686 Debian squeeze/sid
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
disable_plaintext_auth = no
auth_master_user_separator = *
mail_location = maildir:~/Maildir
# wegen webmail!
mail_max_userip_connections = 1024
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date
# Namespace für courier-compatibilitaet
namespace {
inbox = yes
location =
prefix = INBOX.
separator = .
type = private
}
# fuer user*masteruser logins
passdb {
args = /usr/dovecot-2/etc/dovecot/dovecot.masteruser
driver = passwd-file
master = yes
pass = yes
}
# Authorisierung via PAM, /etc/pam.d/dovecot
auth_cache_size = 64 M
passdb {
args = cache_key=%u
driver = pam
}
# User via passwd
userdb {
driver = passwd
}
# plugin Konfiguration
plugin {
# mailboxen anlegen und subscriben
autocreate = Trash
autocreate2 = spam
autocreate3 = Sent
autocreate4 = Drafts
autosubscribe = Trash
autosubscribe2 = spam
autosubscribe3 = Sent
autosubscribe4 = Drafts
# FullTextSearch
fts = squat
# Quota
quota = maildir
quota_rule = INBOX.Trash:storage=+2048M
quota_warning = storage=99%% quota-warning 99 %u
quota_warning2 = storage=95%% quota-warning 95 %u
quota_warning3 = storage=90%% quota-warning 90 %u
quota_warning4 = storage=85%% quota-warning 85 %u
# Sieve
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
# Trash (wenn Mailbox voll, dann Trash und spam leeren)
trash = /usr/dovecot-2/etc/dovecot/dovecot-trash.conf
}
# Wir sprechen alles
protocols = imap sieve pop3
# unix domain socket fuer Postfix
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
user = root
}
# imap-login Prozess
# high performance mode
service imap-login {
service_count = 0
}
# der eigentliche IMAPD
service imap {
drop_priv_before_exec = yes
process_limit = 2048
}
# managesieve-login, wird nur genutzt von webmail aus
service managesieve-login {
service_count = 0
inet_listener sieve_deprecated {
port = 2000
}
}
# der eigentliche managesieve
service managesieve {
drop_priv_before_exec = yes
process_limit = 128
}
# pop3-login Prozess
# high performance mode
service pop3-login {
service_count = 0
# kein pop3, nur pop3s!
inet_listener pop3 {
port = 0
}
}
# der eigentliche pop3
service pop3 {
drop_priv_before_exec = yes
process_limit = 512
}
# die ganzen SSL Zertifikate
ssl_ca = /etc/ssl/certs/ca-certificates.crt
ssl_cert = /etc/ssl/certs/cert-188235905-postamt.charite.de-g02.pem
ssl_key = /etc/ssl/private/postamt.key
# schoene Ausgabe in ps auxwww
verbose_proctitle = yes
version_ignore = yes
mail_fsync = never
maildir_very_dirty_syncs = yes
# globale settings, die anderen werden ja nach Protokoll individuell ergaenzt!
mail_plugins = notify mail_log
# imap kann am meisten
protocol imap {
mail_plugins = $mail_plugins quota imap_quota trash fts fts_squat autocreate
}
# pop3 hat workarounds
protocol pop3 {
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_lock_session = yes
pop3_uidl_format = %v-%u
}
# LDA macht noch zusaetzlich sieve quota trash
protocol lda {
mail_plugins = $mail_plugins sieve quota trash
postmaster_address = postmas...@charite.de
quota_full_tempfail = yes
syslog_facility = local4
}
# der schickt die Quota warnmails
service quota-warning {
executable = script /usr/local/scripts/quota-warning2
user = root
unix_listener quota-warning {
mode = 0666
user = vmail
group = users
}
}
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
Re: [Dovecot] Config review (2.0.5)
On 10/7/2010 8:16 AM, Ralf Hildebrandt wrote:
Since I have these performance problems after migration to 2.0.5 I'm
publishing my config for review. Can anybody spot any obvious signs of
trouble?
snip
# plugin Konfiguration
plugin {
# mailboxen anlegen und subscriben
autocreate = Trash
autocreate2 = spam
autocreate3 = Sent
autocreate4 = Drafts
autosubscribe = Trash
autosubscribe2 = spam
autosubscribe3 = Sent
autosubscribe4 = Drafts
# FullTextSearch
fts = squat
I'm probably talking out of the wrong hole again - but have you tried
removing squat from your plugin list to see if it makes a difference?
--
Daniel
Re: [Dovecot] Config review (2.0.5)
* Daniel L. Miller dmil...@amfes.com:
# plugin Konfiguration
plugin {
# mailboxen anlegen und subscriben
autocreate = Trash
autocreate2 = spam
autocreate3 = Sent
autocreate4 = Drafts
autosubscribe = Trash
autosubscribe2 = spam
autosubscribe3 = Sent
autosubscribe4 = Drafts
# FullTextSearch
fts = squat
I'm probably talking out of the wrong hole again - but have you tried
removing squat from your plugin list to see if it makes a difference?
I thought of that as well. I'll try that tonight :)
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hildebra...@charite.de | http://www.charite.de
