Re: [Dovecot] Auth processes/login processes count
On 15.2.2011, at 16.19, Thomas Hummel wrote:
> login_max_processes_count = 1024
>
> auth default {
> [...]
> count = 5
> }
>
> I've got several questions :
>
> 1) how can I count more than login_max_processes_count processes ?
>
> # ps aux | egrep -i '(pop3|imap)-login' | wc -l
> # 1031
It's max. 1024 for imap-login and max. 1024 for pop3-login.
> 2) could be my auth process count the bottleneck ?
> is there a rule of thumb to deduce it from the login process count ?
Well, with v2.0 you couldn't even have more than 1 auth process. LDAP is used
asynchronously, so it should work ok. Although are you using auth binds? That's
less efficient.
I think anyway the problem has to do with login processes. In v1.x new
processes are forked in a bit dummy way, which I just copied from Apache but I
don't think it really helps with anything.. Anyway, it can only double the
process count once every second. So if there's a large burst of connections,
some of them start hanging. I think you should just try if
login_process_per_connection=no solves everything. Yeah, it's less secure than
"yes", but in practice that's pretty much what all large installations use
anyway. There are no known Dovecot or OpenSSL security holes now, so it should
be safe. :)
> 3) on dovecot login/master/auth design, is my understanding correct ?
>
> - dovecot-1.2.x :
>
> . passdb and userdb lookup are done by the auth process, on behalf of either
> the
>login process which wants to know if the client's password is correct or
> of the
>master process which wants to check if it was actually correct and which
> wants
>to have the userdb info.
>
> . client connection fd is sent to the master which creates the mail process
> and
>transfers the fd to it.
Yes.
> - dovecot-2.x :
>
> . client connection fd does not go through the master anymore but through a
> socket in the login/ directory
>
> . it's the mail process which ask the auth process for the userdb lookup
Yep. The code and idea is similar, but now it's running in the mail process
rather than master process. This change also removed the possibility of having
more than one auth process (if it becomes necessary, I guess they could be
added back by having each auth process listen on a differently named unix
socket, like auth-master-).
> 4) with LDAP backend using {CRYPT} passwords dans with plaintext auth enabled
> in dovecot,
> is the clear password crypted before comparing it to the LDAP crypted
> password
Yes, if auth_bind=no.
> or does dovecot try to simply bind to LDAP with the clear password ?
Yes, if auth_bind=yes.
Re: [Dovecot] auth processes
On 18.2.2010, at 23.08, Stan Hoeppner wrote:
> Timo Sirainen put forth on 2/18/2010 12:54 PM:
>
>> Hmm. You could try setting auth_worker_max_request_count=1 to see if
>> that gets rid of the processes after they've handled the request.
>
> Restarting IMAP/POP3 mail server: dovecotError: Error in configuration file
> /etc/dovecot/dovecot.conf line 1: Unknown setting: worker_max_request_count
> Fatal: Invalid configuration in /etc/dovecot/dovecot.conf
>
> FYI I'm running 1.2.10
Oh, right, it changed in v1.2 to:
passdb pam {
args = max_requests=1
}
Re: [Dovecot] auth processes
Eric Rostetter put forth on 2/18/2010 3:21 PM:
> Quoting Stan Hoeppner :
>
>> Timo Sirainen put forth on 2/18/2010 12:54 PM:
>>
>>> Hmm. You could try setting auth_worker_max_request_count=1 to see if
>>> that gets rid of the processes after they've handled the request.
>>
>> Restarting IMAP/POP3 mail server: dovecotError: Error in configuration
>> file
>> /etc/dovecot/dovecot.conf line 1: Unknown setting:
>> worker_max_request_count
>> Fatal: Invalid configuration in /etc/dovecot/dovecot.conf
>>
>> FYI I'm running 1.2.10
>>
>> --
>> Stan
>
> Could be a typo, could be your problem, but:
>
> auth_worker_max_request_count != worker_max_request_count
>
> (i.e., did you forget the "auth_" at the start?)
I think it's sad that sometimes some sysadmins assume their fellow sysadmins are
less than capable, to put it politely.
[02:55:20][r...@greer]/home/stan$ dovecot -a|grep auth_worker_max_request_count
[02:55:26][r...@greer]/home/stan$ man dovecot
[02:56:36][r...@greer]/home/stan$ man dovecot.conf
No manual entry for dovecot.conf
[02:56:40][r...@greer]/home/stan$ vi /etc/dovecot/dovecot.conf
[03:02:06][r...@greer]/home/stan$ dovecot restart
Usage: dovecot [-F] [-c ] [-p] [-n] [-a]
[--version] [--build-options] [--exec-mail []]
Fatal: Unknown argument: restart
[03:02:11][r...@greer]/home/stan$ /etc/init.d/dovecot
Usage: /etc/init.d/dovecot {start|stop|restart|force-reload|status}
[03:02:41][r...@greer]/home/stan$ /etc/init.d/dovecot status
dovecot is running.
[03:03:10][r...@greer]/home/stan$ /etc/init.d/dovecot restart
Restarting IMAP/POP3 mail server: dovecotError: Error in configuration file
/etc/dovecot/dovecot.conf line 868: Unknown setting: worker_max_request_count
Fatal: Invalid configuration in /etc/dovecot/dovecot.conf
failed!
[03:04:33][r...@greer]/home/stan$ vi /etc/dovecot/dovecot.conf
[03:06:09][r...@greer]/home/stan$ /etc/init.d/dovecot restart
Restarting IMAP/POP3 mail server: dovecotError: Error in configuration file
/etc/dovecot/dovecot.conf line 1: Unknown setting: worker_max_request_count
Fatal: Invalid configuration in /etc/dovecot/dovecot.conf
failed!
[03:06:50][r...@greer]/home/stan$ vi /etc/dovecot/dovecot.conf
[03:07:38][r...@greer]/home/stan$ /etc/init.d/dovecot restart
Restarting IMAP/POP3 mail server: dovecot.
Now, does that string of actions look to you like I fucked up, didn't verify my
edits and results, before reporting back? Pristine? No. Perfect? No.
_Thorough_? YES.
Dovecot strips the "auth_" portion of the parameter name in that error message.
I don't know why. Do you? If you're running 1.2.10 or prior on one of your
systems, insert that parameter into dovecot.conf, do a restart or force reload,
and report back the error message, if any, that you receive.
Maybe then you might have a constructive, positive comment for me.
--
Stan
Re: [Dovecot] auth processes
Quoting Stan Hoeppner : Timo Sirainen put forth on 2/18/2010 12:54 PM: Hmm. You could try setting auth_worker_max_request_count=1 to see if that gets rid of the processes after they've handled the request. Restarting IMAP/POP3 mail server: dovecotError: Error in configuration file /etc/dovecot/dovecot.conf line 1: Unknown setting: worker_max_request_count Fatal: Invalid configuration in /etc/dovecot/dovecot.conf FYI I'm running 1.2.10 -- Stan Could be a typo, could be your problem, but: auth_worker_max_request_count != worker_max_request_count (i.e., did you forget the "auth_" at the start?) -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns!
Re: [Dovecot] auth processes
Timo Sirainen put forth on 2/18/2010 12:54 PM: > Hmm. You could try setting auth_worker_max_request_count=1 to see if > that gets rid of the processes after they've handled the request. Restarting IMAP/POP3 mail server: dovecotError: Error in configuration file /etc/dovecot/dovecot.conf line 1: Unknown setting: worker_max_request_count Fatal: Invalid configuration in /etc/dovecot/dovecot.conf FYI I'm running 1.2.10 -- Stan
Re: [Dovecot] auth processes
On Thu, 2010-02-18 at 12:47 -0600, Stan Hoeppner wrote: > > Are you using passwd/pam combination? The second process is "auth > > worker", which does blocking passdb/userdb lookups. The v1.0 behavior > > was causing bugs. > > Yep: > auth default: > worker_max_count: 1 > process_size: 16 > passdb: > driver: pam > userdb: > driver: passwd > > If this is what the worker does, what does the "master" do? With v1.0 each PAM lookup caused dovecot-auth to fork a new process, which then did the PAM stuff and then exited. With v1.1+ the auth worker process does this so that there's no need for the forking (that caused problems with some nss/pam combinations). So dovecot-auth master gets the actual auth requests and starts handling them. When it needs to do a blocking passdb/userdb lookup, it connects to auth worker, which then does the actual (potentially long running) lookup. If you've enough load, more auth worker processes are created as necessary. Hmm. You could try setting auth_worker_max_request_count=1 to see if that gets rid of the processes after they've handled the request. > > Currently there's no way to avoid that, except if you switch from PAM to > > e.g. shadow. v2.0 stops idling processes after a minute (but there are a > > some other new long running processes, and also it looks like that > > idle-stopping isn't currently working for some reason). > > IIRC both dovecot-auth processes are staying resident even with no clients > connected. Yes, only v2.0 has the idle-stopping feature. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] auth processes
Timo Sirainen put forth on 2/18/2010 9:17 AM: > On Tue, 2010-02-16 at 08:16 -0600, Stan Hoeppner wrote: >> With 1.0.15 my configuration I a single dovecot-auth process, which is what I >> want now. Since upgrading to 1.2.10 I have two such processes: >> >> root 16992 16990 0 Feb15 ?00:00:00 dovecot-auth >> root 16997 16990 0 Feb15 ?00:00:00 dovecot-auth -w >> >> I can't seem to tweak dovecot.conf to get this down to one auth process >> again. > > Are you using passwd/pam combination? The second process is "auth > worker", which does blocking passdb/userdb lookups. The v1.0 behavior > was causing bugs. Yep: auth default: worker_max_count: 1 process_size: 16 passdb: driver: pam userdb: driver: passwd If this is what the worker does, what does the "master" do? > Currently there's no way to avoid that, except if you switch from PAM to > e.g. shadow. v2.0 stops idling processes after a minute (but there are a > some other new long running processes, and also it looks like that > idle-stopping isn't currently working for some reason). IIRC both dovecot-auth processes are staying resident even with no clients connected. >> running all the dovecot processes >> from inetd is even a valid option, although I've not yet attempted that. > > That won't help anyway. You'd still have the same processes. It's not that big a deal. I just didn't realize this had changed since 1.0.15, so I thought I just needed to tweak something to ditch the extra process. If both auth processes are required for pam/passwd to function correctly in 1.2.10 then that's fine by me. Thanks Timo. -- Stan
Re: [Dovecot] auth processes
On Tue, 2010-02-16 at 08:16 -0600, Stan Hoeppner wrote: > With 1.0.15 my configuration I a single dovecot-auth process, which is what I > want now. Since upgrading to 1.2.10 I have two such processes: > > root 16992 16990 0 Feb15 ?00:00:00 dovecot-auth > root 16997 16990 0 Feb15 ?00:00:00 dovecot-auth -w > > I can't seem to tweak dovecot.conf to get this down to one auth process again. Are you using passwd/pam combination? The second process is "auth worker", which does blocking passdb/userdb lookups. The v1.0 behavior was causing bugs. > Given my IMAP user load, I know for a fact that I should only need one auth > process, and for that matter, given my load, Currently there's no way to avoid that, except if you switch from PAM to e.g. shadow. v2.0 stops idling processes after a minute (but there are a some other new long running processes, and also it looks like that idle-stopping isn't currently working for some reason). > running all the dovecot processes > from inetd is even a valid option, although I've not yet attempted that. That won't help anyway. You'd still have the same processes. signature.asc Description: This is a digitally signed message part
