Re: [Dovecot] too many open files and v1.2

[Michael Moritz]

Hi Timo

thanks for your reply. I understand you're basically saying "upgrade to v2" but 
maybe this could be of interest for you. After we lowered the  
login_max_processes_count again the problem reappeared so I'm not sure whether 
that doesn't indicate a different problem.

Regards,

Michael

On Tuesday 18 October 2011 15:57:23 Timo Sirainen wrote:
> On Thu, 2011-10-06 at 12:42 +0100, Michael Moritz wrote:
> > Hi
> > 
> > I've found an older thread 
> > http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions 
> > that this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got 
> > errors like these recently
> > 
> > Oct  5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files
> > Oct  5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files
> > Oct  5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login 
> > processes, slowing down for now
> 
> These errors come from the dovecot master process. Each child process
> uses up a few fds for pipes.
> 
> > As a work around I've increased login_max_processes_count from 256 to 512 
> > and it seems to have helped (as I suspected stale opened files).
> 
> That only makes it easier to reach, since now there can be more child
> processes eating up more fds.
> 
> > My colleague thinks it's hitting the kernel open file limit ulimit -n 
> > (currently at 1024) but I don't see how that could happen if the number of 
> > process is nowhere near that.
> 
> I don't remember how many fds each process takes in v1.x, probably
> between 1 and 3. So if you for a while had hundreds of imap or pop3
> processes, you could run into that limit.
> 
> v2.x doesn't use as many fds.
> 
> 

Re: [Dovecot] too many open files and v1.2

[Timo Sirainen]

On Thu, 2011-10-06 at 12:42 +0100, Michael Moritz wrote:
> Hi
> 
> I've found an older thread 
> http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions 
> that this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got 
> errors like these recently
> 
> Oct  5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files
> Oct  5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files
> Oct  5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login 
> processes, slowing down for now

These errors come from the dovecot master process. Each child process
uses up a few fds for pipes.

> As a work around I've increased login_max_processes_count from 256 to 512 and 
> it seems to have helped (as I suspected stale opened files).

That only makes it easier to reach, since now there can be more child
processes eating up more fds.

> My colleague thinks it's hitting the kernel open file limit ulimit -n 
> (currently at 1024) but I don't see how that could happen if the number of 
> process is nowhere near that.

I don't remember how many fds each process takes in v1.x, probably
between 1 and 3. So if you for a while had hundreds of imap or pop3
processes, you could run into that limit.

v2.x doesn't use as many fds.

[Dovecot] too many open files and v1.2

[Michael Moritz]

Hi

I've found an older thread 
http://www.dovecot.org/list/dovecot/2010-March/047886.html which mentions that 
this is a bug in version 1. I'm on Debian squeeze (1:1.2.15-7) and got errors 
like these recently

Oct  5 11:29:29 mail dovecot: dovecot: pipe() failed: Too many open files
Oct  5 11:29:32 mail dovecot: dovecot: pipe() failed: Too many open files
Oct  5 11:29:32 mail dovecot: dovecot: Temporary failure in creating login 
processes, slowing down for now

As a work around I've increased login_max_processes_count from 256 to 512 and 
it seems to have helped (as I suspected stale opened files).

My colleague thinks it's hitting the kernel open file limit ulimit -n 
(currently at 1024) but I don't see how that could happen if the number of 
process is nowhere near that.

Any ideas?

Thanks,

Michael

Re: [Dovecot] Too many open files

[Antonio Casado Rodríguez]

Quey escribió:

On 22/10/08 08:09:23, Antonio Casado Rodríguez wrote:

I forgot something,  i use dovecot 1.0.15 in RedHat Enterprise Linux 5.0

Do you know the problem?



Timo has already pointed this out to you.

Perfect.
In your, hrmm not used RH servers in years being a Skackware'r, but I 
think its in file /etc/init.d/dovecot ?

At the top of this file, right after the shell declaration, put in

ulimit -n 8192

Ok, I put it in /etc/init.d/dovecotand now i have:

# more /proc/`cat /var/run/dovecot/master.pid`/limits
Limit Soft Limit   Hard Limit   Units
Max cpu time  unlimitedunlimitedseconds
Max file size unlimitedunlimitedbytes
Max data size unlimitedunlimitedbytes
Max stack size10485760 unlimitedbytes
Max core file size0unlimitedbytes
Max resident set  unlimitedunlimitedbytes
Max processes 8191981919
processes

Max open files8192 8192 files
Max locked memory 3276832768bytes
Max address space unlimitedunlimitedbytes
Max file locksunlimitedunlimitedlocks
Max pending signals   8191981919signals
Max msgqueue size 819200   819200   bytes
Max nice priority 00
Max realtime priority 00


Thanks you.

Re: [Dovecot] Too many open files

[Quey]

On 22/10/08 08:09:23, Antonio Casado Rodríguez wrote:
I forgot something,  i use dovecot 1.0.15 in RedHat Enterprise  
Linux 5.0


Do you know the problem?



Timo has already pointed this out to you..

In your, hrmm not used RH servers in years being a Skackware'r, but I  
think its in file /etc/init.d/dovecot ?

At the top of this file, right after the shell declaration, put in

ulimit -n 8192

Re: [Dovecot] Too many open files

[Quey]

On 22/10/08 07:43:49, Timo Sirainen wrote:



What Dovecot version are you using? If v1.1+, doesn't it give a
warning
at startup that your limit is too low?



Just so you know dovecot does what you coded it to, yes it gives  
warning, dovecot -n in normal shell shows it, adn I think once I was  
lazy and started it without the rc file and was warned as well.

Re: [Dovecot] Too many open files

[Antonio Casado]

I forgot something,  i use dovecot 1.0.15 in RedHat Enterprise Linux 5.0

Do you know the problem?

Thank you.

Antonio Casado Rodríguez escribió:

Hi, other error in "open files":

dovecot: Oct 20 10:52:36 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:52:36 Error: child 1038 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:51 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:51 Error: child 4853 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:53 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:53 Error: child 4898 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:54 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:54 Error: child 4906 (imap) returned error 84 
(exec() failed)


# ulimit -a
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 81919
max locked memory   (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files  (-n) 1024
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 10240
cpu time   (seconds, -t) unlimited
max user processes  (-u) 81919
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited

We work in "High-performance mode" with
login_max_processes_count = 256

Our script "post-login-imap-dovecot" is:
#!/bin/sh
touch ~/.last_login
exec /usr/libexec/dovecot/imap

Now, we have this:
# lsof -p `cat /var/run/dovecot/master.pid` | wc -l
1018

¿Any idea? Thanks a lot.

Bye.

Re: [Dovecot] Too many open files

[Antonio Casado Rodríguez]

I forgot something,  i use dovecot 1.0.15 in RedHat Enterprise Linux 5.0

Do you know the problem?

Thank you.

Hi, other error in "open files":

dovecot: Oct 20 10:52:36 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:52:36 Error: child 1038 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:51 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:51 Error: child 4853 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:53 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:53 Error: child 4898 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:54 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open 
files
dovecot: Oct 20 10:55:54 Error: child 4906 (imap) returned error 84 
(exec() failed)


# ulimit -a
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 81919
max locked memory   (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files  (-n) 1024
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 10240
cpu time   (seconds, -t) unlimited
max user processes  (-u) 81919
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited

We work in "High-performance mode" with
login_max_processes_count = 256

Our script "post-login-imap-dovecot" is:
#!/bin/sh
touch ~/.last_login
exec /usr/libexec/dovecot/imap

Now, we have this:
# lsof -p `cat /var/run/dovecot/master.pid` | wc -l
1018

¿Any idea? Thanks a lot.

Bye.

Re: [Dovecot] Too many open files

[Timo Sirainen]

On Tue, 2008-10-21 at 10:47 +0200, Antonio Casado Rodríguez wrote:
> Hi, other error in "open files":
> 
> dovecot: Oct 20 10:52:36 Fatal: 
> execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open files
> dovecot: Oct 20 10:52:36 Error: child 1038 (imap) returned error 84 
> (exec() failed)

This comes from the dovecot master process.

> open files  (-n) 1024

And it's limited by this.

> Now, we have this:
> # lsof -p `cat /var/run/dovecot/master.pid` | wc -l
> 1018

So increase ulimit -n before starting Dovecot.

What Dovecot version are you using? If v1.1+, doesn't it give a warning
at startup that your limit is too low?


signature.asc
Description: This is a digitally signed message part

[Dovecot] Too many open files

[Antonio Casado Rodríguez]

Hi, other error in "open files":

dovecot: Oct 20 10:52:36 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open files
dovecot: Oct 20 10:52:36 Error: child 1038 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:51 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open files
dovecot: Oct 20 10:55:51 Error: child 4853 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:53 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open files
dovecot: Oct 20 10:55:53 Error: child 4898 (imap) returned error 84 
(exec() failed)
dovecot: Oct 20 10:55:54 Fatal: 
execv(/usr/local/sbin/post-login-imap-dovecot) failed: Too many open files
dovecot: Oct 20 10:55:54 Error: child 4906 (imap) returned error 84 
(exec() failed)


# ulimit -a
core file size  (blocks, -c) 0
data seg size   (kbytes, -d) unlimited
scheduling priority (-e) 0
file size   (blocks, -f) unlimited
pending signals (-i) 81919
max locked memory   (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files  (-n) 1024
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority  (-r) 0
stack size  (kbytes, -s) 10240
cpu time   (seconds, -t) unlimited
max user processes  (-u) 81919
virtual memory  (kbytes, -v) unlimited
file locks  (-x) unlimited

We work in "High-performance mode" with
login_max_processes_count = 256

Our script "post-login-imap-dovecot" is:
#!/bin/sh
touch ~/.last_login
exec /usr/libexec/dovecot/imap

Now, we have this:
# lsof -p `cat /var/run/dovecot/master.pid` | wc -l
1018

¿Any idea? Thanks a lot.

Bye.

Re: [Dovecot] Too many open files

[Tim Tyler]

   We were running 1.015r (since that is what Redhat distributes) 
and had the "Too many open files" problem.  We were using PAM at 
first and then userdb ldap.  We upgraded to 1.10 and all seems to be 
fixed for us now.  I have no idea about the in between 
versions.   Upgrading to 1.10 seems to work.

Tim

At 03:30 AM 1/10/2008, you wrote:

On Thu, 2008-01-10 at 07:41 +0100, Claude Frantz wrote:
> Hello !
>
> The problem with "Too many open files" has disappeared here when using
> the version dovecot.i386 1:1.0.9-0_65.fc7 on a system running Fedora 7.
> With the more recent version dovecot.i386 1:1.0.10-0_66.fc7, the problem
> occurs again as it did previously.

There's hardly any difference between 1.0.9 and 1.0.10, except if you're
using LDAP passdb/userdb. Unless the RPM was packaged differently, I'd
say it's just a chance that 1.0.9 happens to (appear to) work and 1.0.10
doesn't.




Tim Tyler
Network Engineer - Beloit College
[EMAIL PROTECTED] 

Re: [Dovecot] Too many open files

[Claude Frantz]

Timo Sirainen wrote:

The problem with "Too many open files" has disappeared here when using 
the version dovecot.i386 1:1.0.9-0_65.fc7 on a system running Fedora 7.
With the more recent version dovecot.i386 1:1.0.10-0_66.fc7, the problem 
occurs again as it did previously.


There's hardly any difference between 1.0.9 and 1.0.10, except if you're
using LDAP passdb/userdb. Unless the RPM was packaged differently, I'd
say it's just a chance that 1.0.9 happens to (appear to) work and 1.0.10
doesn't.


In fact, I'm using LDAP passdb/userdb.

Re: [Dovecot] Too many open files

[Timo Sirainen]

On Thu, 2008-01-10 at 07:41 +0100, Claude Frantz wrote:
> Hello !
> 
> The problem with "Too many open files" has disappeared here when using 
> the version dovecot.i386 1:1.0.9-0_65.fc7 on a system running Fedora 7.
> With the more recent version dovecot.i386 1:1.0.10-0_66.fc7, the problem 
> occurs again as it did previously.

There's hardly any difference between 1.0.9 and 1.0.10, except if you're
using LDAP passdb/userdb. Unless the RPM was packaged differently, I'd
say it's just a chance that 1.0.9 happens to (appear to) work and 1.0.10
doesn't.



signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] Too many open files

[Claude Frantz]

Hello !

The problem with "Too many open files" has disappeared here when using 
the version dovecot.i386 1:1.0.9-0_65.fc7 on a system running Fedora 7.
With the more recent version dovecot.i386 1:1.0.10-0_66.fc7, the problem 
occurs again as it did previously.


Thanks a lot !
Claude

Re: [Dovecot] Too many open files

[Harrison Metzger]

The version I was running was 1.0-rc15 from Redhat 5. I'm not sure how many
dovecot -auth processes were spawed. We upgraded to dovecot 1.0.10 and our
problem was solved. Unfortunately, this problem happened on a live system
and we can not put back the older version. It would appear that somwhere
between these version the problem got  resolved.

Harrison Metzger

On Jan 9, 2008 10:16 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote:

> On Tue, 2008-01-08 at 09:42 -0600, Harrison Metzger wrote:
> > Hello,
> >
> > I have dovecot installed and it works for about 3 hours then I get this
> > error (in /var/log/maillog):
> >
> > Jan  8 08:42:53 stu dovecot: auth(default): pam(grossmsm,69.131.100.47):
> > pipe() failed: Too many open files
> >
> > I set pam_limits.so to allow the user 'dovecot' to have 8192 open files,
> and
> > I also changed:
>
> dovecot-auth isn't (or shouldn't!) be running as "dovecot" user. See
> http://wiki.dovecot.org/UserIds. Also I don't think pam_limits settings
> matter for dovecot-auth, since it's started by dovecot master which was
> started as root, and PAM is never called in the middle. So the only
> thing that affects number of fds is "ulimit -n" value before dovecot was
> started.
>
> What Dovecot version are you using? How many dovecot-auth processes do
> you have running after a while?
>
>

Re: [Dovecot] Too many open files

[Timo Sirainen]

On Tue, 2008-01-08 at 09:42 -0600, Harrison Metzger wrote:
> Hello,
> 
> I have dovecot installed and it works for about 3 hours then I get this
> error (in /var/log/maillog):
> 
> Jan  8 08:42:53 stu dovecot: auth(default): pam(grossmsm,69.131.100.47):
> pipe() failed: Too many open files
> 
> I set pam_limits.so to allow the user 'dovecot' to have 8192 open files, and
> I also changed:

dovecot-auth isn't (or shouldn't!) be running as "dovecot" user. See
http://wiki.dovecot.org/UserIds. Also I don't think pam_limits settings
matter for dovecot-auth, since it's started by dovecot master which was
started as root, and PAM is never called in the middle. So the only
thing that affects number of fds is "ulimit -n" value before dovecot was
started.

What Dovecot version are you using? How many dovecot-auth processes do
you have running after a while?



signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] Dovecot too many open files???

[Charles Marcus]

Tim Tyler, on 1/8/2008 12:54 PM, said the following:

Dovecot users,
  I guess I should complain to Redhat.  Their latest rhl release of 
Dovecot is 1.0.rc15 on redhat 5.   Its even older if you are running on 
redhat 4 which ironically is much more stable.  I even tried to run 
dovecot directly to ldap using userdb ldap, but that would fail after a 
few minutes.  We are now downloading and installing 1.010.  So far it 
appears to be stable, but we shall see over the rest of the afternoon.  
Too early to judge.  I hate having to run source code installs, but it 
may be the only way in this case.  I guess I would expect better from a 
commercial linux company in the stability of what they release.


ATRPMs.net keeps up to date rpms for RHEL 3, 4 and 5...

For 5 is here:

http://atrpms.net/dist/el5/dovecot/

--

Best regards,

Charles

Re: [Dovecot] Dovecot too many open files???

[Tim Tyler]

Dovecot users,
  I guess I should complain to Redhat.  Their latest rhl release of 
Dovecot is 1.0.rc15 on redhat 5.   Its even older if you are running 
on redhat 4 which ironically is much more stable.  I even tried to 
run dovecot directly to ldap using userdb ldap, but that would fail 
after a few minutes.  We are now downloading and installing 
1.010.  So far it appears to be stable, but we shall see over the 
rest of the afternoon.  Too early to judge.  I hate having to run 
source code installs, but it may be the only way in this case.  I 
guess I would expect better from a commercial linux company in the 
stability of what they release.

 Tim

At 10:00 AM 1/8/2008, you wrote:

On 1/8/2008, Tim Tyler ([EMAIL PROTECTED]) wrote:

 We just recently deployed Dovecot 1.0.rc15


Ouch... old... upgrade...

--

Best regards,

Charles


Tim Tyler
Network Engineer - Beloit College
[EMAIL PROTECTED] 

Re: [Dovecot] Dovecot too many open files???

[Charles Marcus]

On 1/8/2008, Tim Tyler ([EMAIL PROTECTED]) wrote:

 We just recently deployed Dovecot 1.0.rc15


Ouch... old... upgrade...

--

Best regards,

Charles

[Dovecot] Dovecot too many open files???

[Tim Tyler]

Dovecot experts,
 We just recently deployed Dovecot 1.0.rc15 on a Redhat 5 system.  I 
have over 1400 users.  At first it appeared to work fine.  But soon, 
we had complaints of students that couldn't retrieve their email.  I 
looked in the logs and found this error.


Jan  7 22:27:31 stu dovecot: auth(default): 
pam(pucklisl,144.89.40.6): pipe() failed: Too 
many open files


We are running Dovecot to authenticate via pam which authenticates ldap.

Below is the output of
[EMAIL PROTECTED] ~]# dovecot -n
# /etc/dovecot.conf
listen: *
ssl_cert_file: /etc/mail/certs/server.pem
ssl_key_file: /etc/mail/certs/server.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_process_size: 512
login_max_processes_count: 32
mail_location: mbox:%h/mail:INBOX=/var/spool/mail/%u
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /etc/custom-pop3.sh
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/pop3
auth default:
  passdb:
driver: pam
  userdb:
driver: passwd


Any solutions for this problem?


Tim Tyler
Network Engineer - Beloit College
[EMAIL PROTECTED] 

[Dovecot] Too many open files

[Harrison Metzger]

Hello,

I have dovecot installed and it works for about 3 hours then I get this
error (in /var/log/maillog):

Jan  8 08:42:53 stu dovecot: auth(default): pam(grossmsm,69.131.100.47):
pipe() failed: Too many open files

I set pam_limits.so to allow the user 'dovecot' to have 8192 open files, and
I also changed:

login_process_size = 512

However, it still wont work for more than a few hours. When I run:

lsof -p `ps -o pid= -C dovecot-auth`

I get:

[...]
dovecot-a 1385 root  248u  sock0,5 374241289 can't identify
protocol
dovecot-a 1385 root  249u  sock0,5 374241391 can't identify
protocol
dovecot-a 1385 root  250u  sock0,5 374243642 can't identify
protocol
dovecot-a 1385 root  251u  sock0,5 374241701 can't identify
protocol
[...]

And the number followed by a u keeps growing. As of writing this email that
number is now at 274u. Is dovecot not closing its fd's or something?

Harrison Metzger