Re: Frequent Out of Memory for service(config)
Hello Aki + group. The following is the latest dump output that I have. Does this give any indication of what is wrong? I am not sure what this is really showing me... # coredumpctl gdb -1 PID: 14432 (config) UID: 0 (root) GID: 0 (root) Signal: 6 (ABRT) Timestamp: Wed 2019-06-05 13:25:32 UTC (4h 52min ago) Command Line: dovecot-Pop3 Mail Service/config Executable: /usr/libexec/dovecot/config Control Group: /system.slice/dovecot.service Unit: dovecot.service Slice: system.slice Boot ID: 9ae422871a814d699f5feb4ca52d3b69 Machine ID: 05cb8c7b39fe0f70e3ce97e5beab809d Hostname: <*REMOVED**> Coredump: /var/lib/systemd/coredump/core.config.0.9ae422871a814d699f5feb4ca52d3b69.14432.155974113200.xz Message: Process 14432 (config) of user 0 dumped core. Stack trace of thread 14432: #0 0x7fd25a809207 raise (libc.so.6) #1 0x7fd25a80a8f8 abort (libc.so.6) #2 0x7fd25ac79567 fatal_handler_real (libdovecot.so.0) #3 0x7fd25ac79651 i_internal_fatal_handler (libdovecot.so.0) #4 0x7fd25abdf2d9 i_fatal_status (libdovecot.so.0) #5 0x7fd25ac9b900 pool_system_malloc (libdovecot.so.0) #6 0x7fd25aca220a o_stream_grow_buffer (libdovecot.so.0) #7 0x7fd25aca2506 o_stream_add (libdovecot.so.0) #8 0x7fd25aca3388 o_stream_file_sendv (libdovecot.so.0) #9 0x7fd25aca03f5 o_stream_sendv_int (libdovecot.so.0) #10 0x7fd25aca0a5c o_stream_nsendv (libdovecot.so.0) #11 0x7fd25aca0aca o_stream_nsend (libdovecot.so.0) #12 0x556e753617dc config_connection_input (config) #13 0x7fd25ac91e0f io_loop_call_io (libdovecot.so.0) #14 0x7fd25ac9384b io_loop_handler_run_internal (libdovecot.so.0) #15 0x7fd25ac91f16 io_loop_handler_run (libdovecot.so.0) #16 0x7fd25ac92138 io_loop_run (libdovecot.so.0) #17 0x7fd25ac07973 master_service_run (libdovecot.so.0) #18 0x556e7535edc9 main (config) #19 0x7fd25a7f53d5 __libc_start_main (libc.so.6) #20 0x556e7535ee3b _start (config) GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/dovecot/config...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/config.debug...done. done. [New LWP 14432] Core was generated by `dovecot-Pop3 Mail Service/config'. Program terminated with signal 6, Aborted. #0 0x7fd25a809207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 55return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); Thanks, Kevin On Wed, May 15, 2019 at 4:34 AM Aki Tuomi wrote: > > On 13.5.2019 22.56, Root Kev via dovecot wrote: > > Hello Group, > > > > We have dovecot deployed as solely a Pop3 service that is used by our > > applications to pass mail from one application to another internally. > > We have roughly 4 applications that connect to the Pop3 service every > > 2 seconds, to check for new messages and pop them for processing if > > they are present. Depending on the site, we have between 1024-2048MB > > of memory set for default_vsz_limit. In all systems we see the Out of > > memory alert several times a day. We previously did not see this at > > all when running on CentOS6, with less memory. > > > > We have tried increasing the memory to the vsz_limit up to 2gb without > > success. > > > > We are running on CentOS 7 servers, running dovecot 2.3.6 (7eab80676) > > (from the dovecot repo). > > > > Can anyone advise any other settings that could be modified in order > > to correct these out of memory issues? > > > > # dovecot -n > > # 2.3.6 (7eab80676): /etc/dovecot/dovecot.conf > > # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release > > 7.6.1810 (Core) > > # Hostname: ** # > > auth_cache_size = 10 M > > auth_verbose = yes > > default_vsz_limit = 1 G > > instance_name = Pop3 Mail Service > > listen = 10.*.*.* # > > log_path = /var/log/dovecot.log > > login_greeting =
Re: Frequent Out of Memory for service(config)
Hey Tom, We don't run any quota-service in dovecot (I just verified just in case as well). Thanks for the suggestion though. Kevin On Sun, May 19, 2019 at 9:21 AM Tom Sommer via dovecot wrote: > > > On 2019-05-13 21:56, Root Kev via dovecot wrote: > > > Hello Group, > > > > We have dovecot deployed as solely a Pop3 service that is used by our > > applications to pass mail from one application to another internally. > > We have roughly 4 applications that connect to the Pop3 service every 2 > > seconds, to check for new messages and pop them for processing if they > > are present. Depending on the site, we have between 1024-2048MB of > > memory set for default_vsz_limit. In all systems we see the Out of > > memory alert several times a day. We previously did not see this at all > > when running on CentOS6, with less memory. > > I see this too on servers running quota-service (dunno if it is > related). > > --- > Tom >
Re: Frequent Out of Memory for service(config)
Hi Aki,
I put in the line that you recommended as well as the " echo
'DAEMON_COREFILE_LIMIT="unlimited"' >> /etc/sysconfig/dovecot".
Unfortunately, the logs are saying that core dumps are still disabled.
May 17 13:30:17 config: Fatal: pool_system_malloc(8192): Out of memory
May 17 13:30:17 pop3(emx-echoworx): Error: Error reading configuration:
read(/var/run/dovecot/config) failed: EOF
May 17 13:30:17 config: Fatal: master: service(config): child 11530 killed
with signal 6 (core dumps disabled -
https://dovecot.org/bugreport.html#coredumps)
On Wed, May 15, 2019 at 4:34 AM Aki Tuomi
wrote:
>
> On 13.5.2019 22.56, Root Kev via dovecot wrote:
> > Hello Group,
> >
> > We have dovecot deployed as solely a Pop3 service that is used by our
> > applications to pass mail from one application to another internally.
> > We have roughly 4 applications that connect to the Pop3 service every
> > 2 seconds, to check for new messages and pop them for processing if
> > they are present. Depending on the site, we have between 1024-2048MB
> > of memory set for default_vsz_limit. In all systems we see the Out of
> > memory alert several times a day. We previously did not see this at
> > all when running on CentOS6, with less memory.
> >
> > We have tried increasing the memory to the vsz_limit up to 2gb without
> > success.
> >
> > We are running on CentOS 7 servers, running dovecot 2.3.6 (7eab80676)
> > (from the dovecot repo).
> >
> > Can anyone advise any other settings that could be modified in order
> > to correct these out of memory issues?
> >
> > # dovecot -n
> > # 2.3.6 (7eab80676): /etc/dovecot/dovecot.conf
> > # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release
> > 7.6.1810 (Core)
> > # Hostname: ** #
> > auth_cache_size = 10 M
> > auth_verbose = yes
> > default_vsz_limit = 1 G
> > instance_name = Pop3 Mail Service
> > listen = 10.*.*.* #
> > log_path = /var/log/dovecot.log
> > login_greeting = Pop3 Mail Service
> > login_trusted_networks = 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.*
> > #
> > mail_location = maildir:~/Maildir
> > namespace inbox {
> > inbox = yes
> > location =
> > mailbox Drafts {
> > special_use = \Drafts
> > }
> > mailbox Junk {
> > special_use = \Junk
> > }
> > mailbox Sent {
> > special_use = \Sent
> > }
> > mailbox "Sent Messages" {
> > special_use = \Sent
> > }
> > mailbox Trash {
> > special_use = \Trash
> > }
> > prefix =
> > }
> > passdb {
> > args = cache_key=#hidden_use-P_to_show#
> > driver = pam
> > }
> > protocols = pop3
> > ssl_cert = > ssl_key = # hidden, use -P to show it
> > userdb {
> > driver = passwd
> > }
> > verbose_ssl = yes
> >
> > May 10 06:44:05 config: Fatal: pool_system_malloc(8192): Out of memory
> > May 10 06:44:05 config: Fatal: master: service(config): child 27887
> > returned error 83 (Out of memory (service config { vsz_limit=1024 MB
> > }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to
> > get core dump)
>
> Can you try setting
>
> import_environment = $import_environment CORE_OUTOFMEM=1
>
> and see if it causes coredump?
>
> Aki
>
>
Frequent Out of Memory for service(config)
Hello Group,
We have dovecot deployed as solely a Pop3 service that is used by our
applications to pass mail from one application to another internally. We
have roughly 4 applications that connect to the Pop3 service every 2
seconds, to check for new messages and pop them for processing if they are
present. Depending on the site, we have between 1024-2048MB of memory set
for default_vsz_limit. In all systems we see the Out of memory alert
several times a day. We previously did not see this at all when running on
CentOS6, with less memory.
We have tried increasing the memory to the vsz_limit up to 2gb without
success.
We are running on CentOS 7 servers, running dovecot 2.3.6 (7eab80676) (from
the dovecot repo).
Can anyone advise any other settings that could be modified in order to
correct these out of memory issues?
# dovecot -n
# 2.3.6 (7eab80676): /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810
(Core)
# Hostname: ** #
auth_cache_size = 10 M
auth_verbose = yes
default_vsz_limit = 1 G
instance_name = Pop3 Mail Service
listen = 10.*.*.* #
log_path = /var/log/dovecot.log
login_greeting = Pop3 Mail Service
login_trusted_networks = 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.*
#
mail_location = maildir:~/Maildir
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = cache_key=#hidden_use-P_to_show#
driver = pam
}
protocols = pop3
ssl_cert =
[no subject]
Dovecot Version: 2.3.2.1
Hello Mailing List,
We are having a random issue in a couple of our production servers, where
one of the child processes randomly dies with an out of memory error (see
below):
Nov 26 11:58:17 config: Fatal: pool_system_malloc(8192): Out of memory
Nov 26 11:58:17 pop3-login: Fatal: Error reading configuration:
read(/var/run/dovecot/config) failed: EOF
Nov 26 11:58:17 config: Fatal: master: service(config): child 29696
returned error 83 (Out of memory (service config { vsz_limit=2048 MB }, you
may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)
We only use dovecot for internal application POP3 mail access from a
mailbox, and there is under 10 connecting applications. Have have
gradually increased the vsz_limit from the default of 256mb up to 2GB now.
Is there anything else that should/could be changed instead of continuing
to through memory at it?
# dovecot -n
# 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-862.11.6.el7.x86_64 x86_64 CentOS Linux release 7.5.1804
(Core)
# Hostname:
doveconf: Warning: please set ssl_dh= /etc/dovecot/dh.pem
auth_cache_size = 10 M
auth_verbose = yes
base_dir = /var/run/dovecot/
default_vsz_limit = 2 G
instance_name = EMX Pop Mailstore
listen =
log_path = /var/log/dovecot-echo.log
login_greeting = Pop3 MailServer Ready.
login_trusted_networks =
mail_location = maildir:~/Maildir
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = cache_key=%u
driver = pam
}
protocols = pop3
ssl_cert =
Re: [Dovecot] High level of pop3 popping causing server to become unresponsive
Thierry,
Had a chance to test this change this morning, and in my test environment,
this does drastically improve the ability to ssh and su during heavy pop3
load (in test environment, change of 10-15sec to 1-2sec login). While this
is better, the popping is still slowing down authentication on the box
during load. Is there anything else that might be changed to increase
speed?
Just a recap, I basically have between 2-8 mailboxes on each server, and an
application that creates two connections to each mailbox, processes each
message that comes in, and then deleted each message from the pop3
mailbox. During high load times (500+ messages) with the settings that I
had, I was unable to reconnect to the server, nor su to root. Ended up
having to stop the dovecot process and revert to popa3d (luckily had a ssh
session still open to the box).
This is the config from my test environment:
root@devsmtp:~# dovecot -c /usr/local/etc/dovecot/dovecot-test.conf -n
# 2.1.4: /usr/local/etc/dovecot/dovecot-test.conf
# OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4
auth_verbose = yes
base_dir = /var/run/dovecot-test/
disable_plaintext_auth = no
instance_name = dovecot-test
listen = removed
login_greeting = Dovecot-Test for removing index
mail_location = mbox:/var/empty:INBOX=/var/mail/%u
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = shadow
}
protocols = pop3
service pop3-login {
inet_listener pop3 {
port = 130
}
service_count = 0
}
ssl = no
userdb {
driver = passwd
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
And from one of my production servers that has the issue:
# 2.1.4: /usr/local/etc/dovecot/dovecot.conf-ml
# OS: Linux 2.6.18-194.17.1.el5 x86_64 Red Hat Enterprise Linux Server
release 5.6 (Tikanga) ext3
auth_verbose = yes
base_dir = /var/run/dovecotml/
disable_plaintext_auth = no
instance_name = Popper
listen = removed
login_greeting = Popper
mail_location =
mbox:/var/empty:INBOX=/opt/mailstore/spool/mail/%u:INDEX=MEMORY
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = shadow
}
protocols = pop3
ssl = no
userdb {
args = blocking=yes
driver = passwd
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
On Fri, May 25, 2012 at 2:24 PM, Thierry de Montaudry thie...@odry.netwrote:
On 25 May 2012, at 20:00, Root Kev root@gmail.com wrote:
So the best way to would be to remove it that part completely, or should
it be stored somewhere on disk?
Thanks again,
Kevin
yes, best is to remove :INDEX=MEMORY part, and it will store indexes in
your INBOX path, which is fine.
If you want to put it in a different path, you can have something like
INDEX=/path/to/indexes/%u
Regards,
Thierry
Hi,
Having a system with a third of our users on POP3 (230 of them), no
trouble with dovecot (v2.1.5, on CentOS 5).
But one thing surprises me in your config, the INDEX=MEMORY in the
location parameter. That means that for each POP3 connection, dovecot has
to read each and every mails to create the index in memory. That might be
why the machine becomes unresponsive.
Unless you have a specific reason to use memory index (and I would be
curious to know about it), I would suggest to remove this and keep standard
file indexes, and your performances should improve a lot.
Regards,
Thierry
Re: [Dovecot] High level of pop3 popping causing server to become unresponsive
Currently cannot use IMAP as our application to access the mailbox
currently is only setup to access pop3 mailboxes. We are currently using
Dovecot 2.1.4. Below is the majority of our configs, it is mostly basic:
protocols = pop3
listen = ***Address here***
base_dir = /var/run/dovecot1/
instance_name = Popper
login_greeting = Popper
mail_location =
mbox:/var/empty:INBOX=/opt/mailstore/spool/mail/%u:INDEX=MEMORY
!include conf.d/*.conf
!include_try local.conf
disable_plaintext_auth = no
auth_mechanisms = plain
log_path = syslog
syslog_facility = mail
auth_verbose = yes
log_timestamp = %b %d %H:%M:%S
namespace inbox {
inbox = yes
}
mail_privileged_group = mail
lock_method = fcntl
mbox_read_locks = fcntl
mbox_write_locks = dotlock fcntl
pop3_uidl_format = %08Xu%08Xv
passdb {
driver = checkpassword
args = /usr/bin/checkpassword
}
userdb {
driver = prefetch
}
Re: [Dovecot] High level of pop3 popping causing server to become unresponsive
Missed CCing on last reply. See below.. Also, would having Nagios checking the number of messages in a mailbox cause issues with the popping of messages? And/Or would having a user accessing the mailbox from two different applications cause issues? Ie. from Outlook and mobile device? I am under the impression that a lock file is generated which should deal with the issue of contention, deleting mail, etc. Thanks for any information. Kevin Sorry for the delay in responding, long weekend in Canada... When trying to SSH into the server, the server prompts for user name then password, then just hangs. Same if there is already a console connection over, when trying to SU to root, it just hangs after entering the password. Our passwords are in the shadow file, and because this is a server dedicated for this task, there is only the default linux users, and maybe 8 other user accounts in the shadow file. There shouldn't be high IO as all this box does is postfix, pop3ad and dovecot. This box is seeing less then 3000 emails a day, and only has 5 mailboxes on it. Thanks for the continued suggestions... Kevin On Sat, May 19, 2012 at 3:33 PM, Timo Sirainen t...@iki.fi wrote: On Fri, 2012-05-18 at 09:21 -0400, Root Kev wrote: During the last time that the load went up, it became unable to login / su to root for the entire period that dovecot was running, we had to kill dovecot and go back to Popa3d until the mailq was cleared up. We are running CentOS 5.6 server. Based on TOP running at the time the CPU usage was running under 10%. Once Dovecot was killed, we were then able to log in /su again. Like Kelsey said, a very high disk IO might explain this, although normally the login should still eventually succeed. Another thing I'm wondering is if some process limit reached. How does the login/su fail, does it just hang or immediately fail with some error? We were under the impression that checking to shadow directly should be the fastest and least amount of overhead, is any of the other ways to connect have less load on authentication to PAM? Your passwords really are in /etc/shadow file, not LDAP/something else? I don't think the problem is with authentication. Reading /etc/shadow is pretty fast (unless maybe if it's a huge file) and it anyway can't block login/su from working.
Re: [Dovecot] High level of pop3 popping causing server to become unresponsive
During the last time that the load went up, it became unable to login / su to root for the entire period that dovecot was running, we had to kill dovecot and go back to Popa3d until the mailq was cleared up. We are running CentOS 5.6 server. Based on TOP running at the time the CPU usage was running under 10%. Once Dovecot was killed, we were then able to log in /su again. We were under the impression that checking to shadow directly should be the fastest and least amount of overhead, is any of the other ways to connect have less load on authentication to PAM? Thanks, Kevin On Thu, May 17, 2012 at 4:57 PM, Timo Sirainen t...@iki.fi wrote: On 17.5.2012, at 18.22, Root Kev wrote: We have put Dovecot 2.1.4 on several of our production servers (CentOS, on Dell R710, with 20GB memory, dual CPU Quad-core). We have a single instance of Dovecot running and currently have several instances of Popa3d. When there are significant amount of popping from 2 mailboxes that dovecot that is popping from (500+ msgs in the mailboxes), the popping of the messages causes the boxes to become unresponsive. We use another application that connects to the Dovecot, downloads 2-10 messages, then processes them, then sends the delete command to Dovecot. Unresponsive for a long time?.. What CentOS version? When this issue occurs we are unable to become Root, or login again if we close our ssh connection. This only occures when Dovecot is doing the popping. If we only run the older Popa3d, this doesn't occur. We believe it is caused by the way dovecot is authenticating. Sounds like PAM is hanging. Is the (CPU) load in general high at this time? We are using auth_mechanisms = plain passdb drive = shadow usedb driver = passwd args = blocking=yes Using shadow/passwd directly shouldn't affect PAM at all. So this is a rather strange problem..
[Dovecot] High level of pop3 popping causing server to become unresponsive
Hello all, We have put Dovecot 2.1.4 on several of our production servers (CentOS, on Dell R710, with 20GB memory, dual CPU Quad-core). We have a single instance of Dovecot running and currently have several instances of Popa3d. When there are significant amount of popping from 2 mailboxes that dovecot that is popping from (500+ msgs in the mailboxes), the popping of the messages causes the boxes to become unresponsive. We use another application that connects to the Dovecot, downloads 2-10 messages, then processes them, then sends the delete command to Dovecot. When this issue occurs we are unable to become Root, or login again if we close our ssh connection. This only occures when Dovecot is doing the popping. If we only run the older Popa3d, this doesn't occur. We believe it is caused by the way dovecot is authenticating. We are using auth_mechanisms = plain passdb drive = shadow usedb driver = passwd args = blocking=yes If anyone could suggest what could be causing the login issue, we would appreciate any incite to fix it! Thanks, Kevin
Re: [Dovecot] POP3 Dovecot Auth CPU usage 75%+
I think my last email may have been bounced due to attachment size, I have
put a snippet of the captures below. The CPU is still going to high
percent of usage when my test mailboxes are used. An ideas on how to bring
down the Auth CPU usage would be greatly appreciated!
Thanks,
Kevin
Stace on the Auth process:
epoll_wait(13, {{EPOLLIN, {u32=150109008, u64=150109008}}}, 29, 149958) = 1
gettimeofday({1334328634, 21072}, NULL) = 0
read(29, VERSION\t1\t1\nREQUEST\t1011351553\t3..., 1024) = 72
time(NULL) = 1334328634
writev(29, [{USER\t1011351553\tservermailbox1\ts..., 108}, {\n,
1}], 2) = 109
gettimeofday({1334328634, 27993}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 149992) = 1
gettimeofday({1334328634, 32215}, NULL) = 0
accept(11, {sa_family=AF_FILE, NULL}, [2]) = 30
fcntl64(30, F_GETFL)= 0x2 (flags O_RDWR)
fcntl64(30, F_SETFL, O_RDWR|O_NONBLOCK) = 0
gettimeofday({1334328634, 32342}, NULL) = 0
fstat64(30, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
_llseek(30, 0, 0xbffd24c0, SEEK_CUR)= -1 ESPIPE (Illegal seek)
getsockname(30, {sa_family=AF_FILE,
path=/usr/local/var/run/dovecot}, [41]) = 0
epoll_ctl(13, EPOLL_CTL_ADD, 30, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP,
{u32=150123496, u64=150123496}}) = 0
write(30, VERSION\t1\t1\nSPID\t2093\n, 22) = 22
gettimeofday({1334328634, 32625}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1
gettimeofday({1334328634, 32721}, NULL) = 0
read(30, VERSION\t1\t1\n, 1024) = 12
gettimeofday({1334328634, 32792}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=150123496, u64=150123496}}}, 29, 1000) = 1
gettimeofday({1334328634, 32883}, NULL) = 0
read(30, REQUEST\t3624009729\t3062\t16\tbe004..., 1012) = 60
time(NULL) = 1334328634
writev(30, [{USER\t3624009729\tservermailbox\tsy..., 105}, {\n,
1}], 2) = 106
gettimeofday({1334328634, 33062}, NULL) = 0
epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150094520, u64=150094520}}},
29, 999) = 1
gettimeofday({1334328634, 33766}, NULL) = 0
read(28, , 6243) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 28, {0, {u32=150094520, u64=150094520}}) = 0
close(28) = 0
epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150109008, u64=150109008}}}, 29, -1) = 1
gettimeofday({1334328634, 40036}, NULL) = 0
read(29, , 952) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 29, {0, {u32=150109008, u64=150109008}}) = 0
close(29) = 0
gettimeofday({1334328634, 40163}, NULL) = 0
gettimeofday({1334328634, 40197}, NULL) = 0
epoll_wait(13, {{EPOLLIN|EPOLLHUP, {u32=150123496, u64=150123496}}},
29, 1000) = 1
gettimeofday({1334328634, 44007}, NULL) = 0
read(30, , 952) = 0
epoll_ctl(13, EPOLL_CTL_DEL, 30, {0, {u32=150123496, u64=150123496}}) = 0
close(30) = 0
gettimeofday({1334328634, 44148}, NULL) = 0
gettimeofday({1334328634, 44184}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 1000) = 1
gettimeofday({1334328634, 52466}, NULL) = 0
read(26, AUTH\t1\tPLAIN\tservice=pop3\tlip=17..., 8170) = 122
gettimeofday({1334328634, 52582}, NULL) = 0
writev(12, [{PENALTY-GET\t172.20.20.110, 25}, {\n, 1}], 2) = 26
gettimeofday({1334328634, 52698}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 992) = 1
gettimeofday({1334328634, 52760}, NULL) = 0
read(12, 0 0\n, 424) = 4
time(NULL) = 1334328634
gettimeofday({1334328634, 93200}, NULL) = 0
writev(26, [{OK\t1\tuser=servermailbox1, 24}, {\n, 1}], 2) = 25
read(12, 0x8f36c14, 420)= -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1334328634, 93651}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=150065544, u64=150065544}}}, 29, 951) = 1
gettimeofday({1334328634, 93715}, NULL) = 0
read(26, AUTH\t2\tPLAIN\tservice=pop3\tlip=17..., 8048) = 118
gettimeofday({1334328634, 93808}, NULL) = 0
writev(12, [{PENALTY-GET\t172.20.20.110, 25}, {\n, 1}], 2) = 26
gettimeofday({1334328634, 93919}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=149924840, u64=149924840}}}, 29, 951) = 1
gettimeofday({1334328634, 93980}, NULL) = 0
read(12, 0 0\n, 420) = 4
time(NULL) = 1334328634
gettimeofday({1334328634, 133578}, NULL) = 0
writev(26, [{OK\t2\tuser=servermailbox, 23}, {\n, 1}], 2) = 24
read(12, 0x8f36c18, 416)= -1 EAGAIN (Resource
temporarily unavailable)
gettimeofday({1334328634, 133998}, NULL) = 0
epoll_wait(13, {{EPOLLIN, {u32=149927248, u64=149927248}}}, 29, 911) = 1
gettimeofday({1334328634, 134064}, NULL) = 0
accept(11, {sa_family=AF_FILE, NULL}, [2]) = 28
fcntl64(28, F_GETFL)= 0x2 (flags O_RDWR)
fcntl64(28, F_SETFL, O_RDWR|O_NONBLOCK) = 0
gettimeofday({1334328634, 134200}, NULL) = 0
fstat64(28, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
_llseek(28, 0, 0xbffd24c0, SEEK_CUR)= -1 ESPIPE (Illegal seek)
Re: [Dovecot] POP3 Dovecot Auth CPU usage 75%+
I tried making the changes that you suggested but it didn't seem to make a
noticeable difference. It should be using the shadow file directly. The
shadow file has the default Ubuntu system accounts and 16 user accounts, so
overall fairly small. The nsswitch.conf file is set as default:
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc Name Service Switch' for information about this file.
passwd: compat
group: compat
shadow: compat
hosts: files dns
networks: files
protocols: db files
services: db files
ethers: db files
rpc:db files
netgroup: nis
An example of users connecting and the Auth process using alot of CPU (from
top):
Cpu(s): 87.4%us, 8.0%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.7%hi, 1.7%si,
0.0%st
Mem: 1026096k total, 533924k used, 492172k free,60340k buffers
Swap: 1757176k total,0k used, 1757176k free, 414212k cached
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+
COMMAND
643 dovecot 20 0 3096 1616 1208 S 50.7 0.2 0:01.76
auth
644 root 20 0 3096 1524 1140 S 1.3 0.1 0:00.08
auth
642 dovenull 20 0 4276 1612 1256 S 1.0 0.2 0:00.03
pop3-login
623 root 20 0 2704 1020 772 S 0.7 0.1 0:00.02
dovecot
627 root 20 0 4344 2808 1056 S 0.7 0.3 0:00.03
config
631 syslog20 0 33916 1924 1036 S 0.3 0.2 0:01.61
rsyslogd
696 serverma 20 0 5464 2564 2040 R 0.3 0.2 0:00.01
pop3
1 root 20 0 2652 1604 1216 S 0.0 0.2 0:01.59
init
2 root 20 0 000 S 0.0 0.0 0:00.00
kthreadd
Thanks for any other ideas
Kevin
On Fri, Apr 13, 2012 at 7:55 AM, Timo Sirainen t...@iki.fi wrote:
On 12.4.2012, at 23.46, Root Kev wrote:
So is it the auth process or auth worker process? What if you add:
passdb {
driver = shadow
}
userdb {
driver = passwd
args = blocking=yes
}
does that move the CPU usage from auth to auth worker process? Is it
using /etc/shadow and /etc/passwd files? Are they large? Do you have
enabled other weird stuff in /etc/nsswitch.conf (and were there some other
files related to them as well?)
Re: [Dovecot] POP3 Dovecot Auth CPU usage 75%+
Hello all,
I hope someone can help me, I have been testing out Dovecot to switch from
popa3d which I use at the moment. When I get several users connecting and
disconnection multiple times, the Dovecot process with command Auth uses
50-90% of the CPU for the period which they are connecting. I am wondering
if there is something that I may have misconfigured, or if there is
something that I can change so that this spike doesn't occur.
If anyone could shed some light on the issue, I would appreciate it,
Kevin
/var/mail# dovecot -n
# 2.1.4: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4
auth_cache_size = 10 M
auth_verbose = yes
disable_plaintext_auth = no
instance_name = Mail Popper 1
listen = 172.20.20.222
login_greeting = Mail Popper 1 Ready
mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = shadow
}
protocols = pop3
service pop3-login {
service_count = 0
}
ssl = no
userdb {
driver = passwd
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
[Dovecot] POP3 Dovecot Auth CPU usage 75%+
Hello all,
I hope someone can help me, I have been testing out Dovecot to switch from
popa3d which I use at the moment. When I get several users connecting and
disconnection multiple times, the Dovecot process with command Auth uses
50-90% of the CPU for the period which they are connecting. I am wondering
if there is something that I may have misconfigured, or if there is
something that I can change so that this spike doesn't occur.
If anyone could shed some light on the issue, I would appreciate it,
Kevin
/var/mail# dovecot -n
# 2.1.4: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-33-generic-pae i686 Ubuntu 10.04.4 LTS ext4
auth_cache_size = 10 M
auth_verbose = yes
disable_plaintext_auth = no
instance_name = Mail Popper 1
listen = 172.20.20.222
login_greeting = Mail Popper 1 Ready
mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = shadow
}
protocols = pop3
service pop3-login {
service_count = 0
}
ssl = no
userdb {
driver = passwd
}
protocol pop3 {
pop3_uidl_format = %08Xu%08Xv
}
