A NOTE has been added to this issue.
======================================================================
http://www.dbmail.org/mantis/view.php?id=921
======================================================================
Reported By: mattix
Assigned To:
======================================================================
Project: DBMail
Issue ID: 921
Category: IMAP daemon
Reproducibility: random
Severity: crash
Priority: normal
Status: feedback
target:
======================================================================
Date Submitted: 28-Aug-11 06:10 CEST
Last Modified: 01-Sep-11 10:57 CEST
======================================================================
Summary: DBmail-imapd hangup with remote Postgresql Database
Description:
strace output
[ffffe424] epoll_ctl(9, EPOLL_CTL_MOD, 131, {EPOLLOUT, {u32=131,
u64=131}}) = 0
[ffffe424] epoll_ctl(9, EPOLL_CTL_DEL, 131, {EPOLLOUT, {u32=131,
u64=131}}) = 0
[ffffe424] brk(0xa061000) = 0xa061000
[b7555753] futex(0xb120046c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb1200468,
{FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
[ffffe424] futex(0x80a8f10, FUTEX_WAKE_PRIVATE, 1) = 1
[ffffe424] epoll_wait(9, {{EPOLLIN, {u32=12, u64=12}}}, 32, 2100000) = 1
[b77c4e16] clock_gettime(CLOCK_MONOTONIC, {247115, 478761306}) = 0
[ffffe424] epoll_ctl(9, EPOLL_CTL_DEL, 12, {EPOLLIN, {u32=12, u64=12}}) =
0
[ffffe424] shutdown(131, 2 /* send and receive */) = -1 ENOTCONN
(Transport endpoint is not connected)
[ffffe424] close(131) = 0
[ffffe424] close(131) = -1 EBADF (Bad file descriptor)
[ffffe424] brk(0x9fdf000) = 0x9fdf000
[ffffe424] read(12, "Q", 128) = 1
[ffffe424] read(12, 0xbfec5270, 128) = -1 EAGAIN (Resource temporarily
unavailable)
[ffffe424] epoll_ctl(9, EPOLL_CTL_ADD, 12, {EPOLLIN, {u32=12, u64=12}}) =
0
[ffffe424] epoll_wait
I believe, after i watched in my tcpdump - out, the main Problem is that
under some Credentials like Spanning-tree Notification, the Communications
is disturbed by this issue, but hangup for perhaps under 1 s, after this
event dbmail-imapd hangup
======================================================================
----------------------------------------------------------------------
(0003250) paul (administrator) - 28-Aug-11 14:20
http://www.dbmail.org/mantis/view.php?id=921#c3250
----------------------------------------------------------------------
Please provide logs at level 511. The strace lines simply show the
controlled shutdown of a disconnected client connection afaict.
Also, I haven't a clue as to what you mean by:
"""main Problem is that under some Credentials like Spanning-tree
Notification, the Communications is disturbed by this issue"""
I assume you mean 'spanning tree topology change notification' but what do
'credentials' have to do with it? I'm not an ethernet expert. But: changes
in the ethernet topology leading to disruptions in the tcp/ip layer at the
kernel level are completely out of scope for an application like DBMail.
Perhaps a work-around is required to make DBMail more robust in adverse
networking circumstances. But the logs really are required for that.
----------------------------------------------------------------------
(0003251) mattix (reporter) - 28-Aug-11 15:33
http://www.dbmail.org/mantis/view.php?id=921#c3251
----------------------------------------------------------------------
the Problem with spanning tree was an buggy bridge configuration that
shutdown switch ports for less then 3 seconds, on this connection listen
dbmail-imapd to its PostgreSQL Server ,
this bridge is replaced, but on heavy network load on this interface
dbmail-imapd sporadic crashed on this conditions
how long can dbmail-imapd survival without Database
Nagios is also check every Minute survival of dbmail perhaps is this also
a problem ?
here is the first Cut
Aug 28 13:12:02 imap-002 dbmail-imapd[21651]: [0x9c27fb8] Error:[server]
_sock_cb(+481): Too many open files
perhaps a Connection Limit ?
i change Syslog Level to 511 Everything more coming soon
thank You for fast Answer
----------------------------------------------------------------------
(0003252) mattix (reporter) - 28-Aug-11 15:49
http://www.dbmail.org/mantis/view.php?id=921#c3252
----------------------------------------------------------------------
Ok dbmail handled it excelent if the database is not present,
ok the 2nd point is the better path to find out
perhaps my dbmail-imapd installation run to a network limit,
but why ?
----------------------------------------------------------------------
(0003253) paul (administrator) - 28-Aug-11 19:13
http://www.dbmail.org/mantis/view.php?id=921#c3253
----------------------------------------------------------------------
'Too many open files' is a kernel limit.
below link describes how to diagnose and fix this.
http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/
----------------------------------------------------------------------
(0003254) mattix (reporter) - 28-Aug-11 19:31
http://www.dbmail.org/mantis/view.php?id=921#c3254
----------------------------------------------------------------------
Thanks a lot, i found another item, dbmail exit with Following Messages in
dmesg dbmail-imapd[24588]: segfault at 99c110f5 ip 99c110f5 sp b52537c8
error 4
i will try it with new Kernelvalues
----------------------------------------------------------------------
(0003256) paul (administrator) - 28-Aug-11 22:10
http://www.dbmail.org/mantis/view.php?id=921#c3256
----------------------------------------------------------------------
Ok, a segfault is _bad_! But without reproduce-ability it's impossible to
fix :-(
----------------------------------------------------------------------
(0003258) mattix (reporter) - 30-Aug-11 10:00
http://www.dbmail.org/mantis/view.php?id=921#c3258
----------------------------------------------------------------------
ok the Segfault i can't reproduce sorry,
but the Issue with to many open files still exist i had increased the file
limit to unlimited but it help conditionally:
lsof -u dbmail show me this:
dbmail-im 7410 dbmail 208u sock 0,6 0t0 349386 can't
identify protocol
dbmail-im 7410 dbmail 209u sock 0,6 0t0 349467 can't
identify protocol
dbmail-im 7410 dbmail 210u sock 0,6 0t0 349248 can't
identify protocol
dbmail-im 7410 dbmail 211u sock 0,6 0t0 349828 can't
identify protocol
dbmail-im 7410 dbmail 212u sock 0,6 0t0 349912 can't
identify protocol
dbmail-im 7410 dbmail 213u sock 0,6 0t0 350131 can't identi
protocol
.......
I should tell you a little bit about my Installation to reproduce it:
my dbmail installation is clustered behind an haproxy (3 Dbmail-imapd
Servers on dedicate Servers), which checking every 1 or 2 Seconds
reachability of Imap (this a normal procedure of Loadbalancing mechanism),
it seems to be DBmail-IMAP 3 do not close this Connections. DBmail 2 did
not have this Problem in this same Installation. Dbmail-timsieved is also
behind this haproxy and has not this problem.
in detail haproxy open from the same source address many connection,
it seem to be, that haproxy trigger dbmail-imapd to running in file
descriptor leak
----------------------------------------------------------------------
(0003262) mattix (reporter) - 01-Sep-11 10:57
http://www.dbmail.org/mantis/view.php?id=921#c3262
----------------------------------------------------------------------
Here some log-ouput what the Problem could be:
Sep 1 10:38:32 myhost dbmail/timsieved[24080]: [0x804dfb8]
Notice:[clientbase] client_init(+185): incoming connection from
[10.xx.xx.xx:56811]
Sep 1 10:38:32 myhost dbmail/timsieved[24080]: [0x804dfb8]
Notice:[clientsession] client_session_read(+133): reached EOF
and
Sep 1 10:38:31 myhost dbmail/imap4d[24090]: [0x8065fb8]
Notice:[clientbase] client_init(+167): incoming connection on
[10.xx.xx.xx:143]....
but never Found and EOF i think it is Missing or MAX_FAULTY_RESPONSES is
not defined
in dbmail-2.2.17/imap4.c you send it here:
125 if (nfaultyresponses >= MAX_FAULTY_RESPONSES) {
126 /* we have had just about it with this user
*/
127 sleep(2); /* avoid DOS attacks */
128 if (dbmail_imap_session_printf(session, "* BYE
[TRY RFC]\r\n") < 0) {
129 dbmail_imap_session_delete(session);
130 return EOF;
131 }
132 done = 1;
133 break;
in dbmail-3.0.0 didn't find a section like this.
perhaps it is helpful
Issue History
Date Modified Username Field Change
======================================================================
28-Aug-11 06:10 mattix New Issue
28-Aug-11 14:20 paul Note Added: 0003250
28-Aug-11 14:20 paul Status new => feedback
28-Aug-11 14:20 paul Resolution open => unable to
reproduce
28-Aug-11 15:33 mattix Note Added: 0003251
28-Aug-11 15:49 mattix Note Added: 0003252
28-Aug-11 19:13 paul Note Added: 0003253
28-Aug-11 19:31 mattix Note Added: 0003254
28-Aug-11 22:10 paul Note Added: 0003256
30-Aug-11 10:00 mattix Note Added: 0003258
01-Sep-11 10:57 mattix Note Added: 0003262
======================================================================
_______________________________________________
Dbmail-dev mailing list
Dbmail-dev@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail-dev
