The following issue has been RESOLVED.
======================================================================
http://www.dbmail.org/mantis/view.php?id=306
======================================================================
Reported By: variable
Assigned To: paul
======================================================================
Project: DBMail
Issue ID: 306
Category: General
Reproducibility: always
Severity: crash
Priority: normal
Status: resolved
Resolution: fixed
Fixed in Version: SVN Trunk
======================================================================
Date Submitted: 28-Feb-06 23:08 CET
Last Modified: 04-May-06 16:27 CEST
======================================================================
Summary: the processes spawn infitnetly until they crash the
machine
Description:
Both on 2.0.7 and 2.0.9 all 3 processes will spawn indefinatly until the
machine crashes, I found that if you set the TRACE_LEVEL=5 it will solve
the problem, and have been running my production server like this for
http://www.dbmail.org/mantis/view.php?id=6#2
months now. Also these open processes keep opening connections to pgsql
and quickly use all the connections for pgsql.
======================================================================
----------------------------------------------------------------------
ajake - 08-Mar-06 19:42
----------------------------------------------------------------------
I can confirm same behaviour with trace_level 3, debian sid, mysql and with
dbmail 2.0.9 (using imap&lmtp). Also with older 2.0.x versions, maybe with
all of them. We just went from 5 users to 75 and now it happens few times a
day (at busy hours).
During night we run dbmail-util and after that restart dbmail services.
Sometimes root imapd or lmtpd has already died. Also there usually are a
lot of glibc errors
during restart (example follows) and during the day.
Removing IP entries older than [2006-02-22 14:36:01]...
Ok. [18] IP entries were older than [2006-02-22 14:36:01].
Vacuuming and optimizing database...
Ok. Database cleaned up.
Maintenance done. No errors found.
Stopping dbmail servers: done.
dbmail-imapd: no process killed <-- just to be sure, sometimes there are
leftovers
dbmail-lmtpd: no process killed
Starting dbmail servers: dbmail-imapd .dbmail-lmtpd .done.
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** corrupted double-linked list: 0x0809a4d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** corrupted double-linked list: 0x080a1d50 ***
*** glibc detected *** corrupted double-linked list: 0x080a22c0 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** double free or corruption (fasttop): 0x0812f350
***
*** glibc detected *** double free or corruption (!prev): 0x081003f8 ***
*** glibc detected *** double free or corruption (fasttop): 0x080f9cb0
***
*** glibc detected *** double free or corruption (!prev): 0x080fdaa0 ***
*** glibc detected *** double free or corruption (fasttop): 0x08096cd0
***
*** glibc detected *** double free or corruption (fasttop): 0x080a4420
***
*** glibc detected *** double free or corruption (fasttop): 0x0810ad98
***
*** glibc detected *** double free or corruption (fasttop): 0x080a4330
***
*** glibc detected *** corrupted double-linked list: 0x0808a3a8 ***
*** glibc detected *** corrupted double-linked list: 0x080a4208 ***
another run:
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x08096c10 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** free(): invalid pointer: 0x080d8628 ***
*** glibc detected *** malloc(): memory corruption: 0x080d6b78 ***
*** glibc detected *** malloc(): memory corruption: 0x080c7370 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x08096b80 ***
*** glibc detected *** corrupted double-linked list: 0x080a01d0 ***
*** glibc detected *** malloc(): memory corruption: 0x0809a740 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
*** glibc detected *** corrupted double-linked list: 0x0808a3d8 ***
It seems that by upping mysql connection limit from 100 to 200 and
lowering default TIMEOUT=4000 to 1200 (MAXCHILDREN=100) solved our problem
with imapd spawning like crazy. Glibc errors are still there..
Other observations:
TRACE_LEVEL=whatever had no effect on dbmails stability
If mysql stops or runs out of max_connections, dbmail goes crazy.
"todays high" of imapd processes when MAXCHILDREN=100 is 105 which it
seems to survive.
can't run "dbmail-util -a -l 12h -y -v" during daytime because it takes a
long time (InnoDB table now 16G and growing) and while it is running
dbmail grinds to a halt or at least clients time out.
----------------------------------------------------------------------
mike - 04-Mar-06 08:29
----------------------------------------------------------------------
Late fall I upgraded a 7.x.x PGSQL to PostgreSQL 8.0.1 on a LAN file server
which does Admin mail for a dept. I upgraded Postfix and wanted to upgrade
DBMail too but *could not* get DBMail-2.0.7 to work properly. Connections
used the limit but dbmail.conf unchanged -- I upped the connections limit;
lasted an hour. Rolled back to DBMail-2.0.6: to this day. I now still have
the src folders so I checked the diff on the ChangeLog. Below is what
happened between 2.0.6 and 2.0.7 if anyone can spot something impacting
their setup. I can't see it.
2005-09-02 Paul J Stevens <paul -at- nfg.nl>
* ChangeLog, sql/mysql/create_tables_innoDB.mysql,
sql/postgresql/create_tables.pgsql:
added ON UPDATE CASCADE to create_table files
2005-08-31 Paul J Stevens <paul -at- nfg.nl>
* ChangeLog, configure, configure.in:
bump the version to 2.0.7
* debian/changelog:
updating sarge packages
* contrib/startup-scripts/debian/dbmail,
contrib/startup-scripts/debian/dbmail-imapd,
contrib/startup-scripts/debian/dbmail-lmtpd,
contrib/startup-scripts/debian/dbmail-pop3d,
contrib/startup-scripts/debian/dbmail.default:
update the included startup scripts
2005-08-24 Paul J Stevens <paul-at-nfg.nl>
* ChangeLog, server.c:
cleanup the signal handler in server.c
* pool.c:
add boundary check in manage_spare_children
2005-08-23 Ilja Booij <ilja -at- ic-s.nl>
* pop3.c: AUTH command no longer returns list of supported
authentication mechanisms when issued without arguments. closes
http://www.dbmail.org/mantis/view.php?id=237
2005-08-22 Paul J Stevens <paul-at-nfg.nl>
* debug.c:
avoid malloc in debug.c,trace by using a static buffer for debug
messages
* server.c:
reset alarm after database recovers
* pool.c:
non-blocking locking and some fixes in manage_spare_children
----------------------------------------------------------------------
paul - 04-May-06 16:27
----------------------------------------------------------------------
I've fixed this in svn-trunk.
Issue History
Date Modified Username Field Change
======================================================================
28-Feb-06 23:08 variable New Issue
01-Mar-06 17:13 ajake Note Added: 0001016
01-Mar-06 17:14 ajake Note Edited: 0001016
04-Mar-06 08:28 mike Note Added: 0001020
04-Mar-06 08:29 mike Note Edited: 0001020
08-Mar-06 19:42 ajake Note Edited: 0001016
04-May-06 16:27 paul Note Added: 0001152
04-May-06 16:27 paul Assigned To => paul
04-May-06 16:27 paul Status new => resolved
04-May-06 16:27 paul Resolution open => fixed
04-May-06 16:27 paul Projection none => minor fix
04-May-06 16:27 paul Fixed in Version => SVN Trunk
======================================================================
