Re: Connection related SIG 11 crash in 3.23
hey,
i've been observing a similar kind of a problem
with mysql 3.23.32 on a Linux 2.4.2 kernel. The
backtrace that i'm getting is different, but even that
fails a sanity check. When i resolved the incomplete
backtrace, it showed me thread creation functions and
something of that sort. The mysql server that i have
crashes when the machine approaches 100% CPU usage.
Someone told me that a 2.2 kernel downgrade will solve
the problem; but i dont want to do that. I've tried
on different machines and the problem remains.
Tx.
Vinod.
--- Voytek Lapinski [EMAIL PROTECTED] wrote:
Ok... I've posted briefly regarding this, but heres
a complete bug report.
Yeah I know its kinda long, but I thought it better
to include more rather
than less.
We are seeing a problem which results in mysqld
dying with a SIGSEGV. The
problem has been confirmed on both these machines
Linux lindt 2.2.18 #2 SMP Mon Feb 26 15:23:32 EST
2001 i686 unknown
Dell PowerEdge 4300 - Dual Pentium 3-500 - 1Gb RAM,
1.6Gb swap
Linux lisa 2.4.1 #7 SMP Fri Feb 16 15:21:45 EST 2001
i686 unknown
IBM Rackmount Server 325 - Dual Pentium Pro-200 -
512Mb RAM, 1Gb Swap
And using mysqld versions 3.23.32,3.23.33 and
3.23.35. To confirm that the
problem isn't related to libraries, compilers or
configuration specific to our
build environment, I have also tested the standard
linux-i386 binary distro of
3.23.35, and the problem also exists there.
The error message takes the following form:
mysqld got signal 11;
The manual section 'Debugging a MySQL server' tells
you how to use a
stack trace and/or the core file to produce a
readable backtrace that may
help in finding out why mysqld died.
Attempting backtrace. You can use the following
information to find out
where mysqld died. If you see no messages after
this, something went
terribly wrong...
Cannot determine thread, ebp=0xb108, backtrace
may not be correct.
Stack range sanity check OK, backtrace follows:
0x813fefa
0x813f6dd
0x8081690
0x8081b14
0x808127e
0x8145e4b
0x8048111
New value of ebp failed sanity check, terminating
backtrace!
Naturally with different values for the stack
backtrace depending on version
(the addresses look similarly spaced though, so it's
probably the same thing).
The problem appears to be related to connections
which occur at roughly the
same time. I have written a script which reliably
causes it to occur and have
included it below. It operates by forking off lots
of processes all of which
connect and disconnect to the database repeatedly
until it goes down. In most
cases, the database will crash within seconds,
although rarely it seems
to take significantly longer (several minutes). The
problem I am referring to
is associated with local UNIX domain socket
connections, but when I briefly
tested connecting over TCP/IP it (or something else
similar) still occurs in
that it eventually stops accepting connections on
that port (although the
database is still running and generates no errors).
This may be something
else entirely though, and I have not investigated
that at all.
While this case seems somewhat artificial, we were
seeing the same crashes
under normal operation of our application which
seemed to be associated with
the activation of a process that frequently will
fork of 6 new processes, all
of which attempt to connect into the database
immediately after starting.
The test script follows:
--START SCRIPT--
#!/usr/bin/perl -w
use DBI;
for(my $i=0;$i100;$i++) {
my $fork_ret=fork;
if (not defined($fork_ret)) {
die("$! fork failed\n");
} elsif ($fork_ret == 0) {
while(1) {
my$
dbh=DBI-connect('DBI:mysql:database=test','test','test',{PrintError=1,RaiseError=0});
exit unless (defined($dbh));
$dbh-disconnect();
}
}
}
my $start=time();
my $wait_ret;
do {
$wait_ret=wait;
} until $wait_ret == -1;
print ((time()-$start)."s\n");
--END OF SCRIPT---
The [mysql] section of our my.cnf is:
--START CONFIG--
[mysqld]
port=
socket = /tmp/mysql.sock
skip-locking
set-variable= key_buffer=32M
set-variable= max_allowed_packet=1M
set-variable= thread_stack=128K
set-variable= tmp_table_size=8M
set-variable= max_connections=200
--END CONFIG--
But I have tried more conservative values for
key_buffer (down to 4M),
thread_stack(64K) and tmp_table_size(2M). The
problem also seems to occur
with values of max_connections anywhere from 20 to
1 (Yeah I know 1 is
silly with LinuxThreads, but I thought I'd do it to
see what happens).
When invoking mysql, many options have been used,
but the most conservative
settings we tried were
/usr/local/mysql-3.23.35-bindist/bin/mysqld
--datadir=/data --user=root
The problem can be made to take longer to happen by
a few things, but of
course this is rather
Re: Mysql restarts at 100% CPU
hi, i've not reconfigured linuxthreads. /proc/sys/kernel/threads-max has 57344. At the time when the machine hits 100% CPU, apache processes are taking most of the CPU time, and i have around 250 processes on the entire system, with around 10-15 running processes. mysqladmin status shows me around 15 mysql threads running. Agreed that something like this should not happen, the CPU touching 100% is not an acceptable situation, but what i'm worried abt is mysql crashing. If the OS goes for a toss, i know that it is a problem with the load, but mysql is getting something like a 100-150 queries at that time. Tx. Vinod --- "Gerald L. Clark" [EMAIL PROTECTED] wrote: You have mysqld set for 2000 connections. How many threads do you have Linux configured for, and how many are open when Mysqld crashs? vinod p wrote: hi, thanks for the interest. The error log just prints an incomplete backtrace that is exactly the same after every crash. I ran the incomplete backtrace through resolve, and it showed me functions such as thread start... etc... This is the backtrace that i've been getting for all the crashes - --- Cannot determine thread, ebp=0xb24c, backtrace may not be correct stack range sanity check, ok, backtrace follows 0x40077552 0x40076f0f 0x80bdb6a 0x80bdfdf 0x80bd75d 0x400f99cb 0x80884b1 New value of ebp failed sanity check terminating backtrace We have been doing testing of our software and in all the runs, with all the mysql servers, this problem has been experienced. As soon as the server reaches 100% CPU, the mysql server restarts. Without a single exception till now. The servers are RedHat Linux 6.2 with kernel 2.4.2, PHP 4.0.4pl1, apache 1.3.14 and mysql 3.23.32. The my.cnf file i'm using is this - [mysqld] port= 3306 socket = /tmp/mysql.sock skip-locking set-variable= key_buffer=640M set-variable= max_allowed_packet=10M set-variable= table_cache=640 set-variable= sort_buffer=6M set-variable= record_buffer=6M set-variable= thread_cache=16 set-variable= thread_concurrency=16 set-variable= myisam_sort_buffer_size=64M log-bin server-id = 1 set-variable= max_connections=2000 set-variable= max_connect_errors=1 set-variable= back_log=2900 set-variable= connect_timeout=15 set-variable= wait_timeout=57600 set-variable= interactive_timeout=57600 Tx. Vinod. --- "Jeremy D. Zawodny" [EMAIL PROTECTED] wrote: On Mon, Mar 19, 2001 at 12:34:30AM +, vinod p wrote: my mysql server restarts automatically when the machine touches 100% CPU usage. any ideas why? What does the error log say? Anything? -- Jeremy D. Zawodny, [EMAIL PROTECTED] Technical Yahoo - Yahoo Finance Desk: (408) 328-7878Fax: (408) 530-5454 Cell: (408) 439-9951 Do You Yahoo!? Get your free @yahoo.co.in address at http://mail.yahoo.co.in - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php Do You Yahoo!? For regular News updates go to http://in.news.yahoo.com - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: am I alone? (scale)
hey.. thanks for the info... This is the my.cnf file that i'm using - [mysqld] skip-locking set-variable= key_buffer=640M set-variable= max_allowed_packet=10M set-variable= table_cache=640 set-variable= sort_buffer=6M set-variable= record_buffer=6M set-variable= thread_cache=16 set-variable= thread_concurrency=16 set-variable= myisam_sort_buffer_size=64M log-bin server-id = 1 set-variable= max_connections=2000 set-variable= max_connect_errors=1 set-variable= back_log=2900 set-variable= connect_timeout=15 set-variable= wait_timeout=57600 set-variable= interactive_timeout=57600 Now the problem that i've managed to track down is - When the CPU hits 100%, mysql crashes. This behaviour has been consistent across many many runs. I would definitely not like the database server to crash. A loss in performance is expected but a crash is most definitely not. As u can see, the limits are set pretty high on the system. Memory usage is not even 150MB. What's the reason? and the partial backtrace that i'm getting is the same after every crash. Any ideas? Tx. Vinod. Do You Yahoo!? Get your free @yahoo.co.in address at http://mail.yahoo.co.in - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Mysql restarts at 100% CPU
hi, my mysql server restarts automatically when the machine touches 100% CPU usage. any ideas why? tx. vinod. Do You Yahoo!? Get your free @yahoo.co.in address at http://mail.yahoo.co.in - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Re: Mysql restarts at 100% CPU
hi, thanks for the interest. The error log just prints an incomplete backtrace that is exactly the same after every crash. I ran the incomplete backtrace through resolve, and it showed me functions such as thread start... etc... This is the backtrace that i've been getting for all the crashes - --- Cannot determine thread, ebp=0xb24c, backtrace may not be correct stack range sanity check, ok, backtrace follows 0x40077552 0x40076f0f 0x80bdb6a 0x80bdfdf 0x80bd75d 0x400f99cb 0x80884b1 New value of ebp failed sanity check terminating backtrace We have been doing testing of our software and in all the runs, with all the mysql servers, this problem has been experienced. As soon as the server reaches 100% CPU, the mysql server restarts. Without a single exception till now. The servers are RedHat Linux 6.2 with kernel 2.4.2, PHP 4.0.4pl1, apache 1.3.14 and mysql 3.23.32. The my.cnf file i'm using is this - [mysqld] port= 3306 socket = /tmp/mysql.sock skip-locking set-variable= key_buffer=640M set-variable= max_allowed_packet=10M set-variable= table_cache=640 set-variable= sort_buffer=6M set-variable= record_buffer=6M set-variable= thread_cache=16 set-variable= thread_concurrency=16 set-variable= myisam_sort_buffer_size=64M log-bin server-id = 1 set-variable= max_connections=2000 set-variable= max_connect_errors=1 set-variable= back_log=2900 set-variable= connect_timeout=15 set-variable= wait_timeout=57600 set-variable= interactive_timeout=57600 Tx. Vinod. --- "Jeremy D. Zawodny" [EMAIL PROTECTED] wrote: On Mon, Mar 19, 2001 at 12:34:30AM +0000, vinod p wrote: my mysql server restarts automatically when the machine touches 100% CPU usage. any ideas why? What does the error log say? Anything? -- Jeremy D. Zawodny, [EMAIL PROTECTED] Technical Yahoo - Yahoo Finance Desk: (408) 328-7878Fax: (408) 530-5454 Cell: (408) 439-9951 Do You Yahoo!? Get your free @yahoo.co.in address at http://mail.yahoo.co.in - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
Heavy Load
hey, i'm running serveral mysql servers on powerful machines - dual processor, 2 GB memory, etc... Most of the databases store temporary data only. I need the maximum performance from these servers and have a lot of connections to the database. My problem is this - whenever the number of queries sent to the database increases, or the CPU usage increases under heavy load, mysql crashes. This has been a consistent behaviour. The system is under testing phase and the servers just give up under a reasonably heavy load. This is the my.cnf file - [mysqld] skip-locking set-variable= key_buffer=640M set-variable= max_allowed_packet=10M set-variable= table_cache=640 set-variable= sort_buffer=6M set-variable= record_buffer=6M set-variable= thread_cache=16 set-variable= thread_concurrency=16 set-variable= myisam_sort_buffer_size=64M log-bin server-id = 1 set-variable= max_connections=2000 set-variable= max_connect_errors=1 set-variable= back_log=2900 set-variable= connect_timeout=15 set-variable= wait_timeout=57600 set-variable= interactive_timeout=57600 [mysqldump] quick set-variable= max_allowed_packet=16M [mysql] no-auto-rehash [isamchk] set-variable= key_buffer=256M set-variable= sort_buffer=256M set-variable= read_buffer=2M set-variable= write_buffer=2M [myisamchk] set-variable= key_buffer=256M set-variable= sort_buffer=256M set-variable= read_buffer=2M set-variable= write_buffer=2M According to me, the database should become slow under heavy load - it should not just crash. The backtrace is also incomplete. And it is consistent for every crash. Any ideas why this could be happening? I'm using PHP to access the database on RH 6.2 with kernel 2.4.2. I'm using latest versions of everything. Thanks, Vinod Do You Yahoo!? Get your free @yahoo.co.in address at http://mail.yahoo.co.in - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
