mysqld post-mortem

2005-05-12 Thread Mark C. Stafford
Hello everyone. I've got a MySQL server that behaves
wonderfully...most of the time. It has crashed three times this week
and I'm at a loss for why.

Below I've included all the following:
  error.log
  os info
  memory check
  results from resolve_stack_dump
  my.cnf

Does the information point toward a problem you can see? Thanks in advance.

 error.log
 050510 15:57:48 InnoDB: Started
 050510 15:57:49 Found invalid password for user: '5014'@'10.0.%'; Ignoring user
 /usr/libexec/mysqld: ready for connections.
 Version: '4.0.23a-log' socket: '/var/run/mysql/mysql.sock' port: 3306
Source distribution

 050511 14:52:19 Found invalid password for user: '5014'@'10.0.%'; Ignoring user
 mysqld got signal 11;
 This could be because you hit a bug. It is also possible that this
binary or one of the libraries it was linked against is corrupt,
improperly built, or misconfigured. This error can also be caused by
malfunctioning hardware. We will try our best to scrape up some info
that will hopefully help diagnose the problem, but since we have
already crashed, something is definitely wrong and this may fail.

 key_buffer_size=1073741824
 read_buffer_size=1044480
 max_used_connections=125
 max_connections=150
 threads_connected=65
 It is possible that mysqld could use up to
 key_buffer_size + (read_buffer_size +
sort_buffer_size)*max_connections = 3659174 K
 bytes of memory
 Hope that's ok; if not, decrease some variables in the equation.

 thd=(nil)
 Attempting backtrace. You can use the following information to find
out where mysqld died. If you see no messages after this, something
went terribly wrong...
 Cannot determine thread, fp=0xb068, backtrace may not be correct.
 Stack range sanity check OK, backtrace follows:
 0x81048f1
 0xb7f83c85
 0xb7e4633e
 0xb7e437a3
 0x82d0163
 0x82ce417
 0x82cfee1
 0x80f9080
 0x81065ea
 0x8105964
 0xb7df7469
 0x80b62b1
 New value of fp=(nil) failed sanity check, terminating stack trace!
 Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html
and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please
do resolve it
 The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
 050512 9:43:47 InnoDB: Started
 050512 9:43:47 Found invalid password for user: '5014'@'10.0.%'; Ignoring user
 /usr/libexec/mysqld: ready for connections.
 Version: '4.0.23a-log' socket: '/var/run/mysql/mysql.sock' port: 3306
Source distribution

 slackware linux info
 uname - a
 Linux b5 2.6.11 #1 SMP Sat Apr 16 23:35:33 MDT 2005 i686 unknown
unknown GNU/Linux

 memory check:
 googled 3659174 KB in MB == 3 659 174 kilobytes = 3 573.41211 megabytes
 this machine has 4GB RAM
 4GB in MB == 4 gigabytes = 4 096 megabytes
 4 096 - 3 573.41211 = 522.58789
 1/2GB RAM available for non-mysql functionality is more than enough, right?

 i ran resolve_stack_dump on the backtrace from
error.log per http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html
resolve_stack_dump -s ./mysqld.sym -n ./mysqld.stack and got the
results below, but the information provided doesn't mean anything to
me now what?
 0x81048f1 handle_segfault + 641
 0xb7f83c85 _end + -1346321187
 0xb7e4633e _end + -1347621994
 0xb7e437a3 _end + -1347633157
 0x82d0163 my_malloc + 35
 0x82ce417 init_io_cache + 295
 0x82cfee1 open_cached_file + 145
 0x80f9080 _ZN3THDC1Ev + 928
 0x81065ea handle_connections_sockets + 666
 0x8105964 main + 2180
 0xb7df7469 _end + -1347945279
 0x80b62b1 _start + 33

 my.cnf
[mysqld]
# BASICS
  datadir=/var/lib/mysql
  pid_file=/var/run/mysql/mysql.pid
  port=3306
  socket=/var/run/mysql/mysql.sock
  tmpdir=/tmp/
  tmp_table_size=64M
  max_connections=150
  # IMHO, long_query_time=6 is too big...but we'll lower this after
nailing the worst offenders
  long_query_time=6
  log_long_format
  # /var/log/mysqld doesn't exist by default, so i created it and ran
chown mysql:mysql on it
  log_slow_queries=/var/log/mysqld/slow.log
  log_error=/var/log/mysqld/error.log
  # discourages brute force attacks, unblock blocked hosts with FLUSH HOSTS
  max_connect_errors=5
#
# REPLICATION
  server_id=1
  read_only=OFF
  # security hazard...a client with the SUPER privilege can disable
binary logging of its own statements by using a SET SQL_LOG_BIN=0
  log_bin=b5
  # *ignores errors re: revokation of non-existent grants Error: 1147
SQLSTATE: 42000
  slave-skip-errors=1147
  max_allowed_packet=16M
  character_set=latin1
#
# PERFORMANCE TUNING (low_priority_updates, max_join_size and
max_seeks_for_key feel a bit bleeding-edge and should be removed if
the results suck)
  # low_priority_updates makes INSERT, UPDATE AND lower priority than
SELECT and LOCK TABLE
  low_priority_updates=ON
  max_join_size=32M
  max_seeks_for_key=100
  # caches
table_cache=768
max_binlog_cache_size=2GB
binlog_cache_size=999MB
thread_cache_size=50
# 

Re: mysqld post-mortem

2005-05-12 Thread Gleb Paharenko
Hello.



 Linux b5 2.6.11 #1 SMP Sat Apr 16 23:35:33 MDT 2005 i686 unknown



Is it a 32-bit arch?



 sort_buffer_size)*max_connections =3D 3659174 K

 bytes of memory



Usually the process size on 32-bit machines can't be more than 2G.





Mark C. Stafford [EMAIL PROTECTED] wrote:

 Hello everyone. I've got a MySQL server that behaves

 wonderfully...most of the time. It has crashed three times this week

 and I'm at a loss for why.

 

 Below I've included all the following:

  error.log

  os info

  memory check

  results from resolve_stack_dump

  my.cnf

 

 Does the information point toward a problem you can see? Thanks in advance.

 

 error.log

 050510 15:57:48 InnoDB: Started

 050510 15:57:49 Found invalid password for user: '5014'@'10.0.%'; Ignoring=

 user

 /usr/libexec/mysqld: ready for connections.

 Version: '4.0.23a-log' socket: '/var/run/mysql/mysql.sock' port: 3306

 Source distribution

 

 050511 14:52:19 Found invalid password for user: '5014'@'10.0.%'; Ignoring=

 user

 mysqld got signal 11;

 This could be because you hit a bug. It is also possible that this

 binary or one of the libraries it was linked against is corrupt,

 improperly built, or misconfigured. This error can also be caused by

 malfunctioning hardware. We will try our best to scrape up some info

 that will hopefully help diagnose the problem, but since we have

 already crashed, something is definitely wrong and this may fail.

 

 key_buffer_size=3D1073741824

 read_buffer_size=3D1044480

 max_used_connections=3D125

 max_connections=3D150

 threads_connected=3D65

 It is possible that mysqld could use up to

 key_buffer_size + (read_buffer_size +

 sort_buffer_size)*max_connections =3D 3659174 K

 bytes of memory

 Hope that's ok; if not, decrease some variables in the equation.

 

 thd=3D(nil)

 Attempting backtrace. You can use the following information to find

 out where mysqld died. If you see no messages after this, something

 went terribly wrong...

 Cannot determine thread, fp=3D0xb068, backtrace may not be correct.

 Stack range sanity check OK, backtrace follows:

 0x81048f1

 0xb7f83c85

 0xb7e4633e

 0xb7e437a3

 0x82d0163

 0x82ce417

 0x82cfee1

 0x80f9080

 0x81065ea

 0x8105964

 0xb7df7469

 0x80b62b1

 New value of fp=3D(nil) failed sanity check, terminating stack trace!

 Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html

 and follow instructions on how to resolve the stack trace. Resolved

 stack trace is much more helpful in diagnosing the problem, so please

 do resolve it

 The manual page at http://www.mysql.com/doc/en/Crashing.html contains

 information that should help you find out what is causing the crash.

 050512 9:43:47 InnoDB: Started

 050512 9:43:47 Found invalid password for user: '5014'@'10.0.%'; Ignoring =

 user

 /usr/libexec/mysqld: ready for connections.

 Version: '4.0.23a-log' socket: '/var/run/mysql/mysql.sock' port: 3306

 Source distribution

 

 slackware linux info

 uname - a

 Linux b5 2.6.11 #1 SMP Sat Apr 16 23:35:33 MDT 2005 i686 unknown

 unknown GNU/Linux

 

 memory check:

 googled 3659174 KB in MB =3D=3D 3 659 174 kilobytes =3D 3 573.41211 meg=

 abytes

 this machine has 4GB RAM

 4GB in MB =3D=3D 4 gigabytes =3D 4 096 megabytes

 4 096 - 3 573.41211 =3D 522.58789

 1/2GB RAM available for non-mysql functionality is more than enough, right=

 ?

 

 i ran resolve_stack_dump on the backtrace from

 error.log per http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html

 resolve_stack_dump -s ./mysqld.sym -n ./mysqld.stack and got the

 results below, but the information provided doesn't mean anything to

 me now what?

 0x81048f1 handle_segfault + 641

 0xb7f83c85 _end + -1346321187

 0xb7e4633e _end + -1347621994

 0xb7e437a3 _end + -1347633157

 0x82d0163 my_malloc + 35

 0x82ce417 init_io_cache + 295

 0x82cfee1 open_cached_file + 145

 0x80f9080 _ZN3THDC1Ev + 928

 0x81065ea handle_connections_sockets + 666

 0x8105964 main + 2180

 0xb7df7469 _end + -1347945279

 0x80b62b1 _start + 33

 

 my.cnf

 [mysqld]

 # BASICS

  datadir=3D/var/lib/mysql

  pid_file=3D/var/run/mysql/mysql.pid

  port=3D3306

  socket=3D/var/run/mysql/mysql.sock

  tmpdir=3D/tmp/

  tmp_table_size=3D64M

  max_connections=3D150

  # IMHO, long_query_time=3D6 is too big...but we'll lower this after

 nailing the worst offenders

  long_query_time=3D6

  log_long_format

  # /var/log/mysqld doesn't exist by default, so i created it and ran

 chown mysql:mysql on it

  log_slow_queries=3D/var/log/mysqld/slow.log

  log_error=3D/var/log/mysqld/error.log

  # discourages brute force attacks, unblock blocked hosts with FLUSH HOSTS

  max_connect_errors=3D5

 #

 # REPLICATION

  server_id=3D1

  read_only=3DOFF

  # security hazard...a client with the SUPER privilege can disable

 binary logging of its own statements by using a SET SQL_LOG_BIN=3D0

  log_bin=3Db5

  # *ignores errors re: revokation of non-existent grants Error: