Master database server continues to crash

2007-09-08 Thread Matt Juszczak

Hi all,

We've been having our master database server continually crash about once or 
twice per week.  The most recent output in the error log is contained at the 
bottom of this message.


The funny part is, that not only is mysql crashing often, but we are also 
seeing some very weird errors in some of our queries... such as:

[nativecode=1062 ** Duplicate entry 'aa59442538' for key
That error comes from a query where the query actually has an ON DUPLICATE KEY 
UPDATE clause.


Also, we're getting errors with crashed tables:
Table '' is marked as crashed and should be repaired

A check table on that table and other tables usually results in invalid row 
count (these are MyISAM tables).  There is no pattern to the errors other than 
that they tend to occur on high traffic tables.


We are running mysql version:
mysql  Ver 14.12 Distrib 5.0.27

We are also running these mysql version elsewhere with no problems.  I am 
beginning to think more and more that this issue is related to the hardware of 
the master server, not necessarilly the mysql version itself.


Although we do need to upgrade, does anyone know of any existing bugs in 5.0.27 
that may be causing our issues?  Or is this most likely a hardware issue?


Thanks,

Matt


One of the outputs in the Error log (there are many more):

Version: '5.0.27-standard-log'  socket: '/var/lib/mysql/mysql.sock'  port: 3306 
MySQL Community Edition - Standard (GPL)

mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1073741824
read_buffer_size=131072
max_used_connections=19
max_connections=300
threads_connected=15
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 
1701373 K

bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x2ac0865b40
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x450c9f78, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x5f7374656c796568
Stack trace seems successful - bottom reached
Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and follow 
instructions on how to resolve the stack trac

e. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd-query at 0x2ac05362b0  is invalid pointer
thd-thread_id=15401
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
mysqld: my_new.cc:51: int __cxa_pure_virtual(): Assertion `Pure virtual method 
called. == Aborted' failed.

Fatal signal 6 while backtracing

Number of processes running now: 0
070907 07:21:51  mysqld restarted
070907  7:21:51  InnoDB: Database was not shut down normally!


--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]



Re: Master database server continues to crash

2007-09-08 Thread Mathieu Bruneau
Are you running on a 32 bits architecture ? I have seen case where 1.7G 
is way too much for the total space mysql is allowed to use, especially 
if you have innodb buffer set to bigger than default value(8MB). Also 
you should check that the user is really allowed that size (ulimit).

The duplicate for key is most certainly due to the crashing of mysql in 
the middle of the update of the key...

Could it be different query on that server that makes it crashed ?

Sorry don't know what to suggest really, it must be a real pain :(

Regards
Math
aka ROunofF

On Sat, Sep 08, 2007 at 06:30:08PM -0400, Matt Juszczak wrote:
 Hi all,
 
 We've been having our master database server continually crash about once 
 or twice per week.  The most recent output in the error log is contained at 
 the bottom of this message.
 
 The funny part is, that not only is mysql crashing often, but we are also 
 seeing some very weird errors in some of our queries... such as:
 [nativecode=1062 ** Duplicate entry 'aa59442538' for key
 That error comes from a query where the query actually has an ON DUPLICATE 
 KEY UPDATE clause.
 
 Also, we're getting errors with crashed tables:
 Table '' is marked as crashed and should be repaired
 
 A check table on that table and other tables usually results in invalid row 
 count (these are MyISAM tables).  There is no pattern to the errors other 
 than that they tend to occur on high traffic tables.
 
 We are running mysql version:
 mysql  Ver 14.12 Distrib 5.0.27
 
 We are also running these mysql version elsewhere with no problems.  I am 
 beginning to think more and more that this issue is related to the hardware 
 of the master server, not necessarilly the mysql version itself.
 
 Although we do need to upgrade, does anyone know of any existing bugs in 
 5.0.27 that may be causing our issues?  Or is this most likely a hardware 
 issue?
 
 Thanks,
 
 Matt
 
 
 One of the outputs in the Error log (there are many more):
 
 Version: '5.0.27-standard-log'  socket: '/var/lib/mysql/mysql.sock'  port: 
 3306 MySQL Community Edition - Standard (GPL)
 mysqld got signal 11;
 This could be because you hit a bug. It is also possible that this binary
 or one of the libraries it was linked against is corrupt, improperly built,
 or misconfigured. This error can also be caused by malfunctioning hardware.
 We will try our best to scrape up some info that will hopefully help 
 diagnose
 the problem, but since we have already crashed, something is definitely 
 wrong
 and this may fail.
 
 key_buffer_size=1073741824
 read_buffer_size=131072
 max_used_connections=19
 max_connections=300
 threads_connected=15
 It is possible that mysqld could use up to
 key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 
 1701373 K
 bytes of memory
 Hope that's ok; if not, decrease some variables in the equation.
 
 thd=0x2ac0865b40
 Attempting backtrace. You can use the following information to find out
 where mysqld died. If you see no messages after this, something went
 terribly wrong...
 Cannot determine thread, fp=0x450c9f78, backtrace may not be correct.
 Stack range sanity check OK, backtrace follows:
 0x5f7374656c796568
 Stack trace seems successful - bottom reached
 Please read http://dev.mysql.com/doc/mysql/en/Using_stack_trace.html and 
 follow instructions on how to resolve the stack trac
 e. Resolved
 stack trace is much more helpful in diagnosing the problem, so please do
 resolve it
 Trying to get some variables.
 Some pointers may be invalid and cause the dump to abort...
 thd-query at 0x2ac05362b0  is invalid pointer
 thd-thread_id=15401
 The manual page at http://www.mysql.com/doc/en/Crashing.html contains
 information that should help you find out what is causing the crash.
 mysqld: my_new.cc:51: int __cxa_pure_virtual(): Assertion `Pure virtual 
 method called. == Aborted' failed.
 Fatal signal 6 while backtracing
 
 Number of processes running now: 0
 070907 07:21:51  mysqld restarted
 070907  7:21:51  InnoDB: Database was not shut down normally!
 
 
 -- 
 MySQL General Mailing List
 For list archives: http://lists.mysql.com/mysql
 To unsubscribe:
 http://lists.mysql.com/[EMAIL PROTECTED]
 

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]