Folks,
We've been running a fairly high-traffic mysql server for some time,
and we've been occasionally seeing messages like this in the logs:
InnoDB: Error: semaphore wait has lasted > 600 seconds
InnoDB: We intentionally crash the server, because it appears to be
hung.
040309 18:20:36 InnoDB: Assertion failure in thread 2623773248 in file
sync0arr.c line 934
InnoDB: Failing assertion: 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
This is a fairly critical issue for us (the latest crash borked
replication, which is never fun), so if anyone has an answer it would be
most appreciated. I've attached the full relevant log snippet and the
my.cnf for anyone who's willing to take a look.
I'd provide more information, but it has so far been impossible to tell
what's causing it. This server averages >2000 queries per second (25%
of which is inserts and updates, I'd guess), and is also the replication
master for another database. It seems to be random so far.
This system is running on a dual Xeon (2.4GHz + hyperthread) with 4GB of
memory, Two U160 SCSI arrays (configured for 1TB @ RAID1+0), and the
stock RedHat 2.4.20-8smp kernel (the box is pretty much a stock RedHat 9
install).
Again, any insight would be most appreciated.
--ZS
----%<---[ Cut Here for my.cnf ]----%<---------------------
[client]
port = 3306
socket = /tmp/mysql.sock
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer = 1024M
max_allowed_packet = 16M
table_cache = 512
sort_buffer_size = 256M
read_buffer_size = 256M
myisam_sort_buffer_size = 64M
thread_cache = 8
query_cache_size = 128M
thread_concurrency = 8
max_connections = 1000
interactive_timeout = 1800
wait_timeout = 1800
server-id = 1
log-bin
tmpdir = /tmp/
innodb_data_home_dir = /usr/local/mysql/var/
# 1560GB of DB tablespace
innodb_data_file_path =
aa:1G;i2/ab:1G;ac:1G;i2/ad:1G;ae:1G;i2/af:1G;ag:1G;i2/ah:1G;ai:1G;i2/aj:1G;ak:1G;i2/al:1G;am:1G;i2/an:1G;ao:1G;i2/ap:1G;aq:1G;i2/ar:1G;as:1G;i2/at:1G;au:1G;i2/av:1G;aw:1G;i2/ax:1G;ay:1G;i2/az:1G;aA:1G;i2/aB:1G;aC:1G;i2/aD:1G;aE:1G;i2/aF:1G;aG:1G;i2/aH:1G;aI:1G;i2/aJ:1G;aK:1G;i2/aL:1G;aM:1G;i2/aN:1G;aO:1G;i2/aP:1G;aQ:1G;i2/aR:1G;aS:1G;i2/aT:1G;aU:1G;i2/aV:1G;aW:1G;i2/aX:1G;aY:1G;i2/aZ:1G;a0:1G;i2/a1:1G;a2:1G;i2/a3:1G;a4:1G;i2/a5:1G;a6:1G;i2/a7:1G;a8:1G;i2/a9:1G;ba:1G;i2/bb:1G;bc:1G;i2/bd:1G;be:1G;i2/bf:1G;bg:1G;i2/bh:1G;bi:1G;i2/bj:1G;bk:1G;i2/bl:1G;bm:1G;i2/bn:1G;bo:1G;i2/bp:1G;bq:1G;i2/br:1G;bs:1G;i2/bt:1G;bu:1G;i2/bv:1G;bw:1G;i2/bx:1G;by:1G;i2/bz:1G;bA:1G;i2/bB:1G;bC:1G;i2/bD:1G;bE:1G;i2/bF:1G;bG:1G;i2/bH:1G;bI:1G;i2/bJ:1G;bK:1G;i2/bL:1G;bM:1G;i2/bN:1G;bO:1G;i2/bP:1G;bQ:1G;i2/bR:1G;bS:1G;i2/bT:1G;bU:1G;i2/bV:1G;bW:1G;i2/bX:1G;bY:1G;i2/bZ:1G;b0:1G;i2/b1:1G;b2:1G;i2/b3:1G;b4:1G;i2/b5:1G;b6:1G;i2/b7:1G;b8:1G;i2/b9:1G;ca:1G;i2/cb:1G;cc:1G;i2/cd:1G;ce:1G;i2/cf:1G;cg:1G;i2/ch:1G;ci:1G;i2/cj:1G;ck:1G;i2/cl:1G;cm:1G;i2/cn:1G;co:1G;i2/cp:1G;cq:1G;i2/cr:1G;cs:1G;i2/ct:1G;cu:1G;i2/cv:1G;cw:1G;i2/cx:1G;cy:1G;i2/cz:1G;cA:1G;i2/cB:1G;cC:1G;i2/cD:1G;cE:1G;i2/cF:1G;cG:1G;i2/cH:1G;cI:1G;i2/cJ:1G;cK:1G;i2/cL:1G;cM:1G;i2/cN:1G;cO:1G;i2/cP:1G;cQ:1G;i2/cR:1G;cS:1G;i2/cT:1G;cU:1G;i2/cV:1G;cW:1G;i2/cX:1G;cY:1G;i2/cZ:1G;c0:1G;i2/c1:1G;c2:1G;i2/c3:1G;c4:1G;i2/c5:1G;c6:1G;i2/c7:1G;c8:1G;i2/c9:1G;da:1G;i2/db:1G;dc:1G;i2/dd:1G;de:1G;i2/df:1G;dg:1G;i2/dh:1G;di:1G;i2/dj:1G;dk:1G;i2/dl:1G;dm:1G;i2/dn:1G;do:1G;i2/dp:1G;dq:1G;i2/dr:1G;ds:1G;i2/dt:1G;du:1G;i2/dv:1G;dw:1G;i2/dx:1G;dy:1G;i2/dz:1G;dA:1G;i2/dB:1G;dC:1G;i2/dD:1G;dE:1G;i2/dF:1G;dG:1G;i2/dH:1G;dI:1G;i2/dJ:1G;dK:1G;i2/dL:1G;dM:1G;i2/dN:1G;dO:1G;i2/dP:1G;dQ:1G;i2/dR:1G;dS:1G;i2/dT:1G;dU:1G;i2/dV:1G;dW:1G;i2/dX:1G;dY:1G;i2/dZ:1G;d0:1G;i2/d1:1G;d2:1G;i2/d3:1G;d4:1G;i2/d5:1G;d6:1G;i2/d7:1G;d8:1G;i2/d9:1G;ea:1G;i2/eb:1G;ec:1G;i2/ed:1G;ee:1G;i2/ef:1G;eg:1G;i2/eh:1G;ei:1G;i2/ej:1G;ek:1G;i2/el:1G;em:1G;i2/en:1G;eo:1G;i2/ep:1G;eq:1G;i2/er:1G;es:1G;i2/et:1G;eu:1G;i2/ev:1G;ew:1G;i2/ex:1G;ey:1G;i2/ez:1G;eA:1G;i2/eB:1G;eC:1G;i2/eD:1G;eE:1G;i2/eF:1G;eG:1G;i2/eH:1G;eI:1G;i2/eJ:1G;eK:1G;i2/eL:1G;eM:1G;i2/eN:1G;eO:1G;i2/eP:1G;eQ:1G;i2/eR:1G;eS:1G;i2/eT:1G;eU:1G;i2/eV:1G;eW:1G;i2/eX:1G;eY:1G;i2/eZ:1G;e0:1G;i2/e1:1G;e2:1G;i2/e3:1G;e4:1G;i2/e5:1G;e6:1G;i2/e7:1G;e8:1G;i2/e9:1G;fa:1G;i2/fb:1G;fc:1G;i2/fd:1G;fe:1G;i2/ff:1G;fg:1G;i2/fh:1G;fi:1G;i2/fj:1G;fk:1G;i2/fl:1G;fm:1G;i2/fn:1G;fo:1G;i2/fp:1G;fq:1G;i2/fr:1G;fs:1G;i2/ft:1G;fu:1G;i2/fv:1G;fw:1G;i2/fx:1G;fy:1G;i2/fz:1G;fA:1G;i2/fB:1G;fC:1G;i2/fD:1G;fE:1G;i2/fF:1G;fG:1G;i2/fH:1G;fI:1G;i2/fJ:1G;fK:1G;i2/fL:1G;fM:1G;i2/fN:1G;fO:1G;i2/fP:1G;fQ:1G;i2/fR:1G;fS:1G;i2/fT:1G;fU:1G;i2/fV:1G;fW:1G;i2/fX:1G;fY:1G;i2/fZ:1G;f0:1G;i2/f1:1G;f2:1G;i2/f3:1G;f4:1G;i2/f5:1G;f6:1G;i2/f7:1G;f8:1G;i2/f9:1G;ga:1G;i2/gb:1G;gc:1G;i2/gd:1G;ge:1G;i2/gf:1G;gg:1G;i2/gh:1G;gi:1G;i2/gj:1G;gk:1G;i2/gl:1G;gm:1G;i2/gn:1G;go:1G;i2/gp:1G;gq:1G;i2/gr:1G;gs:1G;i2/gt:1G;gu:1G;i2/gv:1G;gw:1G;i2/gx:1G;gy:1G;i2/gz:1G;gA:1G;i2/gB:1G
innodb_log_group_home_dir = /usr/local/mysql/var/
innodb_log_arch_dir = /usr/local/mysql/var/
innodb_buffer_pool_size = 1024M
innodb_additional_mem_pool_size = 128M
innodb_log_file_size = 256M
innodb_log_buffer_size = 64M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50
[mysqldump]
quick
max_allowed_packet = 64M
[mysql]
no-auto-rehash
[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
-----------------------------------------------------------
--
Steven M. Doyle, Evil Programmer
[EMAIL PROTECTED]
(and still Keeper of the Bamboo Stick!) [EMAIL PROTECTED]
Don't the hours grow shorter as the days go by
You never get to stop and open your eyes
One minute you're waiting for the sky to fall
The next you're dazzled by the beauty of it all
(Barenaked Ladies / Lovers in a Dangerous Time)
040305 21:15:01 mysqld started
040305 21:15:02 InnoDB: Started
/usr/local/mysql/libexec/mysqld: ready for connections.
Version: '4.0.17-log' socket: '/tmp/mysql.sock' port: 3306
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 242.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 274.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 306.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 338.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 265.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 264.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 264.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 370.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 297.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 296.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 296.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 402.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 329.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 328.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 328.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 434.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 361.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 360.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 360.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 466.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 393.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 392.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 392.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 498.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 425.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 424.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 424.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 530.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 457.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 456.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 456.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 562.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 489.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 488.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 488.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 594.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 2671042368 has waited at ../../innobase/include/trx0sys.ic line 99 for 521.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Warning: a long semaphore wait:
--Thread 1103296448 has waited at trx0trx.c line 600 for 520.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: Warning: a long semaphore wait:
--Thread 2615384768 has waited at srv0srv.c line 2589 for 520.00 seconds the semaphore:
Mutex at 4a131168 created file srv0srv.c line 1653, lock var 1
Last time reserved in file trx0trx.c line 600, waiters flag 1
InnoDB: ###### Starts InnoDB Monitor for 30 secs to print diagnostic info:
InnoDB: ###### Diagnostic info printed to the standard output
InnoDB: Warning: a long semaphore wait:
--Thread 2694511424 has waited at ../../innobase/include/trx0sys.ic line 99 for 626.00
seconds the semaphore:
X-lock on RW-latch at 94635b64 created in file buf0buf.c line 444
a writer (thread id 2694511424) has reserved it in mode wait exclusive
number of readers 1, waiters flag 1
Last time read locked in file buf0flu.c line 469
Last time write locked in file ../../innobase/include/trx0sys.ic line 99
InnoDB: Error: semaphore wait has lasted > 600 seconds
InnoDB: We intentionally crash the server, because it appears to be hung.
040309 18:20:36 InnoDB: Assertion failure in thread 2623773248 in file sync0arr.c
line 934
InnoDB: Failing assertion: 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Send a detailed bug report to [EMAIL PROTECTED]
mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=1073741824
read_buffer_size=268431360
max_used_connections=110
max_connections=1000
threads_connected=93
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 1044568 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd=(nil)
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x9c6390fc, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x80f74ea
0x40043618
(nil)
0x4003e2b6
0x420de407
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://www.mysql.com/doc/en/Using_stack_trace.html and follow instructions
on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.
Number of processes running now: 0
040309 18:20:38 mysqld restarted
040309 18:20:44 InnoDB: Database was not shut down normally.
InnoDB: Starting recovery from log files...
InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 12 2044636752
InnoDB: Doing recovery: scanned up to log sequence number 12 2049879552
InnoDB: Doing recovery: scanned up to log sequence number 12 2055122432
InnoDB: Doing recovery: scanned up to log sequence number 12 2060365312
InnoDB: Doing recovery: scanned up to log sequence number 12 2065608192
InnoDB: Doing recovery: scanned up to log sequence number 12 2070851072
InnoDB: Doing recovery: scanned up to log sequence number 12 2076093952
InnoDB: Doing recovery: scanned up to log sequence number 12 2081336832
InnoDB: Doing recovery: scanned up to log sequence number 12 2086579712
InnoDB: Doing recovery: scanned up to log sequence number 12 2091822592
InnoDB: Doing recovery: scanned up to log sequence number 12 2097065472
InnoDB: Doing recovery: scanned up to log sequence number 12 2102308352
InnoDB: Doing recovery: scanned up to log sequence number 12 2107551232
InnoDB: Doing recovery: scanned up to log sequence number 12 2112794112
InnoDB: Doing recovery: scanned up to log sequence number 12 2118036992
InnoDB: Doing recovery: scanned up to log sequence number 12 2123279872
InnoDB: Doing recovery: scanned up to log sequence number 12 2128522752
InnoDB: Doing recovery: scanned up to log sequence number 12 2133518516
040309 18:20:51 InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: In a MySQL replication slave the last master binlog file
InnoDB: position 0 757943028, file name rdb-bin.010
InnoDB: Last MySQL binlog file position 0 111773384, file name ./bdb-bin.008
040309 18:29:26 InnoDB: Flushing modified pages from the buffer pool...
040309 18:29:41 InnoDB: Started
/usr/local/mysql/libexec/mysqld: ready for connections.
Version: '4.0.17-log' socket: '/tmp/mysql.sock' port: 3306
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
